Compare commits

...
Author SHA1 Message Date
windmill-internal-app[bot] 4ad1f82441 docs(changelog): add new entries from changelog 2026-06-05 16:51:42 +00:00
fad1a549d9 feat(otel): connect jobs to the inbound distributed trace (#9456)
* feat(otel): propagate inbound W3C traceparent to job spans

Capture the inbound traceparent header at the run endpoints
(WebhookArgs::to_args_from_format) into a reserved _wm_traceparent arg key
(gated on OTEL_TRACING_ENABLED), riding the args jsonb like
_ENTRYPOINT_OVERRIDE. At pickup, create_span_with_name attaches a span link
from the job's worker span to the originating distributed trace, so a job
triggered by an instrumented service is connected to the caller's trace
while keeping its UUID-derived trace id (trace-by-job-id unaffected).

The link/parse logic lives in the EE otel modules; this OSS side only
captures the header and calls the (no-op outside EE) hook. Companion EE PR
required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref to inbound-trace-propagation EE branch

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(agents): don't attribute work to specific customers in repo content

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(otel): relocate job + script spans into the inbound trace

Builds on the captured _wm_traceparent: the worker job span is re-parented on
the inbound caller context, the script subprocess's TRACEPARENT env is the
inbound context (so its spans join the caller's trace), and the context is
propagated to flow steps so the whole flow relocates. Carried to the worker via
a new LogContext.inbound_traceparent field. Non-inbound jobs are unchanged.

Adds a relocation integration test. Companion EE PR required.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref to inbound-trace-propagation relocate commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(otel): harden inbound traceparent capture

Address review feedback:
- strip any caller-supplied _wm_traceparent from args/extra before stashing the
  header-captured value, so the reserved key is Windmill-controlled only
- valid_w3c_traceparent: reject version ff and require lowercase hex, so we don't
  forward an inbound header that downstream OTel parsers would reject
- clarify that the capture helper does not validate the W3C format (done at use)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a

This commit updates the EE repository reference after PR #604 was merged in windmill-ee-private.

Previous ee-repo-ref: 8fc04fb105dc49769205f7174d551a0d134d1bec

New ee-repo-ref: 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-06-05 16:50:07 +00:00
Ruben Fiszelandrubenfiszel 3887bf67dc chore(main): release 1.718.0 (#9450)
* chore(main): release 1.718.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-05 14:55:10 +00:00
90677872f6 fix: distinguish canceled jobs in runs (#9452)
* fix: distinguish canceled jobs in runs

* fix: order status=failure|canceled by completed_at to use partial index

The new `status` query param replaced the legacy `success=false` filter on
the Runs page, but the ORDER BY switch in list_completed_jobs_query only
flipped to v2_job_completed.completed_at for success==Some(false). With
status=failure|canceled (and success=None), the query fell back to ordering
by v2_job.created_at, which the partial index
ix_v2_job_completed_failure_workspace (workspace_id, completed_at DESC WHERE
status IN ('failure','canceled')) cannot serve.

EXPLAIN ANALYZE on 500k rows (1% failure/canceled): ordering by completed_at
uses the partial index (~150 buffers, 0.3ms); ordering by created_at scans
the v2_job created_at index and probes/discards 99% of rows via the join
(~49k buffers, 31ms). Switch the ordering to completed_at for
failure/canceled so the partial index serves both filtering and ordering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: trim order-by regression test to the failure/canceled case

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: only treat canceled as a terminal status icon for completed jobs

Guard the canceled branch in JobStatusIcon and getJobStatusKind with
`'success' in job` so a job that is still running while being canceled keeps
its running icon/favicon until it completes, instead of immediately showing
the gray Canceled state. Also clarify the openapi `status` param is an exact
match (status=success excludes skipped, unlike success=true).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 14:50:07 +00:00
Ruben FiszelandClaude Opus 4.8 7590b28108 feat(sandbox): pull/extract images with crane instead of podman (#9455)
* feat(sandbox): pull/extract images with crane instead of podman (+ add to image)

The sandboxed container runtime (`# sandbox <image>`) only ever pulls + flattens an
image (nsjail does the run), so a full container engine is overkill — and podman was
never actually in any Dockerfile, so the merged feature couldn't run in the shipped
image. Switch to crane (google/go-containerregistry): a single ~25MB static binary,
no daemon/store/root/privileged.

- docker_v2.rs: crane export -> flattened rootfs tar, crane config -> OCI config,
  crane digest -> content-addressed rootfs+config cache (cross-job dedup + automatic
  freshness), crane manifest -> pre-download size guard. DOCKER_CONFIG authfile dir.
  Cache eviction prunes the rootfs-tar cache by mtime (LRU). Pull policy honored via a
  ref->digest cache (missing/never reuse without a registry hit).
- Dockerfile + docker/DockerfileSlim{,Ee}: install the crane binary (Full/FullEe and
  the EE image inherit it via FROM the base image).
- docs + UI text + instance-setting descriptions updated (download size is compressed;
  cache is the rootfs-tar cache).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): address CI review — digest-pinned fetch, size cap on every job, eviction race

Codex P1s:
- Fetch by the resolved digest (name@digest), not the mutable tag, so content can't
  diverge from the digest the cache is keyed under if a tag moves mid-fetch.
- Enforce the size cap on EVERY job via a cached {digest}.size sidecar (no registry call
  on cache reuse), so lowering the limit rejects already-cached oversized images.
- Eviction race: hardlink the cache tar into the job dir before tar -xf (pins the inode
  against concurrent eviction) and re-fetch if it was evicted first.
Claude P2s: atomic config sidecar (tmp+rename) + tolerate torn parse; soften the LRU
comment (mtime = creation order); sweep orphaned *.tmp.* and .size on eviction.
+digest_key/ref_key unit tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): P1 cross-fs cache staging (EXDEV), Dockerfile arch fail-fast

CI re-review (Claude + Codex P1): the eviction-race hardlink crosses filesystems in the
shipped deployments — the cache is its own volume (/tmp/windmill/cache) while the job dir
is on the container fs — so hard_link returns EXDEV (not NotFound) and every sandbox job
fails. Fall back to tokio::fs::copy on a non-NotFound link error; copy reads through the
source inode so it still survives a concurrent eviction.
Also: Dockerfiles fail fast with a clear error on an unsupported arch instead of building
a 404 crane URL; ref->digest file written via tmp+rename (no torn read under missing/never).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(sandbox): say 'oldest by creation time' not 'LRU' for cache eviction

Codex P2: the code evicts by tar creation time (cache hits don't touch mtime), so the
user-facing docs + instance-setting text shouldn't claim true LRU.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 13:52:55 +00:00
Ruben FiszelandClaude Opus 4.8 9a609bf08a feat: make C# dotnet target framework configurable via DOTNET_TARGET_FRAMEWORK (#9454)
* feat: make C# dotnet target framework configurable via DOTNET_TARGET_FRAMEWORK

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: include dotnet target framework in C# binary cache key

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 09:43:40 +00:00
Ruben FiszelandClaude Opus 4.8 1727271e19 feat: sandboxed daemonless container runtime via '# sandbox <image>' (#9453)
* feat: add sandboxed docker v2 runtime via '# docker <image>'

Run a container image as a subprogram of the job's own nsjail sandbox:
extract the image rootfs with podman (rootless) and run it chrooted inside the
job's nsjail, so the container inherits the job's confinement and is safe under
nsjail / for untrusted code. Selected by '# docker <image>'; a bare '# docker'
keeps the v1 (dind) path untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: default to daemonless docker (drop dind from compose, allow docker on cloud)

docker-compose no longer ships the dind sidecar (v2 is daemonless: podman + nsjail
in the worker); removed the dind service, DOCKER_HOST env, depends_on and volume.
Removed the language-picker guard that blocked Docker scripts on the multi-tenant
platform, now that v2 makes docker safe to run sandboxed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: select sandboxed container via # sandbox <image>; add pull policy + size guards

- Surface moved from '# docker <image>' to '# sandbox <image>' (groups under the
  sandbox annotation; '# docker' stays v1-only, '# sandbox' stays nsjail-bash).
- SANDBOX_IMAGE_PULL_POLICY (default 'newer') so moving tags don't go stale.
- SANDBOX_IMAGE_MAX_SIZE_MB rejects oversized images before extraction.
- SANDBOX_IMAGE_CACHE_MAX_MB best-effort LRU eviction of podman's image store.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): support # volume, honor nsjail tmp instance settings, v2 docker template

- Thread shared_mount into the sandbox container nsjail config so '# volume' mounts
  (and the same-worker /tmp/shared folder) apply inside the container.
- Use resolve_nsjail_tmp_mount_block for the container's /tmp so it honors the same
  nsjail_tmp_backing / nsjail_tmpfs_size_mb instance settings as other nsjail jobs.
- docker-compose comment + the editor's Docker template now use '# sandbox <image>'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): make image size/cache/pull-policy UI instance settings

Convert SANDBOX_IMAGE_* from worker env vars to DB-backed instance settings
(sandbox_image_max_size_mb, sandbox_image_cache_max_mb, sandbox_image_pull_policy),
hot-reloaded via the same mechanism as nsjail_tmpfs_size_mb and configurable in
#superadmin-settings. No worker restart needed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(sandbox): windmill-managed registry — default registry + private auth

Two new instance settings:
- sandbox_image_default_registry: prepended to unqualified image refs (alpine ->
  <registry>/alpine); fully-qualified refs untouched.
- sandbox_registry_auth: docker/podman auth.json blob written to a per-job authfile
  (0600, removed with the job) and passed to podman --authfile for private registries.
Both hot-reloaded and configurable in #superadmin-settings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): protobuf-safe proto_str escaper, atomic 0600 authfile, registry tests

Addresses local-review P2s: proto_str now emits valid protobuf octal escapes for
control/non-ASCII bytes (not Rust \u{..} that nsjail would reject); the registry
authfile is created 0600 atomically (no world-readable window); add a
registry_qualified table test + a non-ASCII proto_str case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): P0 — deliver image env via nsjail envar:, never the launcher process env

CI review (P0): the image's OCI Env (attacker-controlled keys+values) was applied to
the nsjail launcher process via .envs(), so a hostile image could set LD_PRELOAD/
LD_LIBRARY_PATH/LD_AUDIT on nsjail itself and execute code as the worker outside the
jail. Now the image env is rendered as proto-escaped 'envar:' directives (child-only)
and nsjail's process env carries only windmill-trusted keys (reserved vars + proxy).
Also: warn instead of silently bypassing the size guard on inspect failure; reset the
eviction guard via a Drop guard (no stuck flag on panic/early-return). +render_envars test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): P0 symlink-write escape via rootfs script; P1 redact registry-auth logging

CI review:
- P0 (Codex): the body was written into the image-controlled rootfs as
  .windmill_docker_main.sh via write_file (follows symlinks) — a hostile image could
  plant that path as a symlink to a host file and capture the worker's write before
  nsjail starts. Now the body is passed straight to 'sh -c <body> sh <args>'; no file
  is written into the rootfs at all.
- P1 (Codex): sandbox_registry_auth flowed through the generic setting loader which
  logs the value (raw auth.json credentials). Replaced with a secret-aware reload that
  loads directly and logs only a redacted 'configured=' message.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(sandbox): redact sandbox_registry_auth in instance-settings write log too

The settings API also logs 'Set global setting <key> to <value>' via format_setting_value;
add sandbox_registry_auth to SENSITIVE_SETTINGS so the credential is redacted there as
well as on reload.

* fix(sandbox): don't silently disable cache eviction on podman images parse error

Re-review (cubic/Claude P2): serde_json::from_slice(...).unwrap_or_default() meant any
parse hiccup (e.g. podman omitting Size/Created via omitempty for a zero value, or
schema drift) silently degraded to an empty Vec and disabled eviction with no log.
Now Size/Created are #[serde(default)] (a missing omitempty key -> 0, not a whole-array
parse failure) and a real parse error warns + breaks instead of being swallowed.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 08:35:51 +00:00
fb175e1c9d fix ee repo ref dynamic oauth urls (#9451)
* ee repo ref

* fix(ee-ref): pin to EE commit that includes read_only create_session_token fix

The previous pin (f7a83d9) carried only the connect_config_template change and
dropped Ruben's read_only=false fix (EE 3742e06). CE #9371 made
create_session_token require 6 args, so the EE overlay fails check_ee_full with
an arity error without it. Bump the pin to 9be38de, which includes both fixes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to fb106b89cdf4088b004dac6062adb029f3923887

This commit updates the EE repository reference after PR #603 was merged in windmill-ee-private.

Previous ee-repo-ref: 9be38def879f702cd0b134d9e71bbb17fbb9cfa4

New ee-repo-ref: fb106b89cdf4088b004dac6062adb029f3923887

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-06-05 10:07:01 +02:00
Ruben Fiszel fee23a5185 threat_model v0 2026-06-05 01:00:21 +00:00
00a96b82f3 add databricks icon (#9445)
Adds DatabricksIcon.svelte (brand mark, #FF3621) and registers it under
`databricks` in the shared APP_TO_ICON_COMPONENT map, so both the app and
hub frontends pick it up for the new Databricks hub integration.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-06-04 19:02:51 +00:00
hugocasaandClaude Opus 4.8 dad2bd0b93 add adobe acrobat sign icon (#9447)
Adds AdobeAcrobatSignIcon.svelte and registers `adobe_acrobat_sign` in
APP_TO_ICON_COMPONENT, for the Adobe Acrobat Sign hub integration
(windmill-labs/windmill-integrations#143).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 19:01:57 +00:00
hugocasaandClaude Opus 4.8 93a74f229a oauth: add ServiceNow + make per-instance OAuth providers registry-driven (#9449)
* oauth: add ServiceNow provider; make per-instance OAuth registry-driven

ServiceNow's OAuth endpoints are per-instance
(https://<instance>.service-now.com/oauth_auth.do + /oauth_token.do), like
Snowflake's. Rather than add another bespoke special-case, generalize:
a registry entry may carry a `connect_config_template` (label/placeholder/
help_url + {instance}-templated auth_url/token_url + req_body_auth +
optional extra_params_key/strip_suffix). The instance-settings UI renders
one generic instance-name input for any such provider and substitutes
{instance} to build the per-client connect_config — a new per-instance
provider needs only a JSON entry, no frontend code.

- oauth_connect.json: servicenow + snowflake_oauth now carry a
  connect_config_template (snowflake keeps its account_identifier
  extra_params key for backward compatibility).
- windmill-oauth: add the ConnectConfigTemplate struct (frontend-only
  metadata; the backend's existing connect_config override resolves the
  concrete URLs generically — no other backend change).
- AuthSettings/InstanceSettings: replace the Snowflake + ServiceNow
  special-cases with one registry-driven path (instanceInputs map,
  setupTemplatedOauthUrls, loadInstanceInputs); per-instance providers are
  derived from the registry for the builtins list + dropdown.

Pairs with windmill-integrations#139 (ServiceNow hub integration).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: point ee-repo-ref at servicenow-oauth EE branch (revert at merge)

Temporary CI pointer so check_ee_full / cargo_test build against the EE
slack-literal fix (windmill-ee-private#602). Revert to a pinned SHA once
that EE PR is merged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:58:47 +00:00
hugocasaandClaude Opus 4.8 eb55356018 add wiz icon (#9448)
Wiz star logomark (brand blue #0254EC) for the shared icon map
(APP_TO_ICON_COMPONENT), for windmill-integrations#144.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:58:29 +00:00
hugocasaandClaude Opus 4.8 f2f0812a04 feat(flows): opt-in to include the stopping step's result in early-stop errors (#9446)
* feat(flows): early stop can include the stopping step's result in the raised error

When a step uses Early Stop with "Raise an error message if stopped", the
flow result was entirely replaced with a static error object
({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the
stopping step's own output. This made it impossible to stop+fail a flow
while preserving the data the step produced (e.g. an API that returns
HTTP 200 with a userErrors payload).

Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on
the raise-error path, the raised payload becomes
{"error": {...}, "result": <step result>} instead of dropping the result.
Default is false, so existing behavior is unchanged. The option is threaded
through the worker's stop-after-if handling (including stop_after_all_iters_if
for loops/branchall) and exposed in the flow editor's Early Stop panel.

Fixes WIN-2012

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(flows): cover early-stop error_include_result payload shaping

Add a regression test asserting that a step using Early Stop with a raised
error message and error_include_result=true fails the flow while preserving
the step output as {"error": {..}, "result": <step result>}, and that with
the flag off the result is the bare {"error": {..}} object.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(flows): nest early-stop step result inside the error object

Embed the stopping step's result under `error.result` rather than as a
top-level sibling of `error`. This keeps the flow result shape as
`{ "error": { .. } }` — identical to a normal error — so consumers that
key off the top-level shape (single `error` key) keep working, while the
data is still preserved for those that look inside the error object.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(flows): always include the stopping step's result in early-stop errors

Drop the opt-in `error_include_result` gate. Since the step result is nested
inside the error object (`error.result`), the top-level result shape stays
`{ "error": .. }` — identical to a normal error — so consumers that detect or
parse failures by the top-level shape are unaffected. Gating it added schema
surface, plumbing, and a UI toggle for no real compatibility benefit.

Now, whenever a step early-stops with a raised error message, the flow fails
and the raised error embeds the stopping step's own result under
`error.result` (aggregated iteration results for loops/branchall). This
reverts the `StopAfterIf.error_include_result` field, its threading, the
OpenAPI/generated-client surface, and the editor toggle; the "Raise an error
message" tooltip now notes that the step result is included.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(flows): gate early-stop result inclusion behind opt-in flag

Re-introduce the per-step `error_include_result` flag (default off) instead
of always embedding the step result. Although nesting the result under
`error.result` keeps the result *shape* backward-compatible, it does not
address data exposure: a failed flow's result is propagated to synchronous
webhook callers, the flow's failure module, and the workspace/global error
handler (commonly a Slack/email/outbound-webhook notifier). Always including
the step output would surface previously-redacted intermediate data to all of
those sinks for every existing error-stop flow.

Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and
only embeds `error.result` when the flow author explicitly opts in, matching
the original issue's intent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(flows): omit error_include_result when false; refresh generated prompts

- Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result`
  so serialized flows are byte-identical when the flag is off. Fixes the
  `flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing
  flows.
- Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts`
  for the new OpenFlow `error_include_result` property. Fixes check-freshness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(flows): cover error_include_result for the loop "stop after all iters" path

Add a regression test for the stop_after_all_iters_if branch, where `nresult`
already holds the aggregated iteration results — confirming `error.result`
carries each iteration's output (distinct from the per-step fallback path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:57:05 +00:00
Ruben Fiszelandrubenfiszel 24fa61d3c0 chore(main): release 1.717.1 (#9444)
* chore(main): release 1.717.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-04 10:20:28 +00:00
Ruben FiszelandClaude Opus 4.8 f595787409 fix: invalidate relative-import cache when imported script changes (#9443)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 10:09:06 +00:00
Ruben Fiszelandrubenfiszel 6b6c16e6bc chore(main): release 1.717.0 (#9439)
* chore(main): release 1.717.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-04 07:37:15 +00:00
Ruben FiszelandClaude Opus 4.8 b5a6a1eeab fix(cli): push whole raw app instead of treating frontend files as scripts (#9442)
* fix(cli): push whole raw app instead of treating frontend files as scripts

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(cli): shorten raw-app handleFile comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 07:30:56 +00:00
centdixandClaude Opus 4.8 819ba5e150 fix: read latest db draft for scripts/flows in global mode read tool (#9441)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 07:06:05 +00:00
hugocasaandClaude Opus 4.8 468aa230e5 refactor: resolve workspace imports via /f/,/u/ not $f/,$u/ aliases (#9438)
* refactor: resolve workspace imports via /f/,/u/ not $f/,$u/ aliases

Keep the CLI managed tsconfig.wmill.json / `refresh tsconfig` / Deno
import-map QoL from #9378, but re-key it on the existing /f/,/u/ workspace
paths instead of the new $f/,$u/ specifiers. Verified /f/,/u/ resolves in
tsc, Bun, Deno, the in-app ATA editor, and the worker, so the $-prefixed
alias added no value. Drop the $f/,$u/ handling from the parser, dep-map,
deno_executor, bun loaders, ATA, relative_imports and monaco paths; revert
the windmill-parser-wasm-ts bump (1.714.0 -> 1.695.0). Also fold in the
cli/package-lock.json sync for the already-committed pg-gateway dependency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: drop duplicate relative-path check and restore rustfmt formatting

Follow-up cleanups to the previous commit's full-file reverts, which
restored pre-#9378 state that main had since improved:

- relative_imports.ts: remove the redundant duplicate d.startsWith('/')
  (pre-#9378 had it; #9378 had repurposed that line, so main has no dup).
- windmill-parser-ts/src/lib.rs: restore the multi-line new_source_file(...)
  formatting required by backend/rustfmt.toml (the single-line revert would
  fail `cargo fmt --check`). Now differs from main only by the $f//$u/ removal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 17:27:24 +00:00
GuilhemandClaude Opus 4.8 e4e0984e55 feat: let flow AI chat create and edit sticky notes (#9412)
* feat: let flow AI chat create and edit sticky notes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: strengthen flow AI guidance to prefer groups for organizing flows

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: harden flow note validation (validate position/size, document color default and group acceptance)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: make AI-created free notes draggable by seeding default position and size

Free notes need explicit geometry to be draggable/resizable in the editor; UI-created notes always set position+size but agent-created notes omitted both, so they couldn't be moved until resized. Seed defaults in validateFlowNotes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 17:27:01 +00:00
Ruben Fiszelandrubenfiszel d537c82c4f chore(main): release 1.716.0 (#9430)
* chore(main): release 1.716.0

* Apply automatic changes

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-03 14:01:02 +00:00
Ruben FiszelandClaude Opus 4.8 e3acb7bbd9 break main.ts <-> utils.ts circular import causing TDZ crash (#9436)
`cli/src/utils/utils.ts` imported `VERSION` from `cli/src/main.ts`, while
`main.ts` transitively imports `utils.ts` (via `workspace.ts`). When a module
load order entered the graph through `workspace.ts -> utils.ts -> main.ts`,
`main.ts`'s top-level command tree ran while `workspace.ts` was still
mid-initialization, so the `workspace` binding was still in its temporal dead
zone at `.command("workspace", workspace)`:

    ReferenceError: Cannot access 'workspace' before initialization

This surfaced as 56 failing CLI tests on Windows CI (the Windows runner's test
module-load order triggers the bad path; it reproduces on any platform via
`bun -e 'await import("./src/commands/workspace/workspace.ts")'`).

Move `VERSION` to `cli/src/core/constants.ts` (already the "minimal imports"
module), re-export it from `main.ts` for backwards compatibility, and have
`utils.ts` read it from `constants.ts` — eliminating the cycle. Release tooling
(`.github/change-versions*.sh`) is updated to rewrite the `VERSION` line in its
new location.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 13:48:26 +00:00
Ruben FiszelandClaude Opus 4.8 073857ac0a fix(apps): relock no longer reverts raw app to a stale version (#9432)
When a dependency job for an app is triggered by a relative/workspace
import (e.g. an imported script was updated), handle_app_dependency_job
re-appended the version captured at job-creation time to the versions
array. On a git-sync/CLI push that deploys both the imported script and
the importing app in the same batch, the script's dependency job
snapshots the app's old version; the app push then creates a newer
version (uploading its bundle against that new version); finally the
relock runs and re-appends the old version, making it latest again.

For raw apps this is fatal: bundle_secret is computed from the latest
version, so the served HTML requests /apps_u/get_data/v/<secret>.{js,css}
for a version that has no stored bundle -> 404 and a white screen.
Manually redeploying fixes it until the next merge re-triggers the revert.

Two changes:
- Re-query the current latest version to relock (mirrors the flow
  dependency handler, #8673), so we don't lock a stale snapshot.
- Guard the re-publish append with `versions[array_upper(...)] = $1` so
  it is a single atomic, never-demoting statement: it can only re-append
  the version that is already latest, never revert to an older one. A
  relock never creates a new app_version, so there is never a version to
  legitimately promote here.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 13:21:54 +00:00
Guilhem 732dc0f617 apply COEP headers to dev static files for raw app editor (#9433) 2026-06-03 13:21:09 +00:00
centdix 79178f6f5a feat: use metadata model for small AI tasks (#9431) 2026-06-03 12:13:14 +00:00
hugocasaandClaude Opus 4.8 220cd35cf7 feat: support $f/ and $u/ import path aliases for scripts (#9378)
* feat: support $f/ and $u/ import path aliases for scripts

$f/ and $u/ are local-friendly aliases for the absolute workspace
import paths /f/ and /u/. Unlike the /-prefixed form (which local tools
treat as a filesystem-root path), the $-prefixed form is a bare specifier
that can be remapped via tsconfig paths / Deno import maps, so the same
import resolves on the Windmill worker and in a local editor.

- worker: recognize $f//$u/ in the Deno import map and both Bun loaders
- dep-map/parser: normalize $f/->f/, $u/->u/ for lockgen + dep tracking
- cli: emit $f/$u path aliases in generated tsconfig.json / deno.json
- frontend: ATA + Monaco paths resolve $f//$u/ type hints in the editor

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): split generated tsconfig into managed + user file with refresh command

Mirror the AGENTS.cli.md/AGENTS.md prompts model for the IDE tsconfig so the
recommended settings can evolve without ever clobbering user customizations:

- tsconfig.wmill.json: wmill-managed, always refreshed, holds recommended
  compilerOptions incl. the $f/$u path aliases (Deno: import_map.wmill.json)
- tsconfig.json: user-owned, created once, just extends the managed file;
  warn (never auto-edit) when an existing one doesn't reference it
- add 'wmill refresh tsconfig'; init generates it unconditionally (no longer
  gated behind resource-type namespace / a bound workspace)
- regenerate CLI guidance docs for the new subcommand

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review on $f/ tsconfig generation

- handle existing deno.jsonc so we don't shadow it with a new deno.json
  (P1 identified by cubic)
- fix the bun-types hint that pointed users at the managed do-not-edit
  tsconfig.wmill.json; tell them to install + re-run 'wmill refresh tsconfig'
- document the .ts-extension-only local-resolution limitation (cross-flavor
  .bun.ts/.deno.ts/.fetch.ts scripts won't resolve in a local editor)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): warn when a project's tsconfig isn't wired to tsconfig.wmill.json

Mirror the prompts freshness check for the managed tsconfig so users with an
existing setup actually discover they're missing $f//$u/ resolution:

- embed a version hash in tsconfig.wmill.json (excludes the env-dependent
  bun-types 'types' entry so it doesn't false-positive)
- add warnIfTsconfigStale to the main.ts freshness hook, gated identically to
  the prompts check (skips init/refresh/help/version). When a tsconfig.json
  exists it warns one line (stderr) if the managed file is missing, not
  referenced via extends, or out of date; silent for non-TS projects.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(cli): make tsconfig setup equivalent to prompts (auto-wire + stale-only)

Unify the two managed-file systems so they behave identically:

- auto-wire an existing unlinked tsconfig.json/deno.json on init/refresh
  (add extends / importMap; merge into an array extends), instead of only
  warning. Parses JSON and falls back to a warning when it can't round-trip
  (JSONC comments, or a conflicting deno imports/importMap) — never corrupts.
- narrow warnIfTsconfigStale to stale-only, gated on the managed file
  existing, exactly like warnIfPromptsStale: it no longer nags about a
  missing or unlinked tsconfig.json, so a deliberately-custom/unlinked setup
  stays silent and a not-yet-initialized project isn't bothered.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): place tsconfig.wmill.json first in extends to preserve user base config

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): migrate legacy tsconfig and require consent for custom configs

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(cli): align prompts wiring to the same consent model

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(cli): bump windmill-parser-wasm-ts to 1.714.0 for $f/ $u/ aliases

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(worker): resolve $f/ and $u/ in deno lock generation

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: narrow relative-imports lock-gen guard to deno import-map failure

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(cli): sync bun.lock with windmill-parser-wasm-ts 1.714.0

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): warn when a custom tsconfig's paths would shadow $f/ $u/ aliases

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 11:58:12 +00:00
Ruben Fiszel b3027c35cb nit version priting 2026-06-03 11:22:50 +00:00
centdix 26b7270418 feat: auto-generate AI session names (#9399) 2026-06-03 10:35:03 +00:00
centdixandClaude Opus 4.8 343368fb5e test: add datatable tool coverage to global AI evals (#9398)
* test: add datatable tool coverage to global ai_evals (stage 0+1)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: add seeded datatable difficulty-ladder global ai_evals (stage 2)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: skipJudge datatable evals and make stringIncludesAnyOf existential

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: make ai_evals datatable mock reflect SQL writes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 10:34:37 +00:00
GuilhemandClaude Opus 4.8 c3d4c6474b gate session fork creation on CE workspace cap (#9411)
* feat: gate workspace fork creation in sessions behind enterprise license

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: gate session fork creation on CE workspace cap, not EE license

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 10:34:16 +00:00
centdix cf5fefb521 feat: add metadata generation model setting (#9418) 2026-06-03 10:33:16 +00:00
Ruben FiszelandClaude Opus 4.8 0ba128afe7 fix(security): scope variable and resource value caches by caller identity (#9427)
The variable and resource value caches (backing
`GET /api/w/{w}/variables/get_value/{path}?allow_cache=true` and
`.../resources/get_value_interpolated/{path}?allow_cache=true`) are consulted
before the per-folder RLS query and store the already-decrypted value. The
resource cache was keyed only by `workspace:path` with no caller identity, so a
cache entry warmed by a privileged peer using `allow_cache=true` could be
returned to a caller with no access to the resource's folder on a cache hit
within the 30s TTL — leaking another folder's decrypted secrets.

Scope both caches to the caller's full authorization identity. The key is now
`auth_identity(authed):workspace:path`, where `auth_identity` is a SHA-256 of the
caller's effective authorization context (email, username, is_admin, is_operator,
sorted groups, sorted folders, sorted scopes) — mirroring
`job_read_access_cache_key`. Email alone is insufficient: the same email can
resolve to different effective permissions via job/owner-scoped tokens, so a
lower-privilege context must not reuse a higher-privilege context's entry.

Job-context resource interpolation is handled correctly: only `$WM_*` contextual
variables are resolved (and only when a `job_id` is present). The interpolation
reports whether the value contains a `$WM_*` placeholder
(`transform_json_value_tracked` + an `AtomicBool`). A value containing one is
job-dependent — even on a no-job read where it's left unresolved — and is never
cached (so a later job read never gets a stale placeholder or another job's
context). Any value without a `$WM_*` placeholder is job-independent and cached
under the identity key, shared across job contexts, so reads carrying a `job_id`
still hit the cache.

BEHAVIOR CHANGE: custom workspace environment variables are no longer interpolated
into resource values via `$NAME` (this was undocumented and prevented caching of
any `$`-prefixed value). Custom envs remain available to scripts/workers as before.
Built-in `$WM_*` contextual variables in resource values are unchanged.

The variable cache previously wrote with an identity-scoped key but read with the
unscoped key, so it never hit (a latent functional bug that happened to be safe).
Aligning the read path enables the cache and makes it identity-scoped by
construction. Secret variables are cached too, but the entry carries the
`is_secret` flag so a cache hit re-runs the per-read side effects a secret read
performs — the EE `variables.decrypt_secret` audit and running-job secret
registration (factored into `audit_decrypt_secret`, shared by both paths).

The unused `invalidate_{variable,resource}_cache` helpers can no longer target
identity-scoped entries; documented the constraint and refreshed the stale
key-format docs on the cache statics.

Tests:
- integration regression for both caches: a folder-scoped user warms the cache via
  allow_cache=true, then a user without folder access is denied (401) and never
  receives the cached value.
- integration regression that variables (secret included) are served from cache.
- integration regression for job context: plain and non-`$WM_` `$`-string resources
  stay cached and are served under a job_id, while a `$WM_*` resource (warmed without
  a job_id) is not cached.
- unit tests for `auth_identity`.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 10:31:45 +00:00
Ruben Fiszelandrubenfiszel 47c96204de chore(main): release 1.715.0 (#9421)
* chore(main): release 1.715.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-03 10:08:16 +00:00
centdixandClaude Opus 4.8 11d1ad9a87 fix: omit temperature for gpt-5+ and o-series models on all providers (#9422)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 09:29:21 +00:00
Ruben FiszelandClaude Opus 4.8 8053266f88 fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url (#9428)
* fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(mcp): clone user_db for oauth2 refresh and drop advisory ids from comments

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(mcp): disable redirects on MCP client to prevent SSRF bypass

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 09:00:43 +00:00
Ruben FiszelandClaude Opus 4.8 7031744a19 fix(nsjail): precompile python stdlib + raise download rlimit_as (#9429)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 08:47:19 +00:00
Ruben FiszelandClaude Opus 4.8 3b2e748daf feat(frontend): add rebuild dependency map button to workspace settings (#9424)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 08:42:39 +00:00
Ruben FiszelandClaude Opus 4.8 7edf3f0212 fix(auth): filter script/flow listings by token scope (GHSA-2ppx-66jv-wpw5) (#9426)
A token scoped to a single script or flow path (e.g.
`scripts:read:f/allowed/*`) could call `GET .../scripts/list_search` (or
`/list`) and receive `path` + full `content` for every script the
underlying user could see — likewise `flows/list_search` leaked the full
flow `value`. Route-level scope checks only validate `domain:action`, and
the listing handlers did no per-row scope filtering, leaking out-of-scope
source/definitions to narrowly-scoped tokens.

Apply `build_scope_path_predicate` (added in #9302 for resources/variables)
to `list_search_scripts`, `list_scripts`, `list_search_flows`, and
`list_flows`, mirroring the resources/variables fix exactly. Unscoped
tokens and tokens whose only scopes are `if_jobs:filter_tags:*` are
unaffected.

Adds integration regression tests (scripts + flows) covering: path-scoped
token sees only in-scope paths, broad `*:read` token still sees all
RLS-visible items, tag-filter-only and unscoped tokens unchanged.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 08:34:43 +00:00
Ruben FiszelandClaude Opus 4.8 89a7a37776 fix(backend): authorize single-job read endpoints by job/flow visibility (#9416)
* fix(backend): authorize single-job read endpoints by job/flow visibility

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(jobs): share read links + cached access checks for run visibility

- Cache the job read-access RLS probe (size-bounded LRU keyed by the caller's
  authz-relevant identity + job id; no TTL since job-side inputs are immutable).
- Inherit visibility along the full parent_job chain so any flow you can see lets
  you read its (deeply nested) steps.
- Share read links: GET /jobs/job_view_token/{id} mints a stateless
  HMAC(workspace_key, job_id) token (only if the caller can read the job); the
  token grants an authenticated member read of that job and its flow subtree via a
  ?view_token query param or X-View-Token header. Run page gains a Share button and
  honors a ?view_token link.
- Denied-but-existing reads now return 403 with guidance to request a share link
  (vs 404 for non-existent), and the run page renders that case with instructions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): address PR review — scope-tag check on mint, constant-time view-token verify

- P1 (Codex): get_job_view_token now enforces the caller's if_jobs:filter_tags
  scope before minting, so a tag-scoped token can't mint a transferable link for a
  job outside its tags. Adds a scoped-token regression test (allowed + denied).
- Constant-time view-token verification (HmacSha256::verify_slice) instead of
  comparing hex strings (Claude/Pi nit).
- get_completed_job_result: an authed reader passing an invalid suspended-secret
  triple now falls through to the normal visibility gate instead of erroring out
  (Claude nit); unauthenticated callers still rejected.
- Length-prefix the read-access cache key fields so no input values can collide.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(api): add job_view_token to openapi spec; use generated client in run page

Addresses Codex review nit: the new GET /jobs/job_view_token/{id} endpoint was
missing from openapi.yaml (the source the frontend client is generated from). Adds
the path + operationId getJobViewToken, and switches the run page's Share button
from a raw fetch to JobService.getJobViewToken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): carry view_token on share-link downloads

Addresses Codex review: download actions bypass the request interceptor that adds
X-View-Token (downloadViaClient uses raw fetch; cookie-mode downloads use plain
hrefs), so a share-link viewer got 403 downloading logs/results/args. Append the
view_token query param to the job download paths (result/logs/args/flow-all-logs)
via a new appendViewToken() helper, covering both client-fetch and href modes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): enforce tag scope in require_job_read_access (view-token use side)

Addresses Codex P1: the view_token use-side bypassed if_jobs:filter_tags on
handlers that don't tag-filter their data query (result_by_id,
get_flow_job_debug_info, get_otel_traces) — a tag-scoped token could use someone
else's valid share token to read out-of-scope job data. Move the tag-scope check
into require_job_read_access (runs before any created_by/view_token/RLS grant), so
it applies uniformly to every gated handler; removes the now-redundant explicit
check in get_job_view_token. Adds a use-side regression test (scoped token + valid
out-of-scope view_token denied on otel/result_by_id; in-scope still allowed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): include workspace in share read link

Addresses Codex P1: the copied share URL omitted the workspace. The token is
signed with the run's workspace key and the logged layout only switches
$workspaceStore when the URL carries workspace=, so a recipient whose persisted
active workspace differs would open the link against the wrong workspace and the
token would fail validation. Pin workspace= alongside view_token in the link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): authorize get_result_maybe get_started branch for queued jobs

Addresses Codex P1: get_completed_job_result_maybe only gated when a completed row
existed; with ?get_started=true a non-reader reached the fallback branch and got
started:true for a running private job. Now fetches created_by and authorizes
(created_by/view_token/RLS, or anonymous for unauth) before disclosing
running-state; a non-existent job still returns started:false (leaks nothing).
Adds a regression test with a queued (no completed row) private job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 22:10:16 +00:00
Ruben FiszelandClaude Opus 4.8 fefa8e438d chore(docker): use nsjail runtime packages instead of dev packages (#9419)
Swap the final-stage nsjail dependencies (libprotobuf-dev,
libnl-route-3-dev) for their runtime-only counterparts (libprotobuf32,
libnl-route-3-200, libnl-3-200) across the root Dockerfile, DockerfileSlim,
and DockerfileSlimEe. nsjail is already compiled in the build stage, so the
final image only needs the runtime shared libraries. This shrinks the
images and reduces CVE scan noise from unused dev packages.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 21:36:47 +00:00
Ruben Fiszelandrubenfiszel 00cd89fff3 chore(main): release 1.714.1 (#9408)
* chore(main): release 1.714.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-02 12:58:41 +00:00
GuilhemandClaude Opus 4.8 2bff250f89 feat(frontend): harmonize diff button placement in script and raw app editors (#9410)
* feat(frontend): harmonize diff button placement across editors

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(frontend): address review nits — drop unused diffDrawer param, fix stale comments

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 12:58:21 +00:00
Ruben FiszelandClaude Opus 4.8 ab2a15b2a8 fix(triggers): prevent Zoom challenge handler from being used as a signing oracle (#9413)
The Zoom URL-validation challenge handler in `handle_challenge_request`
would HMAC-sign any arbitrary `plainToken` and return the result. Since
Zoom webhook verification checks `HMAC-SHA256(secret, "v0:{ts}:{body}")`,
an attacker could craft a `plainToken` in that format to obtain a valid
signature for a forged body, bypassing authentication on a later request.

Unlike the Twitch handler, the Zoom handler verifies no signature on the
challenge request (Zoom's protocol does not include one). Reject any
`plainToken` containing `:` or longer than 128 chars: legitimate Zoom
validation tokens are short random hex strings that never contain colons,
while the exploit requires the colon-bearing `v0:{ts}:{body}` format.

Fixes WIN-2008

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 12:47:45 +00:00
Ruben FiszelandClaude Opus 4.8 9e6559a6f6 fix(nsjail): raise python download fd limit for --compile-bytecode (WIN-2009) (#9414)
#9393 added `--compile-bytecode` to the uv pip install run inside the
python download nsjail. uv spawns a Python interpreter that compiles .py
files with parallelism scaling to the host CPU count, opening many file
descriptors at once. The download nsjail capped `rlimit_nofile` at 64,
which is exhausted on high-core machines, failing every install with
"Failed to bytecode-compile ... Too many open files (os error 24)".

Low-core VMs never hit the cap, so this surfaced only as a regression on
larger workers after upgrading.

Raise `rlimit_nofile` to 10000, matching the runtime configs
(run.python3 / run.ansible) that already use that value.

Fixes WIN-2009

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 12:44:30 +00:00
Ruben FiszelandClaude Opus 4.8 73edebc833 fix(backend): route //native TypeScript previews to native workers (WIN-2007) (#9407)
* fix(backend): route //native TypeScript previews to native workers

Previewing a TypeScript script carrying the `//native` annotation was
pushed with `language = bun` (what the editor sends), so the job was
tagged `bun` and routed to a regular bun worker. A native-mode worker
neither matches the `bun` tag nor accepts a non-native `script_lang`
(worker.rs rejects with "cannot execute non-native job with language
'bun'"), so previewing a `//native` script on a native-only worker setup
failed — even though the deployed version of the same script runs fine
as `bunnative` / tag `nativets`.

`push` now reconciles the preview language with the `//native`
annotation for `JobPayload::Code`, mirroring the deploy-time logic in
`worker_lockfiles`: `bun` + `//native` is promoted to `bunnative` (tag
`nativets`), and `bunnative` without `//native` is demoted back to
`bun`. This makes a preview run exactly like the deployed script would,
and covers every preview entry point (run_preview_script, inline
preview, codebase preview) since they all go through `JobPayload::Code`.

Adds regression tests asserting the queued job's `script_lang`/`tag` for
all four (declared language × annotation) combinations.

Fixes WIN-2007

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(backend): add sqlx cache for preview_native_tag test query

The regression test's `sqlx::query!` for `v2_job` (tag, script_lang) needs
a cached entry so `SQLX_OFFLINE=true` CI compiles it. Adds exactly one new
cache file; no existing (OSS or EE) caches removed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(backend): trim preview native-tag tests to the essentials

Keep the core regression (bun + //native → bunnative/nativets) and the
guard that plain bun previews are unaffected. Drop the two bunnative-
declared cases, which only re-verified the mirrored demote logic and
weren't the reported issue. The shared query is unchanged, so the sqlx
cache stays valid.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 10:23:39 +00:00
Ruben Fiszelandrubenfiszel 2ac198396e chore(main): release 1.714.0 (#9390)
* chore(main): release 1.714.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-02 08:22:19 +00:00
8ad699d27b Refresh slim image runtime packages (#9396)
* Refresh slim runtime packages

* fix(docker): bump pre-baked python to 3.12.12, drop redundant pip/setuptools upgrade

Align the slim images' pre-baked uv-managed Python with the backend default
(PyVAlias::Py312), which previously requested 3.12 while the image baked 3.11.10
— a minor mismatch that made the pre-bake unusable (every default job re-downloaded
3.12 at runtime).

Pinning 3.12.12 (latest 3.12 in uv's list) also drops bundled setuptools entirely
and ships current pip via python-build-standalone, so the explicit
`uv pip install --upgrade pip setuptools` step is now redundant and removed.

Also remove the dead PYTHON_IMAGE ARG from RHEL8/RHEL9 Dockerfiles (declared but
never referenced in any FROM stage).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 08:16:11 +00:00
Ruben FiszelandClaude Opus 4.8 24e3ef27be fix(cli): stop git-sync promotion deploys from dropping triggers/schedules (#9403)
* fix(cli): stop git-sync promotion deploys from dropping triggers/schedules

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: bump git-sync hub script to hub/28261 (windmill-cli 1.713.2)

Points LATEST_GIT_SYNC_SCRIPT_PATH at the republished sync-script-to-git-repo
that pins windmill-cli@1.713.2, which carries the promotion include-derivation
fix in this PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 07:44:04 +00:00
GuilhemandClaude Opus 4.8 30057445f9 avoid crypto.randomUUID in WorkspaceItemDrillPicker (#9405)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 07:16:55 +00:00
Ruben FiszelandClaude Opus 4.8 e356bb1f5d fix(cli): make encryption key push non-interactive-safe + add --skip-reencrypt-on-key-change (#9402)
When encryption_key.yaml changes and is pushed via `wmill sync push`,
pushWorkspaceKey prompted interactively to confirm re-encrypting the
remote secrets with the new key. That prompt ignored `--yes` and had no
TTY guard, so a CI/non-interactive push that included the key would
block (or behave undefinedly) on the prompt.

Thread a key-push options object (non-interactive flag + explicit
re-encryption choice) through pushObj into pushWorkspaceKey:

- Non-interactive (`--yes` or no TTY) and no explicit choice: skip the
  prompt and default to re-encrypting all remote secrets with the new
  key (matches the interactive default), preserving their plaintext
  values.
- New `--skip-reencrypt-on-key-change` flag (and the
  WMILL_NO_REENCRYPT_ON_KEY_CHANGE=true env var for CI) opt out of
  re-encryption — only safe when the remote ciphertexts are already
  encrypted with the new key (e.g. workspace/instance migration).
- Interactive behavior (TTY, no `--yes`) is unchanged.

Regenerates system_prompts for the new option and adds unit tests for
the no-op, re-encrypt-by-default, flag-skip, and env-skip paths.

Fixes WIN-2005

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 07:11:29 +00:00
Ruben FiszelandClaude Opus 4.8 d71d553ba4 Windows build broken by #[cfg] on tokio::select! branch (#9404)
#9400 (WIN-2003) added the ctrl_break() handler as a `#[cfg(windows)]`
branch inside the two Windows-path `tokio::select!` blocks in
shutdown_signal. tokio's `select!` macro does not accept `#[cfg(...)]`
attributes on individual branches, so windmill-common fails to compile
on Windows ("no rules expected this token in macro call").

This slipped through CI because the only job that builds the backend on
Windows is cli-tests.yml's `test-windows`, which triggers only on
`cli/**` changes — #9400 was backend-only.

Fix: define `ctrl_break()` for the whole `not(any(linux, macos))` scope
instead of just `windows`. On Windows it awaits the real CTRL_BREAK
signal; on other non-unix targets it is a never-resolving future, so the
branch is inert there. The select! branches become plain (no per-branch
`#[cfg]`), which the macro accepts.

Verified: the `#[cfg]`-on-branch form reproduces the exact macro error
against tokio 1.46.1, and the fixed form compiles clean.

Fixes WIN-2003 (Windows build regression)

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-02 07:10:53 +00:00
Ruben FiszelandClaude Opus 4.8 e8ad53dae9 fix: resolve username rename failing on apps with runnable deps (#9401)
The instance username-conflict resolver rewrote
workspace_runnable_dependencies.app_path to the new user path before the
app row itself was renamed, violating fk_workspace_runnable_dependencies_app_path.
That FK is ON UPDATE CASCADE, so renaming the app already propagates the new
path; the manual rewrite was redundant and mis-ordered. Any user owning an app
under u/<username>/ with a tracked runnable dependency hit HTTP 500 and could
not have their username conflict resolved.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 06:43:03 +00:00
Ruben FiszelandClaude Opus 4.8 2e1445616a feat: handle CTRL_BREAK_EVENT for graceful shutdown on Windows (#9400)
On Windows, shutdown_signal only registered ctrl_c() (CTRL_C_EVENT).
CTRL_BREAK_EVENT — the default kill signal sent by Nomad's raw_exec
driver on Windows — had no handler, so the worker terminated
immediately without graceful shutdown, interrupting running jobs.

Add a ctrl_break() helper (mirroring the Unix terminate() helper) and
register it as an additional branch in both Windows tokio::select!
blocks in shutdown_signal.

Fixes WIN-2003

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 23:37:35 +00:00
GuilhemandClaude Opus 4.7 de76668c10 fix(frontend): align Monaco editor font size with text-xs (#9161)
* fix(frontend): align Monaco editor font size with text-xs across viewports

* fix(frontend): make placeholder lineHeight reactive to fontSize

* fix(frontend): align GraphQL schema viewer font size with text-xs

The read-only GraphQL schema viewer was the lone Monaco instance still
inheriting Monaco's 14px default. Wire it through editorFontSize like
the other editors so it stays in sync with text-xs across viewports.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 23:29:19 +00:00
hugocasaandClaude Opus 4.8 ba0e4c8280 oauth: add salesforce provider (#9380)
* oauth: add salesforce provider

Register Salesforce OAuth (Authorization Code) for Windmill resource connect.
Production uses login.salesforce.com; the sandbox block points at
test.salesforce.com (URL overrides only; scopes inherited) per #9358, so a single
canonical `salesforce` resource type covers both with separate `salesforce_sandbox`
instance credentials.

Paired with the hub integration: windmill-labs/windmill-integrations#131.
The Salesforce icon already exists in the frontend (SalesforceIcon.svelte).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Fix JSON syntax error in oauth_connect.json

* fix: add salesforce production tile to OAuth settings dropdown

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 17:49:33 +00:00
GuilhemandClaude Opus 4.8 1275487f02 feat: refine ask-user-question chat display and keyboard nav (#9392)
* feat: refine ask-user-question chat display and keyboard nav

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style: use text-accent for ask-user-question icon

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: focus active choice when clicking ask-user-question card

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: disable chat input while an ask-user-question is pending

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: focus active choice on card click instead of pointerdown

Preserves text selection on the question card; wired as a use: action so the non-interactive card needs no keyboard handler.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: extract isActiveUserQuestion shared predicate

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: cover isActiveUserQuestion predicate

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 17:49:12 +00:00
centdixandClaude Opus 4.8 943ef6eb20 feat: add workspace datatable tools to global AI chat mode (#9395)
* feat: add workspace datatable tools to global AI chat mode

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: cover global-mode datatable tools pure logic

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: expose datatable SQL SDK reference via get_instructions in global mode

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: make datatable get_instructions language-aware, default TypeScript

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop datatable/whitelist args from global init_app tool

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: flag missing datatable config as an explicit blocking error in global mode

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop dead branch in exec_datatable_sql result handling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 17:48:50 +00:00
Ruben FiszelandClaude Opus 4.8 c19441bc8c perf(python): add --compile-bytecode to uv pip install (#9393)
Python jobs under nsjail experience slow imports (~465ms for `import
requests`) because dependency directories are mounted read-only. Without
pre-compiled `.pyc` files, Python recompiles `.py` source to bytecode
in-memory on every import in every fresh nsjail process, paying the cost
repeatedly.

Add `--compile-bytecode` to both `uv pip install` invocations (the
Rust-driven install in python_executor.rs and the nsjail
download_deps.py.sh script) so `.pyc` files are generated at install
time and available at runtime even through read-only mounts. The files
are included in both the local cache and S3 piptar uploads.

The flag is supported by the uv version (0.9.24+) shipped in the
Dockerfile. The existing `__pycache__` skip only filters top-level dirs
for dedup logic, not subdirs within packages, so there is no conflict.

Fixes WIN-2001

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 15:46:33 +00:00
centdix 5c20d6b4f7 feat: add global ai chat test tools (#9391)
* feat: add global ai chat test tools

* fix: avoid session id in flow test preview

* test: cover global flow preview ids

* test: require script and flow test tools

* fix: harden global flow test fallback

* Revert "fix: harden global flow test fallback"

This reverts commit 97254ef33a.

* fix: fallback from inactive flow test hook

* fix: list nested flow steps in errors
2026-06-01 15:43:07 +00:00
GuilhemandClaude Opus 4.7 075faabf3b feat(frontend): surface local drafts in drawer editors with an unsaved-changes banner (#9335)
* feat(frontend): surface local drafts in drawer editors with an unsaved-changes banner

Drawer-based editors (the 11 trigger types, plus resource and variable)
restore unsaved edits from browser localStorage on open using the same
mechanism as flows/scripts, but only showed a transient "Reset to deployed"
toast with no way to review the diff.

Add a persistent "You have unsaved changes" banner below the drawer header
with Show diff / Discard actions, shown whenever the form diverges from the
deployed baseline. Replaces the toast for these editors; flows/scripts/apps
(full-page) keep their existing toast.

- new shared LocalDraftBanner.svelte (Alert-styled bar + DiffDrawer)
- DrawerContent: optional `banner` snippet rendered below the header
- useTriggerDraftSync: reactive `hasDraft`, `deployed`/`current` getters and
  `resetToDeployed`; drop the restore toast (banner supersedes it)
- wire the banner into all 11 trigger editors + variable; resource lifts its
  dirty state up to ResourceEditorDrawer via a callback + accessors
- fix ScheduleEditorInner.openNew not resetting initialConfig (reused editor
  instance kept a stale baseline, wrongly flagging a new schedule dirty)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(frontend): address PR review on local-draft drawer banners

- LocalDraftBanner: snapshot diff sides at click time so the diff drawer
  doesn't keep updating as the user types behind it.
- VariableEditor / ResourceEditor: scope the banner and its Discard action
  to the selected workspace; the cross-workspace dirty state stays surfaced
  by the existing otherDirty Alert. Forward can_write via a new
  onCanWriteChange callback so the resource banner hides Discard in
  read-only mode (matching the trigger editors).
- useTriggerDraftSync: drop the now-unused path arg from maybeRestore and
  update all 11 trigger editor call sites.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(frontend): deep-clone fallback in UserDraft.discard to avoid baseline aliasing

When a caller passed a live $state proxy as `fallback` (Variable/Resource
editors handed `initialStates[selected]` to the banner's Discard), the
handle's draft cell ended up sharing the same proxy as the caller's
baseline. Subsequent form edits mutated both sides in lock-step and the
dirty check kept reporting equal, so the banner never reappeared and
the Update button stayed disabled until the drawer was reopened.

Cloning the fallback inside `discard` (via `snapshotDraftValue`) gives
the handle a fresh tree and decouples the two reactive graphs. Trigger
editors already cloned at their call site (resetToDeployed); this just
makes the API self-contained for all callers.

Also switch the variable form's "Audit log for each access" alert from
warning to info — it's informational, not a warning.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(frontend): honor disabled prop in LocalDraftBanner's diff drawer

The banner's `disabled` prop hid the inline Discard button but the diff
drawer's "Discard changes" action was still wired unconditionally, so a
read-only user could bypass the hidden inline action via Show diff.
Gate the diff-drawer button on the same flag so both surfaces agree.

Flagged by cubic and Codex on PR #9335.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-01 14:46:30 +00:00
GuilhemandClaude Opus 4.7 e4213c1ab8 feat(flow-ai): constrain flow-group colors to the NoteColor palette (#9343)
The flow AI chat's set_flow_json tool lets the model set a `color` on each
semantic flow group, but nothing told it which colors are valid, so it would
sometimes emit hex codes / arbitrary CSS color names. Those render with
default styling at best and break the group color picker at worst.

- core.ts: the set_flow_json schema `.describe()` and the `groups` system-prompt
  bullet now spell out that `color` MUST be one of the palette names
  (yellow, blue, green, purple, pink, orange, red, cyan, lime, gray) — no hex,
  no CSS colors — and that omitting it lets the editor auto-assign one.
- helperUtils.ts: validateFlowGroups now rejects any color outside that palette,
  sourced from the NoteColor enum so the two can't drift.
- helperUtils.test.ts: tests for reject-unknown / accept-known / accept-omitted.

Split out of the sessions branch (gl/layout-ai), where it had been bundled
into the large feature commit; it's an independent flow-AI improvement.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 11:15:22 +00:00
GuilhemandClaude Opus 4.7 eadeac248b feat: sessions page with isolated AI chat + flow editor (#9034)
* feat(sessions): chat + editor side-by-side with multi-session state

Introduces the Sessions feature: a workspace where the AI chat and an
editor (flow / script / app / raw-app) sit side-by-side, with each session
having its own AIChatManager instance, history, and target item. Sessions
are persisted across reloads and can be staged into forks for review.

Key pieces:

- sessions/ — SessionWrapper (the split-pane shell), SessionPicker
  (sidebar list), SessionForkBar, SessionWorkspaceBar, FlowEditorView /
  ScriptEditorView / AppEditorView / RawAppEditorView, ForkDiffDrawer,
  sessionRuntime (per-session AIChatManager + draft state),
  sessionState (in-memory + persisted index), sessionUnread, sessionScope,
  appDraftCodec / flowDraftCodec, forkEditUrl, /sessions route.

- WorkspaceItemDrillPicker refactor — extracts WorkspaceItemRow + adds
  surfaceAI drafts, stale-while-revalidate. workspacePicker.ts drops
  explicit invalidate() in favor of always re-fetching in the background.

- ForkDiffDrawer + WorkspaceItemDiffViewer — per-kind diff bodies
  reusable from the compare page. FlowGraphDiffViewer / FlowGraphV2 gain
  inlineDiff forwarding + onHeight callback for equal-height layout.

- Global AI chat sessions plumbing — AIChatManager exports the class +
  adds disabledModes, beforeSend hook, scoped instance context. AIChat /
  AIChatDisplay accept session-only props (wideLayout, emptyHint,
  inputPreface, hideHeader, hideModeSelector, forceDisabled). Chat
  preserved across /flows/add → /flows/edit, /scripts/add → /scripts/edit.

- Draft-first loaders — sessions open drafts when present, otherwise
  seed a draft from the last deployed value via globalDraftStore.
  RawAppEditor / AppEditor / AppEditorHeaderDeploy get newApp prop +
  fixes so draft-only apps can deploy.

- Compare page (/forks/compare) — bigger overhaul to plug into the new
  drawer.

- Sidebar — Sessions entry + unread badge + status dot in
  SidebarContent / MenuButton / SideBarNotification.

- Misc fixes — chat group color palette constraint, deploy_workspace_item
  confirmation dropped, open_preview tool, picker drafts surfacing,
  fork archive/delete buttons on compare page.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): bypass UserDraft inside session panes + sessionUnread crash

After merging main's UserDraft PR (#9121) into the sessions branch, two
integration issues surfaced:

1. AppEditor.svelte calls `UserDraft.use<App>('app', path)` at the
   component level — keyed by ($workspaceStore, 'app', path). Sessions
   that haven't materialized a fork yet stay at the user's main
   workspace, so a session targeting an app at the same path as a
   regular /apps/edit tab shared the same LS key. The session would
   read the regular tab's autosave and write its fork-edits back over
   it.

   Gate UserDraft.use on `!getContext('aiChatManager')` — sessions
   inject the manager via setContext, so inside a session pane the
   handle is `undefined`, stateApp falls through to the `app` prop
   the session loaded, and the auto-save $effect bails. Same gate on
   the four UserDraft.remove call sites in AppEditorHeader and
   RawAppEditorHeader so save/deploy from a session pane doesn't wipe
   the LS draft of a non-session tab at the same path.

2. sessionUnread.svelte.ts called useLocalStorageValue at module
   scope. Main's PR added a deep-mutation $effect inside that helper,
   which now requires component-initialization context — every page
   crashed at import time with `Svelte error: effect_orphan`.
   Replaced with a plain module-level $state + manual localStorage
   persist; same reactivity contract for callers.

3. ScriptEditorView.svelte was passing a `replaceStateFn` prop that
   ScriptBuilder dropped on main. Removed.

Verified end-to-end with Playwright:
- /flows/edit/{path} regression: UserDraft handle still created, no
  console errors
- /sessions loads, sessionUnread doesn't crash
- Session targeting non-raw app `u/admin/userdraft_collision_test`
  displays the fork content (FORK_ONLY_MARKER) even with an LS
  poison at `userdraft/w/local/app/{path}` containing a
  POISONED_BY_REGULAR_TAB_AUTOSAVE marker; poison remains untouched
  after the session loads and renders

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): stop fork-create retry loop on first user message

Removed the SessionWrapper $effect that retroactively committed the
session's workspace from the in-memory chat history. When opening a
session whose previous commit attempt had failed (or whose response was
lost) the effect ran in a tight retry loop, flooding the user with
`workspace_pkey` violations from `create_workspace_fork`.

The send path already commits through `AIChatManager.beforeSend` →
`commitSessionWorkspace`, which is the deterministic moment-of-action.
The $effect was a redundant reactive bridge that turned every backend
failure into an infinite retry.

Also hardens `materializeFork`/`commitSessionWorkspace` so the most
common cause of the duplicate-key error self-heals:

- `materializeFork` short-circuits when `fork.id` is already in
  `$userWorkspaces` (the previous create actually succeeded, we just
  lost the response). On a `workspace_pkey` catch, refresh the workspace
  list and adopt the existing row instead of toasting an error.
- On a real `materializeFork` failure, `commitSessionWorkspace` now
  drops `pending_fork` so the session falls through to the
  workspace-pick fallback instead of looping on the same broken intent.

* feat(sessions): show EditorHeader breadcrumb in the not-found state

When a session's target item has been deleted or moved, the editor pane
used to render a bare "Script not found at path X" line — leaving the
user with no way to navigate to a different target without backing out
of the session.

Each editor view now renders a `SessionItemNotFound` shell instead: a
real `EditorHeader` (read-only summary, no pen popover) with a
breadcrumb keyed to the missing kind+path, plus the "not found" copy
below. Clicking any breadcrumb segment opens the workspace picker
scoped to that level — pick a replacement and the session swaps target
via the existing `onNavigate` callback.

`SessionItemNotFound` maps `raw_app` to `EditorHeader`'s `kind: 'app'
+ raw_app: true` so the picker routes through `/apps_raw/...`; the
local label still says "Raw app not found" (not "App not found") so
the user knows which surface is missing.

* fix(picker): stop self-feeding fetch effect that OOM'd the tab

The drill picker's $effect watched `scope` and called `ensureLoaded`
on every change. `ensureLoaded` reads `loaded[kind]` synchronously
(to decide whether to show a spinner), so the effect ended up
subscribed to the very signal it fills. Each fetch result wrote
`loaded[kind] = items`; Svelte 5's $state proxy notifies on every
property set even when the reference is unchanged from cache, which
refired the effect, which called `ensureLoaded` again, which awaited
the cached fetch, which wrote `loaded[kind]` again... runaway loop.

In `/scripts/edit/...` the picker's lifecycle stabilised quickly
enough to mask the loop, but in a session pane (multiple warm
sessions, picker kept alive by the surrounding state) the cycle
spun freely — 29.8 million iterations in <100 ms during testing,
enough to OOM Firefox / kill the Chromium tab.

Two changes:

- Replace the scope-watching $effect with an explicit `setScope()`
  helper called from `drill()`, `goUp()`, and `onMount`. Fetch is
  now a callback reaction to user navigation, never a reactive
  consequence of one. No closed feedback cycle is possible.

- Untrack the `loaded[kind]` read inside `ensureLoaded`. The search
  $effect (which loads every kind on first keystroke) is still a
  reactive caller; the untrack stops it from subscribing to the
  signal `ensureLoaded` fills, so the same loop can't form there.

* feat(script-editor): wire initialTestPanelCollapsed through ScriptBuilder

The `initialTestPanelCollapsed` prop was already declared on
`ScriptBuilderProps` (used by the session preview to start the editor
with the run/test pane closed) but never destructured in
`ScriptBuilder.svelte`, so the value silently dropped on the floor
and the test pane always opened.

- `ScriptBuilder.svelte` — destructure the prop and forward it to
  `<ScriptEditor>`.
- `ScriptEditor.svelte` — accept the prop and seed `rawTestPanelSize`
  to 0 when true, while keeping `storedTestPanelSize` at the default
  30 so the user's first toggle expands the pane to a sensible width
  rather than 0.

Regular `/scripts/edit/...` doesn't pass the prop → default `false`
→ panel still opens by default.

* fix(sessions): resolve aiChatManager via context in AskUserQuestionDisplay

Inside a session the chat uses a per-pane AIChatManager injected via context. AskUserQuestionDisplay imported the global singleton, so answers clicked in a session dispatched to the singleton's callback map and the AI loop stalled. Resolve via getContext with singleton fallback, matching ChatMode / ToolExecutionDisplay.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(raw_apps): let preview start in single-view on the preview tab

Add a defaultSplitWithPreview prop (default true). When false (session preview), the editor boots in single view with the preview tab selected: gate the onMount default-file activation, the setActiveDocument auto-activation, and iframeShouldMount so the UI Builder bundler iframe still mounts when preview is the active tab. RawAppEditorView passes defaultSplitWithPreview={false}.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(copilot): add get_preview_status tool and make open_preview idempotent

So the assistant can tell whether the session preview already shows the item it just edited, instead of re-opening or re-offering it. Mirrors the open_preview handler plumbing (setGetPreviewStatusHandler) and the session runtime registers it alongside open_preview. open_preview now returns 'already open' when the requested target matches the active session's current target. The system prompt steers the AI to check status before offering. Unit tests cover the no-arg schema, the session-only error, and handler dispatch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): make script preview reactive to AI draft writes

ScriptEditorView read the draft via static UserDraft.get inside an effect, which only subscribes to UserDraft's reactive cell when a live entry exists. None did for the preview path, so the chat's writes (UserDraft.save) only touched localStorage and the open preview never updated. Hold a live handle via UserDraft.useMany (reactive getter so it re-acquires when open_preview swaps the path without remounting) and read inbound through handle.draft, materializing the shared $state cell that bridges the chat's writes to the editor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): make raw-app preview reactive to AI draft writes

Mirror of the script-preview fix. RawAppEditorView read the draft via static UserDraft.get inside an effect, which only subscribes to UserDraft's reactive cell when a live entry exists. None did for the preview path, so the chat's raw-app writes (UserDraft.save / setDraftAndMeta, from write_app_file / patch_app_file / write_app_runnable) only touched localStorage and the open preview never updated. Hold a live handle via UserDraft.useMany (reactive getter so it re-acquires when open_preview swaps the path without remounting) and read inbound through handle.draft. Verified in-browser: an external UserDraft.save live-updates the bound summary in the open preview.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): make flow preview reactive to AI draft writes

Mirror of the script/raw-app preview fixes, completing two-way binding for all three session editor kinds. FlowEditorView read the draft via static UserDraft.get inside an effect, which only subscribes to UserDraft's reactive cell when a live entry exists — none did, so the chat's writes (write_flow / patch_flow_json / set_flow_module_code) only touched localStorage and the open preview never updated. Hold a live handle via UserDraft.useMany (reactive getter so it re-acquires when open_preview swaps the path without remounting) and read inbound through handle.draft. Verified in-browser both directions: an external UserDraft.save live-updates the flow header summary and rebuilds the module graph; a preview edit propagates through the debounced save to both UserDraft.get and the chat's getGlobalDraft adapter.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(sessions): surface local-storage drafts in fork diff & compare page

Augments the backend fork-vs-parent comparison with browser-local (UserDraft) drafts so a session's uncommitted AI/user changes are visible in the Fork Diff Viewer and the /forks/compare page. Adds forkDraftDiff.ts (augmentForkComparisonWithLocalDrafts + getForkItemValue), a 'local changes detected' / new-draft warning surface (checkbox-slot warning icon, no-op-baseline filtering, dedup), a 'Local draft <> fork' tab in DiffDrawer, and selectTooltip/nonSelectableTooltip plumbing in Row/WorkspaceDeployLayout.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Revert "feat(sessions): surface local-storage drafts in fork diff & compare page"

This reverts commit 3cfd858e36.

* fix(sessions): leave for home when switching workspace from the session page

An AI session is scoped to its (forked) workspace, so it makes no sense to keep showing it after the user picks a different workspace. The workspace switcher's link href now points home on the session route (the link navigation wins over onClick's preventDefault), and toggleSwitchWorkspace also redirects home there as a fallback. Session-switching uses a separate path (syncWorkspaceTo), so it's unaffected — which is why reacting at the switcher is more robust than watching workspaceStore.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): clear session highlight off the session page; default delete-fork on

Two SessionPicker fixes: (1) only highlight the active session while on the /sessions route — currentSessionId lingers after navigating away, so the row stayed selected in the sidebar; gate the highlight on the route. (2) The 'Also delete forked workspace' toggle in the delete-session modal now defaults to on (the fork is tied to the session and would be orphaned otherwise); resets keep it defaulted-on for the next open. User can still untick it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(copilot): say "local storage" instead of "draft" in write-tool status

The global chat's write tools persist to the browser's localStorage (UserDraft), not a workspace draft. The tool status / result messages now say the item was saved to local storage (and discard says it was discarded from local storage) so users aren't misled into thinking a workspace draft was created. Covers the shared script/flow/trigger/resource/variable helpers and the app tools.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): sidebar collapse, new-session chat, fork delete & not-found nits

- Hide the collapse chevron and make the section header non-interactive when there are no sessions; reset the persisted collapsed state while the list is empty so the first session always appears expanded.
- Stop grafting a recent past chat onto a freshly created session: ensureChatIdsSeeded now skips transient sessions, so the seed only pairs untagged chats with pre-existing sessions.
- After deleting a fork from a session (SessionPicker / SessionWrapper), fall back to the fork's parent workspace when the deleted fork was the active one, instead of stranding the user on a deleted workspace.
- Show a 'Session not found' message (with a New session action) when the URL names a session that doesn't exist, rather than rendering a blank page.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(copilot): expose preview tools only to session chats

open_preview and get_preview_status drive a session's side-panel editor, so they only make sense inside an AI session. They were always present in the global tool list and just errored when called outside a session. Now AIChatManager carries an isSessionChat flag (set by sessionRuntime.createRuntime); the GLOBAL-mode branch uses globalToolsFor({ sessionPreview }) to drop the two tools for the regular side-panel chat, and prepareGlobalSystemMessage omits their guidance unless previewTools is set. The module-level handlers + in-tool error guards stay as defense in depth.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* chore(flow-editor): move intra-editor chat preservation to its own PR

The beforeNavigate / preserveChatOnDestroy guard that keeps the global FLOW
chat alive across same-flow editor remounts is a standalone global-chat fix,
unrelated to sessions. Split out to #9339; FlowEditor reverts to the plain
session-guarded saveAndClear lifecycle here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(raw-app): pre-boot session editor hidden so files open instantly

In single-view (sessions) the UI Builder iframe was mounted inside a
display:none wrapper while the Preview tab was active, so the VS Code
workbench booted at 0x0, threw in its LayoutService ("Unable to figure
out browser width and height"), and wedged the editor on "Loading
editor" with no recovery when later revealed.

Keep the iframe mounted at the editor area's real width and hide it with
visibility instead of collapsing it: Monaco boots correctly while hidden,
and revealing a file is an instant un-hide (no reload, no relayout, no
latency).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(flow-ai): move flow-group color-palette work to its own PR

The flow-group color-palette guidance + validateFlowGroups guard + tests are
an independent flow-AI improvement, not part of sessions. Split out to #9343;
these three flow files revert to their main state here.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): hide the in-editor Flow AI Chat button in the session preview

The flow preview pane in a session already sits next to the session's own AI
chat, so FlowBuilder's in-editor "Flow AI Chat" toggle (which opens the global
singleton chat) is redundant and confusing there. Pass
customUi={{ topBar: { aiBuilder: false } }} from FlowEditorView, reusing the
existing showFlowAiButton gate (!disableAi && customUi?.topBar?.aiBuilder !=
false) that flows down to FlowStickyNode — no new prop needed.

Verified in-browser: the button (WandSparkles) renders in the regular
/flows/edit route but is absent in the session preview for the same flow.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): mirror /scripts/add for never-saved scripts in editor preview

An AI-created script with no backend version yet left savedScript undefined
in the session preview, which disabled Save draft and hid Show diff. Open it
as a new script (empty initialPath) like /scripts/add so Save draft is enabled
and creates it on first save; seed the path as already-chosen
(initialPathChosen) so the summary->path auto-slug does not rename the
AI-assigned path. On first save ScriptBuilder writes savedScript back through
the bind and flips into edit mode (Save draft + Show diff) without navigating
away.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): refresh fork diff count after an editor draft save

The fork-bar diff count reads a cached comparison refreshed only on AI-turn-end or tab refocus. A 'Save draft' in the session editor registers in the backend fork tally asynchronously (~300ms after the create returns), so the count stayed stale until one of those triggers fired. Add SessionRuntime.scheduleForkComparisonRefresh() (re-fetches at 700ms + 2200ms to clear the async tally) and wire it to onSaveDraft in ScriptEditorView and FlowEditorView.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): don't auto-open the settings drawer in script preview

When the AI's open_preview tool previews a never-saved script, ScriptEditorView
passes initialPath='' so ScriptBuilder behaves like /scripts/add. That empty
path also triggered ScriptBuilder's auto-open of the settings drawer, which is
unwanted in the session preview where the AI manages metadata. Pass
neverShowMeta so the drawer stays closed on mount; the Settings button still
opens it manually.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): don't host legacy drag-and-drop apps in the editor preview

The session preview pane only hosts code-based items (flow, script, raw
app). Drop the legacy 'app' kind from SessionTarget and the open_preview
tool, and route a legacy app picked in the drill picker to the standalone
/apps/edit editor instead. Removes the now-dead AppEditorView and its
runtime load path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): don't prompt to discard raw-app changes on navigation

In a session the raw-app editor's content is continuously persisted to the
UserDraft (localStorage), so tearing the editor down on navigation loses
nothing. Skip the UnsavedConfirmationModal (and its beforeNavigate guard)
when the editor is mounted inside a session pane; the standalone /apps_raw
editor still shows it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sidebar): pin Help to the bottom instead of floating

The bottom of the sidebar stacked the User/Settings cluster and the Help
block with a fixed ~40px gap between them, plus a bottom margin that kept
Help from sitting flush — so Help appeared to float. Drop those fixed
margins so the cluster and Help stay glued at the bottom with a small gap
and Help is flush, and let mt-auto own the flexible space above the group.
Add pt-4 so the cluster keeps a minimum gap from the Triggers section when
the sidebar runs out of room and that flexible space collapses.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): surface diff/discard for AI script drafts in preview, refresh diff on deploy

loadScript built the editor's scriptStore by aliasing and mutating savedScript.val, so the deployed baseline got overwritten with the draft content and the diff compared draft-vs-draft. Clone the baseline before layering the AI draft on top. On load, when the local draft diverges from the saved baseline, surface a toast ('AI saved a local draft') with Show diff (opens the diff drawer with a Discard-draft button) and Discard local draft — mirroring the regular /scripts/edit affordance the session's parallel loader omitted. Also wire onDeploy (alongside onSaveDraft) to scheduleForkComparisonRefresh so the fork diff count refreshes after a deploy, not just on an AI turn or tab refocus.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): script preview restore/deploy feedback; drop on-load draft toast

- Implement real restoreDeployed/restoreDraft for the diff drawer: the shared loadScript-based handler was a no-op (loadScript early-returns on the loaded path and would re-read the local draft). Reset the live UserDraft handle to the chosen baseline (deleting the backend draft for 'restore to deployed') so the inbound effect syncs the editor.
- Show a 'Deployed' toast on deploy: the default Deploy takes ScriptBuilder's no-toast branch (the editor navigates away instead); the session stays put, so surface the success toast.
- Remove the on-load 'AI saved a local draft' toast: unnecessary in a session, where the user already expects their changes to be present. Diff/discard remain reachable via ScriptBuilder's Show diff + the diff drawer's restore buttons.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): gate breadcrumb picker draft-merge behind the dev flag

The WorkspaceItemDrillPicker merges localStorage UserDrafts into its
navigable items so in-flight session/chat drafts are reachable. That
merge was ungated, so with the sessions dev flag off it also surfaced
the standalone editors' autosave drafts — they appeared as navigable
rows that 404 on the backend draft fetch. Gate aiDraftsForKind on
isGlobalAiEnabled() so it is a no-op without the flag (no sessions
exist then anyway); inside sessions the merge still works.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(editor): reload script/flow editor on client-side breadcrumb nav

Picking a different item in the editor-header breadcrumb picker calls
goto() for a client-side navigation. SvelteKit reuses the same +page
instance across a path-param change, but the script and flow editor
routes captured `draftPath` and the `UserDraft.use()` handle once at
mount and never remounted ScriptBuilder/FlowBuilder. The URL and title
updated while the editor kept showing the previous item's breadcrumb,
summary and content; only a full reload showed the navigated-to item.

Mirror the pattern the app / raw-app editors already use:
- Derive the draft path from the URL and key the handle off it via
  `UserDraft.useMany` (a stable proxy onto the current handle), so the
  reload reads/writes the navigated-to item's draft instead of the
  previous one's — fixing the stale draft-comparison too.
- Gate the builder subtree on a `renderEditor` flag flipped false when a
  navigation kicks off the reload and true once the data is ready, so
  the builder cleanly unmounts and remounts once against stable data. A
  synchronous `{#key}` swap instead races Monaco's async init against
  the torn-down container.
- Flows also reset `nobackenddraft` per navigation so a fresh load
  reconsiders the backend draft.

The unsaved-changes guard is unaffected (it runs in beforeNavigate,
before the remount). The app and raw-app editors already handled this
and are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sessions): sync preview with the deployed version on editor + chat deploy

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(sessions): reload the preview after a chat raw-app deploy

The deploy-reload-preview callback added previously only fired for script and
flow. Now that the merged deploy_workspace_item tool can deploy raw apps
(bundle + createAppRaw/updateAppRaw), wire raw apps in too. A raw app deploys
under type 'app' but the session preview addresses it as 'raw_app', so the
deploy handler maps 'app' -> 'raw_app'; the runtime open-check gains the
loadedRawAppPath case. syncPreviewWithDeployed already handled 'raw_app'
(discard the local draft + force-reload via loadRawApp), so no runtime change
was needed there.

Adds a unit test asserting deploy_workspace_item(type:'app') notifies the
session handler with { kind: 'raw_app', path }.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): address Claude PR review (3 P1 + 3 P2 + test)

P1:
- Drop the hardcoded placeholder default sessions (u/guilhempw/...). New users
  (empty/cleared/private-browsing localStorage) now start with no sessions and
  see the empty state instead of unresolvable "session not found" rows.
- Scope the preview/deploy tool handlers to the *calling* session. open_preview,
  get_preview_status and the deploy reload handler dispatched via the global
  currentSessionId, so a backgrounded session's tool call mutated the UI-active
  session. The calling session id is now carried in the per-manager tool
  `helpers` (AIChatManager.sessionId, set in createRuntime) and threaded through
  the tool ctx to the handlers, which dispatch to it (falling back to the active
  id only when absent). Keeps backgrounded sessions isolated.
- beforeSend now aborts the send on failure: commitSessionWorkspace throwing used
  to be swallowed, letting the message go out against the wrong workspace
  silently. Now it toasts and returns. Also guarded the unguarded
  listUserWorkspaces refresh in materializeFork's duplicate-key self-heal so a
  second network failure can't rethrow past the toast-and-return contract.

P2:
- disposeRuntime now clears the fork-comparison refresh timers (700ms/2200ms)
  via a new runtime.dispose(), so an evicted/deleted runtime can't fire a stray
  refreshForkComparisonNow/compareWorkspaces after teardown.
- Convert Svelte 4 on:click -> Svelte 5 onclick on the Button components in
  SessionWrapper, SessionForkBar, ForkDiffDrawer, SessionPicker, sessions/+page.
- WorkspaceItemRow's <a href> branch gains role="option" + aria-selected to match
  the <button> branch, for consistent listbox semantics.

Tests:
- core.test.ts: deploy_workspace_item(type:'app') threads the calling session id
  through helpers to the deploy handler ({ sessionId, kind:'raw_app', path }).
- New sessionState.test.ts unit-tests deriveForkStatus + isForkSession across
  all branches (root/fork/unavailable/draft, ahead/behind/diverged/in_sync).

svelte-check 0 errors; 57 frontend unit tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(copilot): collapse deploy preview-reload dispatch to a type→kind map

Replace the if/else-if that mapped deploy type to preview kind with a single Partial<Record<WorkspaceItemType, ...>> lookup + one if. Non-previewable types map to undefined → no dispatch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor(copilot): use getAiChatManager() instead of inlining the context fallback

Six chat components still inlined
`getContext<AIChatManager>('aiChatManager') ?? singletonAiChatManager` even
though aiChatManagerContext.ts already exports getAiChatManager() for exactly
this (the resolve-scoped-instance-or-fall-back-to-singleton pattern, already
used by AIChatDisplay/AIChatInput/AIChatMessage/CodeDisplay). Adopt it in
DatatableCreationPolicy, ChatMode, ToolExecutionDisplay, AIChat,
AskUserQuestionDisplay and flow/FlowAIChat, and drop the now-unused getContext /
AIChatManager / singletonAiChatManager imports (FlowAIChat keeps getContext for
its FlowEditorContext/FlowCopilotContext lookups).

No behavior change — getAiChatManager() is the same resolution.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): consistent script deploy → preview sync; trim session deploy menu

Two related session deploy fixes + clarifying comments.

1. Hide the extra deploy-dropdown options in the session script preview. The
   editor always "stays" and is already scoped to a fork, so Deploy & Stay here,
   Fork, Edit in workspace fork, Exit & See details and Export as YAML/JSON make
   no sense there — only "Show diff" is kept. ScriptBuilder gains
   `inSessionPane = !!getContext('aiChatManager')` (same pattern ScriptEditor
   uses) and gates those items. (They were correctly absent for never-deployed
   session scripts but leaked for deployed ones.)

2. Fire onDeploy on every successful script deploy. ScriptBuilder previously
   skipped onDeploy for "Deploy & Stay here" and lib scripts (it just re-pinned
   parent_hash + toasted), so a session preview wouldn't sync after those. Now
   onDeploy always fires with a `stay` flag; route consumers skip navigation when
   stay (behaviour identical to before — stay → toast only, primary → navigate),
   and the session ignores stay and always syncs. With (1) hiding Deploy & Stay,
   this now covers the lib-script-in-session case.

3. Comments: RawAppEditorHeader / AppEditorHeader note that the
   `if (!inSessionPane) UserDraft.remove` guards are intentional — the editor
   doesn't own the localStorage draft in a session (the runtime does, keyed by
   the fork); the session-side equivalent is the View's onDeploy →
   runtime.syncPreviewWithDeployed (discard fork draft + reload to deployed).

svelte-check 0 errors; session dropdown verified to show only "Show diff" for a
deployed script, route deploy menu unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sidebar): single Menubar so bottom menus hover-switch (WIN-1993)

The bottom sidebar group split Settings/Workers/Folders/Logs and Help across
two separate <Menubar> components. melt-ui's hover-to-switch (open menu closes
when another trigger in the same Menubar is hovered) only coordinates within a
single Menubar, so hovering between the two groups left both menus open
(stacked) instead of switching. Collapse them into one Menubar, wrapping each
group in its own flex container to preserve spacing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(editor): gate external code sync behind opt-in syncExternalCode prop

The unconditional `code` prop->Monaco sync effect added for sessions
live-preview ran for every <Editor> caller (14 call sites). Most either
bind:code with their own external-sync (e.g. ScriptEditor) or treat code as
init-only, so a blanket setValue risked clobbering them. Gate the effect on a
new opt-in `syncExternalCode` prop (default off) and enable it only at the two
flow inline-rawscript editors — the case that actually needs external updates
(AI chat editing a flow module's content reflecting live in the preview).

Verified in-browser: AI-driven external edit to a flow step now reflects live
in Monaco, and typing keeps the caret intact (round-trip guard).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sessions): address P1 review findings (commit-abort, render-stuck, workspace sync)

From the cubic/Claude PR review:

1. beforeSend now aborts the send when the workspace isn't committed. The earlier
   fix only caught a *thrown* error, but commitSessionWorkspace returns undefined
   (never throws) when a staged fork fails to materialise — so the first message
   + its tool calls shipped to get(workspaceStore) (the parent). beforeSend now
   throws on undefined so AIChatManager's catch toasts + aborts.

2. The script/flow edit reload effect set renderEditor=false then called
   loadScript()/loadFlow(); a rejected fetch left renderEditor stuck false, so the
   editor pane vanished and never remounted. Both calls now .catch → toast +
   renderEditor=true (token-safe), so the pane always remounts.

3. SessionWrapper.moveAndActivate now syncWorkspaceTo(target) — moving a session
   off an unavailable workspace was leaving the app pointed at the old one
   (mismatch with moveSessionToNewFork / handleConfirmedDelete).

Test: sessionState.test.ts pins commitSessionWorkspace's failure contract
(returns undefined + drops pending_fork when the fork fails) — the invariant the
beforeSend abort relies on. svelte-check 0 errors; 58 frontend unit tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): address P2 review findings (cubic)

Draft round-trip:
- appDraftCodec: carry custom_path through runtimeRawAppToDraft /
  applyDraftToRuntimeRawApp (+ seed it in loadRawApp) so a session round-trip
  no longer erases a raw-app draft's custom URL.
- sessionRuntime.loadScript "no draft" path: structuredClone the baseline before
  setting parent_hash — it could alias `result` (= savedScript.val) and corrupt
  the pristine deployed baseline the diff drawer reads.
- FlowEditorView: include `summary` in the inbound/outbound dedup sigs so
  summary-only changes propagate/persist.

Workspace-state on navigation:
- SidebarContent (post-delete) and workspace_settings (post-archive): guard the
  listUserWorkspaces() refresh so a transient failure can't strand the user on
  the just-removed workspace, and refresh the list before switching to parent.
- WorkspaceMenu: keep ?workspace=<id> in the session-page workspace href so a
  modifier/middle click (which bypasses onClick) lands in the right workspace.

UI/keyboard:
- WorkspaceItemRow: indent adds to the px-3 base (calc) instead of replacing it.
- ForkDiffDrawer: ArrowLeft maps a 2-segment file path (f/foo) to its scope
  folder (folder:f/foo) instead of a nonexistent folder:f.
- flows/edit: defer flowBuilder setup (primary schedule, draft triggers,
  loadFlowState) until after the builder remounts (renderEditor=true + tick),
  so reload-time state restoration isn't skipped on the unmounted builder.

svelte-check 0 errors; 58 frontend unit tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(sessions): unit-test the P1/P2 review fixes (extract pure helpers)

Extract the pure logic touched by the review fixes into small tested helpers
(behaviour-preserving) and add unit tests:

- appDraftCodec.test.ts — custom_path survives the runtime↔draft round-trip (A1).
- forkDiffNav.ts/.test.ts — parentFolderKey (extracted from ForkDiffDrawer):
  ArrowLeft parent resolution incl. the 2-segment-path case (C2).
- workspaceMenuHref.ts/.test.ts — extracted from WorkspaceMenu: session-route
  href keeps ?workspace=<id>; off-session swaps the param (B2).
- flowDraftSig.ts/.test.ts — extracted from FlowEditorView (dedups 3 sig sites):
  the dedup signature includes summary, so summary-only changes propagate (A3).

(commitSessionWorkspace failure-contract test for the beforeSend P1 landed with
the P1 commit.) svelte-check 0 errors; 75 frontend unit tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(sessions): address second-round review (Pi + Codex)

Three findings flagged post-push (cubic was fully addressed in the prior
commits; this commit covers the new ones):

- [P1] commitSessionWorkspace non-fork branch — when a session created
  inside a fork defaults pending_workspace_id to the family root, commit
  set s.workspace_id but never synced workspaceStore. First send's
  logAiChat + tool calls then ran against the wrong (still-fork)
  workspace. Fix: syncWorkspaceTo(ws) after the commit, mirroring the
  pending_fork branch's switchWorkspace(newId).

- [P1] Warm-session live-editor slot hijack — /sessions keeps up to 3
  warm-mounted sessions; UserDraft stores one live editor per
  (workspace, kind). Each editor view unconditionally claimed the slot,
  so a hidden warm session in the same workspace+kind could overwrite
  the visible session's claim — chat actions like discard /
  "the open editor" then resolved to the wrong session. Fix: thread
  isActiveSession from SessionWrapper into Script/Flow/RawAppEditorView
  and gate setLiveEditorDraft on it.

- [P2] ForkDiffDrawer stale per-item raw diff cache — loadedDiffs /
  summaries persist for the drawer's lifetime; fetchComparison refetched
  on each open() but loadDiffFor short-circuited on cached keys, so an
  edit-then-reopen showed fresh counts but stale expanded content. Fix:
  clear both records at the top of fetchComparison.

Tests:
- sessionState.test.ts: 2 tests pinning commitSessionWorkspace's
  workspaceStore sync (mismatch and matching).
- userDraft.test.ts: 3 tests pinning the live-editor slot collision
  (regression), the active-session gate, and cleanup ordering.
- forkDiffCache.test.ts (new): 2 tests for the drawer cache
  invalidation contract via fetchComparison simulation.

Verified end-to-end in browser: P2 (close+reopen drawer triggered an
identical second batch of per-item get fetches), P1#1 (new-session send
from a fork synced localStorage.workspace to root and posted chat to
/api/w/local/...), P1#2 (raw_app slot for workspace=local correctly
follows the visible session across A→B→A switches while both stay
warm-mounted). svelte-check 0 errors; touched test suites green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:22:50 +02:00
Ruben Fiszelandrubenfiszel 32b4771f19 chore(main): release 1.713.1 (#9389)
* chore(main): release 1.713.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-01 09:10:53 +02:00
Ruben FiszelandClaude Opus 4.8 9d9c5038ce fix(api): handle multi-version scripts when removing granular ACL (#9388)
* fix(api): handle multi-version scripts when removing granular ACL

remove_granular_acl built a CTE whose `old` subquery selects one row per
matching table row, then read it back via the scalar subquery
`(SELECT old_write FROM old)` in the RETURNING clause. For the `script`
table the PK is (workspace_id, hash), so a path with multiple deployed
versions yields several rows sharing the same (workspace_id, path). When
two or more versions carried the ACL key, `old` returned multiple rows and
PostgreSQL rejected the scalar subquery with "more than one row returned by
a subquery used as an expression", making it impossible to remove an ACL
entry from a script's permissions panel.

All versions share the same extra_perms value (the UPDATE applies to every
matching row), so any single row's old_write is representative. Add
`LIMIT 1` to the scalar subquery. Other kinds are unaffected because they
have a unique constraint on (workspace_id, path/name).

Introduced by b3603d872 (#7365). Add a regression test reproducing the
multi-version case at the SQL level.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: remove granular ACL regression test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: trim ACL fix comment

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 06:55:24 +00:00
Ruben Fiszelandrubenfiszel b16828d480 chore(main): release 1.713.0 (#9369)
* chore(main): release 1.713.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-31 08:43:37 +00:00
Ruben FiszelandClaude Opus 4.8 edf340c4d4 fix(security): re-pin cached hub scripts to CVE-patched versions (+ HUB_BASE_URL override for cache mode) (#9387)
* feat(cache): allow overriding hub base url via env in `cache` mode

The `windmill cache hubPaths.json` prebuild step (run in the Dockerfile) never
connects to the DB, so HUB_BASE_URL stays at its compiled default
(https://hub.windmill.dev) — unlike server/worker modes which load it from the
DB global setting. This made it impossible to point the prebuild cache step at
a private or staging hub.

Read HUB_BASE_URL from the environment at the start of cache_hub_scripts and
store it into the existing HUB_BASE_URL ArcSwap (the same static the hub fetch
functions read). No effect unless the env var is set and non-empty; server and
worker modes are unchanged (they still use the DB setting).

This also enables validating hub-script dependency changes end-to-end against a
local fake-hub before pushing to the real hub.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): re-pin cached hub scripts to patched versions

windmill-integrations#133 was merged and pushed to the hub, minting new
versions with regenerated (CVE-free) lockfiles. Bump the hubPaths.json pins so
the prebuild cache step (`windmill cache`) fetches the patched lockfiles instead
of the old vulnerable ones (the hub serves each version_id immutably, so the old
pins keep returning the vulnerable deps until bumped).

- slackErrorHandler     19741 -> 28241
- slackRecoveryHandler   9080 -> 28239
- slackSuccessHandler   28220 -> 28240
- smtpReport             9086 -> 28242
- appReport             28076 -> 28243  (puppeteer screenshot script)
- gitInitRepo           28219 -> 28229  (already-fixed hub version; pin was stale)

Validated end-to-end against the real hub: `windmill cache` with these pins
produces a clean cache_nomount/bun (axios 1.16.1, form-data 4.0.5,
follow-redirects 1.16.0, nodemailer 8.0.10, ws 8.21.0, svelte 5.55.8,
devalue 5.8.1; basic-ftp and ip-address no longer pulled). No vulnerable
versions remain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-31 08:28:27 +00:00
d23d8374dd use libgnutls30 in slim images (#9286)
* Harden slim image security surface

* Limit slim hardening to libgnutls pin

* fix(docker): pin libgnutls30 to exact +deb12u7 in slim images

Replace the +deb12u* wildcard with the exact current security build so
builds fail loudly when Debian ships a newer patch, prompting an
explicit review/bump rather than silently floating forward.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docker): drop libgnutls30 version pin, keep explicit listing

The version pin (exact or wildcard) was not the load-bearing part of
this change. Naming libgnutls30 explicitly in apt-get install is what
forces apt to upgrade it past the base image's older pre-installed
version — transitive deps from wget/curl/git would otherwise leave it
in place because their version constraints are already satisfied.

Dropping the version specifier so each rebuild picks up the current
security build automatically, matching the PR's stated intent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(docker): drop libgnutls30 comment

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-31 08:17:37 +00:00
Ruben FiszelandClaude Opus 4.8 f0301b1605 feat(flows): preserve step/subflow worker tags under a custom-tagged flow (#9375)
* feat(flows): preserve step/subflow worker tags under a custom-tagged flow

A flow running on a custom worker tag force-propagates that tag to every
descendant step, script and nested sub-flow, overriding their own declared
tags. This made it impossible to route a specific step or sub-flow to a
different worker group. The new opt-in FlowValue.preserve_step_tags lets a
step that declares its own non-empty tag run on it; untagged steps still
inherit the flow tag. Defaults off to preserve existing behavior.

* chore: regenerate system prompts for preserve_step_tags

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(flows): nest preserve_step_tags toggle under flow worker tag setting

The toggle only affects routing when the flow has a custom worker tag, so
show it as a sub-setting of the Worker Group tag picker, visible only once a
tag is set, instead of as a standalone option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): allow step worker tag picker when preserve_step_tags is enabled

When a flow defines a worker tag, the per-step tag picker was replaced by a
read-only "Flow's WG" label. With preserve_step_tags enabled the step's own
tag is honored, so the picker must remain editable in that case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): propagate preserve_step_tags to branch and loop bodies

payload_from_modules built the synthetic RawFlow for branch/loop bodies with
a default FlowValue, dropping preserve_step_tags. Tagged steps inside a
branch or loop therefore still inherited the parent flow tag even with the
flag enabled. Thread the flag through to the synthetic FlowValue so the
behavior is consistent for nested containers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): clear preserve_step_tags when flow worker tag is removed

Avoids the flag lingering as invisible state after the flow tag (and its
toggle) are removed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): repair preserve_step_tags propagation to branch/loop bodies

The previous commit added flow.preserve_step_tags at the payload_from_modules
call sites but the parameter and FlowValue field were not actually threaded
through (a failed edit left the function unchanged), so the crate did not
compile. This completes the change: payload_from_modules takes preserve_step_tags
and sets it on the synthetic FlowValue for branch/loop bodies.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): complete preserve_step_tags propagation to branch/loop bodies

Previous two commits left windmill-worker uncompilable: payload_from_modules
received flow.preserve_step_tags at its call sites but the parameter and the
synthetic FlowValue field were not actually added. This adds the parameter,
sets preserve_step_tags on the synthetic FlowValue, and threads
flow.preserve_step_tags through all five call sites.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(flows): clear preserve_step_tags whenever the flow worker tag is removed

The flag was only reset when the Worker Group toggle was switched off, not
when the tag was cleared directly in the picker (or via the YAML editor),
leaving preserve_step_tags=true as invisible state with the advanced badge
still reporting it active. Move the cleanup into the reactive block that
already tracks the flow tag so every clear path is covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 12:10:16 +00:00
Ruben Fiszel 2ddf93de96 fix(auth): enforce monotonic privilege on user token lifecycle endpoints (#9371) 2026-05-30 11:45:42 +00:00
Ruben FiszelandClaude Opus 4.8 def01b8ff6 fix(frontend): sanitize user markdown to prevent stored XSS (#9386)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 10:40:56 +00:00
fddbe4a51c docs(skills): fix //native marker + import rules for bunnative, remove legacy nativets skill (#9382)
* docs(skills): document mandatory //native marker for bunnative and nativets

* docs(skills): clarify windmill-client is the only allowed library in native mode

* docs(skills): remove legacy nativets skill in favor of bunnative

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(skills): fix bunnative import rule - any bundleable lib, not just windmill-client

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 10:21:56 +00:00
Diego ImbertandClaude Opus 4.8 2c0c2c467f fix(apps): make public apps opt into cross-origin isolation via wm_coep (GIT-884) (#9374)
* fix(apps): make public apps opt into cross-origin isolation via wm_coep

Public app pages served at /public/* and custom paths /a/* were not
getting the COEP/COOP/CORP headers, so they were blocked when embedded
as an iframe inside a cross-origin-isolated page (e.g. another raw app,
which sets Cross-Origin-Embedder-Policy: require-corp). A nested
document loaded into a require-corp context must itself set COEP for
the iframe to load.

Rather than applying the isolation headers to all public pages (which
would also force COEP on classic apps and break subresources without
CORP, e.g. external image URLs or embeds), public apps now opt in via
a `wm_coep` query param on the embed URL:

    <iframe src="https://<domain>/public/<ws>/<secret>?wm_coep=on">

The app publish drawer gains a URL/Embed toggle: "URL" shows the plain
shareable link (param-free), "Embed" shows a ready-to-copy iframe
snippet with wm_coep baked in, so the flag is discoverable exactly when
embedding and absent otherwise.

`wm_coep` is consumed internally and stripped from the app `query`
context so it doesn't collide with app-defined params. Only params we
own are stripped (an explicit set), not the whole `wm_` prefix.

Fixes GIT-884

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* nit

* nit

* fix(apps): only bake wm_coep into embed snippet for raw apps

AppEditorHeaderDeploy is shared by the classic (AppEditorHeader) and raw
(RawAppEditorHeader) deploy drawers. The embed snippet unconditionally
appended ?wm_coep=on, which for a classic/low-code app forces COEP
require-corp on the document and breaks no-CORP cross-origin subresources
(external <img> in AppImage/AppStatCard/AppNavbar, {@html} embeds in
AppHtml, CDN import() in AppCustomComponent) — the exact regression the
opt-in design avoids.

Add a `rawApp` prop (default false); the raw header passes rawApp. The
flag is appended only for raw apps; classic apps get a plain iframe
snippet, and the wm_coep helper text is shown only for raw apps.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 10:05:19 +00:00
centdix f300a716a9 test: add global chat resource variable schedule evals (#9379) 2026-05-30 10:02:36 +00:00
hugocasaandClaude Opus 4.8 b0c3b01d31 fix(cli): preserve user drafts on sync push and permissioned-as (#9381)
CLI deploys (sync push, set-permissioned-as) went through the same
create/update endpoints as a UI "deploy from draft", which delete the
draft at that path. That silently wiped teammates' in-progress drafts on
every push. Add a transient skip_draft_deletion deploy flag (mirroring
deployment_message) that the CLI sets; the backend then skips the
DELETE FROM draft for scripts, flows, and apps. UI deploys are unchanged.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 10:02:11 +00:00
Ruben Fiszel 4b06881918 fix(ai): validate token_url for SSRF in OAuth credentials flow (#9385)
get_token_using_oauth resolved the AI OAuth resource's token_url and
POSTed to it without any SSRF validation, while base_url is validated in
get_base_url. A workspace member with resources:write could point
token_url at an internal/metadata address (e.g. 169.254.169.254),
turning the server into an authenticated blind SSRF probe.

Validate the resolved token_url with validate_url_for_ssrf before the
request, gated behind the same ALLOW_PRIVATE_AI_BASE_URLS opt-in as
base_url so private AI deployments keep working consistently for both
URL fields. ALLOW_PRIVATE_AI_BASE_URLS is now pub so windmill-api can
reuse it instead of re-parsing the env var.
2026-05-30 09:33:27 +00:00
centdix 3345837574 chore: add gpt-5.5 eval model (#9377) 2026-05-29 15:33:27 +02:00
Ruben FiszelandClaude Opus 4.7 04a08976ae fix: batch encryption-key rotation into one git-sync job (#9355)
* fix: trigger git sync for re-encrypted secrets on encryption key change

When changing a workspace encryption key, the secret variables get
re-encrypted with the new key, but the git sync was only dispatched for
the encryption_key.yaml metadata file. Repos with Secrets sync enabled
were left with stale ciphertexts until the next per-variable deployment.

Now, after the transaction commits, we also dispatch a Variable git sync
event for each re-encrypted secret so the new encrypted values are
pushed to the configured repos. Errors are logged but don't roll back
the key rotation.

Fixes WIN-1994
Fixes #9344

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: batch encryption-key rotation into one git-sync job

Workspace encryption key rotation now re-encrypts every secret variable
and then dispatches a single batched git-sync job carrying the Key event
plus one Variable item per re-encrypted secret. Repos with Secrets sync
enabled receive every new ciphertext in one commit instead of nothing
(previously only `encryption_key.yaml` was pushed) — and instead of N
separate jobs the debouncer might or might not merge.

Wires through the new `handle_deployment_metadata_batch` entry point
added in the companion EE PR; OSS has a no-op shim so the build stays
green.

Adds an integration test (`workspace_encryption_key_git_sync`) asserting
that rotating the key with 3 secret variables in scope produces exactly
one deployment-callback job whose `items` array contains the Key event
+ all 3 variable entries and `skip_secret=false`.

Fixes WIN-1994
Fixes #9344

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref for git-sync helper simplification

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: cover non-debouncing git-sync fallback on key rotation

Adds a regression test exercising a workspace whose sync script predates
hub version 28103: the rotation must still queue a legacy-format
deployment-callback job per item (encryption_key + each re-encrypted
secret) instead of silently skipping the repo. Bumps ee-repo-ref to the
EE fallback fix.

Addresses the P1 raised in the PR review (Codex/Pi/Claude): batch path
dropped git sync entirely for repos without sync-job debouncing support.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: add sqlx offline cache for encryption-key git-sync test queries

The cargo_test CI job builds with SQLX_OFFLINE=true; the two new
sqlx::query!/query_as! calls in
windmill-api-integration-tests/tests/workspace_encryption_key_git_sync.rs
had no cached entries, failing the build with E0282. Regenerated and
added only the two new query caches (no EE/feature cache loss).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref to updated EE companion PR (08e3b9b)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 05:12:35 +00:00
Ruben FiszelandClaude Opus 4.7 96a8eb63d4 disable redirect following on AI proxy client to close SSRF (#9370)
* fix(ai): disable redirect following on AI proxy client to close SSRF

The AI proxy validates the configured base_url against SSRF rules but the
shared HTTP client followed up to 10 redirects without revalidating the
hops, so a public base_url could 3xx the server into a private/internal
address (e.g. the Docker socket or cloud metadata). Disable redirect
following so the validated host is the only one the server connects to.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(ai): remove heavy redirect SSRF integration test

Drop the integration-test-level regression for redirect following; it
spins up a full API server + DB for a one-line client-config change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 04:59:47 +00:00
Ruben FiszelandClaude Opus 4.7 bb90f4ce83 fix(api): authorize and harden log-file reading endpoints (#9368)
* fix(api): don't follow symlinks when reading service log files

Defense in depth on top of the existing `..` path-traversal check in
the get_log_file handler: reject the request if the final path
component is a symlink, so a planted symlink in the logs directory
cannot be used to read arbitrary files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(api): authorize and harden the jobs_u get_log_file endpoint

The unauthenticated jobs_u get_log_file endpoint served any job's log
file to anyone who knew the job UUID, with no authorization. Gate it the
same way as get_job_logs: look up the job (the log directory name is the
job id) filtered by workspace and the caller's scope tags, and only allow
non-logged-in callers to read logs of jobs created by the anonymous user.

Also add defense in depth: refuse to read through a symlink so a planted
symlink in the logs directory cannot be used to exfiltrate arbitrary files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-29 04:50:31 +00:00
Ruben FiszelandClaude Opus 4.8 12e06bfa6b bump default sync script to hub/28238 (windmill-cli@1.712.0) for fork branch push (#9372)
The fork-branch deployment callback runs the hub script
sync-script-to-git-repo-windmill, which imports `windmill-cli` as a pinned
npm dependency and runs it in-process (it does NOT shell out to a PATH wmill).
hub/28236 pinned windmill-cli@1.706.1, whose `git-deploy --only-create-branch`
path returns early without pushing — so the fork branch was checked out
locally but never published to the remote. #9366 fixed the CLI and shipped it
as windmill-cli@1.712.0, but without a hub-script bump the running callback
still used 1.706.1.

Bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28238, which is identical to 28236
except it pins windmill-cli@1.712.0 (content + lockfile). This fixes
test_workspace_fork_creates_branch and production fork-branch creation.

Also add backend/windmill-common/src/workspaces.rs to the git-sync-test
path-gate so future script-path bumps trigger the e2e (the bump alone is not
otherwise covered by the gate).

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-29 04:36:53 +00:00
hugocasaandwindmill-internal-app[bot] 2bf11dcb15 feat(oauth): support per-provider sandbox URLs (#9358)
* feat(oauth): support per-provider sandbox URLs in registry + instance settings

* fix(oauth): polish sandbox review nits (cc lookup, header label, ee ref)

* refactor(oauth): drop dead build_oauth_clients duplicate in windmill-oauth

* refactor(oauth): derive sandbox-capable provider list from registry

* chore(docker): copy oauth_connect.json into frontend build stage

* test(oauth): cover sandbox helpers (as_sandbox, canonical_name, resolve)

* chore: update ee-repo-ref to 9297d8f790346e6a6ad540c7bca1a67f91ec11a2

This commit updates the EE repository reference after PR #595 was merged in windmill-ee-private.

Previous ee-repo-ref: 3ab3eca9ac15ebab6db991e7964bc5e48ce21f42

New ee-repo-ref: 9297d8f790346e6a6ad540c7bca1a67f91ec11a2

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-28 22:33:44 +00:00
Ruben Fiszelandrubenfiszel 889101b7f0 chore(main): release 1.712.0 (#9340)
* chore(main): release 1.712.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-28 19:52:36 +02:00
centdix 2553fbfe31 feat: add deepseek fim support (#9365) 2026-05-28 16:52:26 +00:00
Ruben Fiszel 9a659b636d fix(frontend): prevent duplicate asset node ids crashing flow graph (#9367) 2026-05-28 16:16:12 +00:00
2fdc51e629 fix(git-sync): publish fork branch on only_create_branch from the CLI (#9366)
* [ee] fix(git-sync): publish fork branch on only_create_branch from the CLI

Fixes WIN-1997. Forking a git-sync-configured workspace must push a
`wm-fork/<branch>/<id>` branch to the repo, but the integration test
`test_workspace_fork_creates_branch` failed: the fork callback job
succeeded yet no branch appeared.

Root cause: the fork-branch callback runs the sync script with
`only_create_branch: true` and no items. The hub sync script delegates
branch checkout to `wmill sync git-deploy --only-create-branch` and runs
its own in-process commit+push ONLY for the `!only_create_branch` path
(`if (!only_create_branch) git_push(...)`). #9284 had moved commit+push
out of the CLI to the caller for the GPG-cache-warmth invariant
(WIN-1974) — but it also dropped the CLI's push for the branch-only
case. A branch-only publish has no commit, so no signing is involved and
the GPG concern does not apply; with neither the CLI nor the hub script
pushing, the empty fork branch was never published.

Restore the CLI push for the `only_create_branch` path (a bare
`git push --porcelain` of the checked-out branch ref). Adds a
deterministic CLI regression test that runs `git-deploy
--only-create-branch` for a fork workspace and asserts the branch
reaches the remote with no caller-side push.

EE companion: format the fork-branch commit message with Display instead
of Debug (no more `Some("...")` leak).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd

This commit updates the EE repository reference after PR #597 was merged in windmill-ee-private.

Previous ee-repo-ref: 8b02336fcebdfae4b9d2795cbb74fa7046530bcb

New ee-repo-ref: a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-28 16:13:38 +00:00
centdix a7d85a39ff refactor: clean up ai provider proxy logic (#9360)
* refactor: clean up ai provider proxy logic

* docs: remove completed ai refactor plan

* fix: audit failed google global proxy calls
2026-05-28 16:05:04 +00:00
Ruben Fiszel 045d12043e feat(queue): duration-weighted fairness admission (#9334)
* [ee] feat(queue): duration-weighted fairness admission atomic

Add the `WORKSPACE_FAIRNESS_ADMISSION_PPM` atomic that the EE
`workspace_fairness_ee::refresh_overloaded` writes on each refresh
(see companion EE PR). The atomic is read on every pull by
`should_admit_capped` to decide whether the dispatch goes down the
standard or fairness path. Defaults to 10_000 (= admit all) so the
pre-fairness behaviour is preserved until the first refresh fires.

OSS stub in `workspace_fairness.rs` continues to return `true`
unconditionally, so non-EE builds are bit-identical.

* docs(queue): consolidate full fairness algorithm into workspace_fairness.rs

Move the algorithm doc — what "overloaded" means in worker-seconds, the
duration-weighted admission derivation, coordinated refresh structure,
audit emission, the SQL perf constraints (no params CTE, drive running
side from v2_job_runtime), and EE gating — into the OSS surface module
where it is readable without EE access. The EE file becomes implementation
only.

Also bump ee-repo-ref to the EE commit that strips the duplicate doc.

* docs(queue): clarify ADMISSION_PPM default is "admit all", not count-based

Addresses CI review (claude[bot]): the `10_000` initial value is the
"admit all" no-op default that applies before the first refresh
classifies an overloaded set — not the count-based value (which would
be `target * 10_000`). The count-based form is the empty-bucket fallback
inside `compute_admission_ppm`, a different thing.

* chore(queue): point ee-repo-ref at EE main (fairness admission merged via #593)

* fix(queue): duration-weighted admission uses unclamped service-time window

Bumps ee-repo-ref to the EE fix (windmill-ee-private#596) that sources
D_c/D_u for the admission probability from a separate 60s service-time
window of true `duration_ms`, instead of the occupancy aggregation whose
per-job contributions are clamped to the 10s occupancy window. The clamp
truncated D_c for capped jobs longer than the window, under-admitting the
duration skew (true 34s jobs → ~86% effective share instead of the target
65%). Occupancy worker-seconds still drive overload classification.

Updates the algorithm doc in workspace_fairness.rs accordingly.

Note: ee-repo-ref points at the EE feature branch; re-point to EE main
once #596 merges.
2026-05-28 14:53:05 +00:00
Ruben FiszelandClaude Opus 4.7 aea00611c4 fix(frontend): prevent MultiSelect crash on undefined value (#9364)
MultiSelect read `value.length` directly while `value` is a bindable
prop with no default, so a parent passing `undefined` (e.g. an
enum-array approval form field with no initial value via ArgInput)
threw a TypeError that blanked the entire approval page. Guard all
reads behind a `value ?? []` derived.

Fixes WIN-1996

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 14:48:54 +00:00
centdix 9e7eaf3684 feat: inject active editor into global chat (#9361) 2026-05-28 13:57:49 +00:00
Diego Imbert a9e5140995 feat: warn when custom instance db is shared across workspaces (#9359)
* feat: warn when custom instance db is shared across workspaces

* Fix leaking workspace names

* sqlx prepare
2026-05-28 13:57:22 +00:00
hugocasaandClaude Opus 4.7 c2b5ba8871 fix(cli): stop re-prompting on wmill refresh prompts (#9357)
referencesIncludeLine required the include token to be the entire
trimmed line. The wmill-default CLAUDE.md template is
`Instructions are in @AGENTS.md` — include mid-sentence — so the
migration prompt fired every run on files wmill itself wrote.

Accept the include as a whitespace-separated token on any non-comment
line.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 10:31:25 +00:00
GuilhemandClaude Opus 4.7 4efc37212a fix: infer script arg schema when deploying via AI chat (#9356)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 23:16:21 +02:00
Ruben FiszelandClaude Opus 4.7 da882c54b2 fix(frontend): close other sidebar menus when hovering Help (#9354)
The Help menu lived in a separate Menubar from the Settings/Workers/
Folders/Logs group, so melt-ui's hover-to-switch logic (which only
spans menus within the same Menubar) did not close the Help popup
when the cursor moved to a sibling group, causing menus to stack.

Merge both bottom Menubars into a single Menubar, wrapping each
group in its own flex container to preserve the visual spacing.

Fixes WIN-1993

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 20:30:34 +00:00
centdix dec58e6c4f feat: deploy raw apps from global chat (#9349)
* feat: deploy raw apps from global chat

* fix: require raw app bundle protocol

* chore: bump ui builder artifact

* docs: explain app custom path deploy handling
2026-05-27 18:55:15 +02:00
Diego Imbert f947b1dfdf fix (frontend): schedule "View runs" url (#9350) 2026-05-27 13:49:02 +00:00
centdix e29dfbaa87 test: add global chat eval coverage (#9320)
* test: improve global chat eval parity

* test: add human-style global chat evals
2026-05-27 12:19:57 +00:00
Ruben FiszelandClaude Opus 4.7 88056f8d4c fix(cli): redact encryption_key diff in stdout by default (#9347)
* fix(cli): redact encryption_key diff in stdout by default

Sync diff output previously printed the full encryption_key contents on
stdout whenever the workspace key changed locally or on the remote, which
made it easy to leak the key via shell history, CI logs, etc. Now the
diff is replaced with a redacted notice for any encryption_key change in
both prettyChanges and showConflict. Pass --show-encryption-key-diff
(also configurable via wmill.yaml's showEncryptionKeyDiff) to opt back
into the full diff.

Fixes WIN-1992

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): redact encryption_key diff with fixed-length mask

Drop the --show-encryption-key-diff opt-in and always redact: the diff
now keeps the first 5 chars of the key so rotations are still visible
(different prefixes), then replaces every remaining char with `*` so the
length of the key is preserved without leaking it.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-27 11:28:11 +00:00
Diego Imbert ae2222febf prevent path component from wrapping (#9345) 2026-05-27 07:59:02 +00:00
centdix 9752f0f909 docs: remove stale planning notes (#9342) 2026-05-27 07:19:51 +00:00
Ruben FiszelandClaude Opus 4.7 59ab038d77 fix(monitor): cleanup stale server_heartbeat background_task_state rows (#9338)
`announce_server_started` writes a `server_heartbeat:{INSTANCE_NAME}` row
on each startup. INSTANCE_NAME is a fresh random string per process, so
the row is never updated again and a new row is inserted on every
restart, growing background_task_state unboundedly.

Add an hourly monitor task that deletes server_heartbeat:* rows older
than 7 days. Older rows cannot influence check_any_server_started (which
only considers heartbeats refreshed after the restart was initiated), so
they are safe to prune.

Fixes WIN-1990.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-26 20:36:53 +00:00
Ruben Fiszelandrubenfiszel 8d72a7a4a4 chore(main): release 1.711.0 (#9337)
* chore(main): release 1.711.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-26 16:47:53 +00:00
hugocasa 6f770346fb fix(cli): handle __flow suffix when deriving the flow's Windmill path (#9333) 2026-05-26 16:40:46 +00:00
Ruben Fiszel 42d2121af9 fix(queue): duration-weighted workspace fairness signal (#9329)
* fix(queue): bump EE ref to include worker_ping fairness signal

The current ee-repo-ref.txt pointed to 31cda7c (an unrelated merge
commit on the asset-graph-view-ee branch) instead of ddc9e80, which
contains the workspace-fairness fix that switches the active-share
signal from v2_job_queue.running=true to worker_ping. As a result
cloud was still computing overload off the legacy signal, so a
workspace with many in-flight/suspended flows (lancom01-prod, with
799 suspended flows × 3 v2_job_queue bookkeeping rows each = 2397
running-true rows) was flagged as 95% of cluster activity despite
consuming zero worker slots.

Bumping to ddc9e80 picks up the worker_ping-based signal, which
naturally excludes (a) suspended jobs (no worker pinging them),
(b) zombie running-rows from dead workers, and (c) flow/flownode
orchestration rows that never run on a worker in the first place.

* test(queue): seed v2_job rows + realistic durations for fairness helpers

The new duration-weighted fairness algorithm joins v2_job_queue and
v2_job_completed to v2_job for the `kind` filter (excluding flow
bookkeeping) and reads `duration_ms` for the completed contribution.
Update the test helpers to mirror that schema:

* `insert_completed` now inserts a matching v2_job row (kind=script)
  and writes `duration_ms = 1000` with a 1-second [started_at,
  completed_at] interval, so each completed row contributes ~1
  worker-second when fully inside the refresh window.
* `insert_queued` likewise pre-inserts v2_job, sets `started_at`
  to NOW() - 1s when running=true (so running rows contribute ~1
  worker-second by the time the refresh runs), and seeds
  v2_job_runtime.ping so the running side accrues real-time worker
  seconds (the algorithm bounds end-of-interval by ping).

The zombie/suspended insert helpers are intentionally left without
v2_job rows — the new algorithm's INNER JOIN excludes them, so they
still correctly contribute zero worker-seconds.

* chore(queue): bump EE ref to duration-weighted fairness algorithm

Companion to windmill-ee-private#<TBD>: switch the EE workspace
fairness aggregation from a count-based UNION (worker_ping snapshot
+ v2_job_completed count) to a worker-seconds aggregation sourced
directly from v2_job_queue and v2_job_completed, with kind/suspend
filters mirroring handle_zombie_jobs and per-row defenses against
zombie inflation on both halves.

* chore(queue): bump EE ref for fairness perf fix (inline window_start)

* chore(queue): bump EE ref for fairness perf rewrite (driver-side flip)

* update ee ref
2026-05-26 11:47:45 +00:00
hugocasa 979b086b08 refactor(cli): fold flow test-step into flow preview --step (#9330) 2026-05-26 11:21:54 +00:00
hugocasa 36f574ff95 feat(cli): add object-storage commands and flow test-step (#9326)
* feat(cli): add object-storage commands and flow test-step

* docs(cli): clarify flow test-step doesn't recurse into aiagent tools

* fix(cli): correct failure step id in docs, handle bare flow.yaml path
2026-05-26 10:43:43 +00:00
Ruben Fiszel a6c51b146d ignore flaky fairness regression tests in CI (#9328)
`fairness_ignores_zombie_running_rows` and
`fairness_ignores_concurrency_suspended_rows` panic intermittently in CI
(both Linux and Windows runs). Mark them `#[ignore]` until the
underlying flakiness is resolved.
2026-05-26 10:22:54 +00:00
Ruben Fiszel 85a128cc34 prevent windows backend tests from running out of disk space (#9325) 2026-05-26 10:15:45 +00:00
392 changed files with 25983 additions and 6680 deletions
+1 -1
View File
@@ -28,7 +28,7 @@ ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.24/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.25/uv-installer.sh | sh && mv /usr/local/cargo/bin/uv /usr/local/bin/uv
ENV TZ=Etc/UTC
+1 -1
View File
@@ -7,7 +7,7 @@ VERSION=$1
echo "Updating versions to: $VERSION"
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/src/main.ts
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
sed -i '' -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i '' -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
+1 -1
View File
@@ -7,7 +7,7 @@ VERSION=$1
echo "Updating versions to: $VERSION"
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/backend/Cargo.toml
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/src/main.ts
sed -i -e "/^export const VERSION =/s/= .*/= \"$VERSION\";/" ${root_dirpath}/cli/src/core/constants.ts
sed -i -e "/^export const VERSION =/s/= .*/= \"v$VERSION\";/" ${root_dirpath}/benchmarks/lib.ts
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/backend/windmill-api/openapi.yaml
sed -i -e "/version: /s/: .*/: $VERSION/" ${root_dirpath}/openflow.openapi.yaml
+32 -6
View File
@@ -74,7 +74,7 @@ jobs:
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.24"
version: "0.9.25"
- uses: shivammathur/setup-php@v2
with:
@@ -98,6 +98,21 @@ jobs:
vcpkg.exe install openssl:x64-windows-static
vcpkg.exe integrate install
- name: Free disk space (post-vcpkg)
shell: pwsh
run: |
# vcpkg leaves multi-GB of buildtrees/downloads after installing openssl;
# we only need the installed/ dir for linking.
$vcpkgRoot = $env:VCPKG_INSTALLATION_ROOT
foreach ($sub in @("buildtrees", "downloads", "packages")) {
$path = Join-Path $vcpkgRoot $sub
if (Test-Path $path) {
Write-Host "Removing $path"
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue $path
}
}
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: Get runtime paths
id: runtime-paths
shell: pwsh
@@ -119,6 +134,10 @@ jobs:
cargo build --release -p windmill_duckdb_ffi_internal
New-Item -ItemType Directory -Path ..\target\debug -Force
Copy-Item target\release\windmill_duckdb_ffi_internal.dll ..\target\debug\
# duckdb is bundled (~2GB of build artifacts); the DLL is the only
# thing we need from this excluded-crate target dir.
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue target
Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: Print runtime versions and env
shell: pwsh
@@ -136,6 +155,10 @@ jobs:
echo "USERPROFILE=$env:USERPROFILE"
echo "HOME=$env:HOME"
- name: Disk space before cargo test
shell: pwsh
run: Get-PSDrive C | Select-Object Used,Free | Format-Table -AutoSize
- name: cargo test
working-directory: backend
timeout-minutes: 60
@@ -144,13 +167,16 @@ jobs:
RUST_LOG: "off"
RUST_LOG_STYLE: never
CARGO_NET_GIT_FETCH_WITH_CLI: true
CARGO_BUILD_JOBS: 12
# 16-vcpu runners with disabled PDB still hit LNK1180 ("insufficient
# disk space") at link time with 12 parallel link jobs: each test
# binary link spikes several hundred MB of transient I/O. Capping at
# 8 trades ~25% wall time for headroom on the ~75GB runner disk.
CARGO_BUILD_JOBS: 8
# backend/Cargo.toml sets split-debuginfo = "unpacked", which on
# windows-msvc is coerced to "packed": every test-binary link spawns
# the mspdbsrv.exe PDB type server and writes a large .pdb. With 12
# parallel link jobs this races the type-server cap (LNK1318 "LIMIT
# (12)") and exhausts the runner disk (LNK1180). CI needs no debug
# info, so disable PDB generation for the dev/test profiles here.
# the mspdbsrv.exe PDB type server and writes a large .pdb. CI needs
# no debug info, so disable PDB generation for the dev/test profiles
# here (avoids both LNK1318 type-server limit and PDB disk usage).
CARGO_PROFILE_DEV_SPLIT_DEBUGINFO: "off"
CARGO_PROFILE_TEST_SPLIT_DEBUGINFO: "off"
# Tests' poll-time stack frames (deep nested async fn chains in
+1 -1
View File
@@ -62,7 +62,7 @@ jobs:
node-version: "20"
- uses: astral-sh/setup-uv@v6.2.1
with:
version: "0.9.24"
version: "0.9.25"
- uses: shivammathur/setup-php@v2
with:
php-version: "8.3"
+3 -1
View File
@@ -8,6 +8,7 @@ on:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "backend/windmill-common/src/workspaces.rs"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
pull_request:
@@ -16,6 +17,7 @@ on:
- "backend/windmill-git-sync/**"
- "backend/windmill-api-integration-tests/tests/git_sync*"
- "backend/ee-repo-ref.txt"
- "backend/windmill-common/src/workspaces.rs"
- "integration_tests/test/git_sync_test.py"
- ".github/workflows/git-sync-test.yml"
@@ -49,7 +51,7 @@ jobs:
echo "$CHANGED_FILES"
# Direct git sync file changes — always relevant
if echo "$CHANGED_FILES" | grep -qE '^(backend/windmill-git-sync/|backend/windmill-api-integration-tests/tests/git_sync|integration_tests/test/git_sync|\.github/workflows/git-sync-test\.yml)'; then
if echo "$CHANGED_FILES" | grep -qE '^(backend/windmill-git-sync/|backend/windmill-api-integration-tests/tests/git_sync|backend/windmill-common/src/workspaces\.rs|integration_tests/test/git_sync|\.github/workflows/git-sync-test\.yml)'; then
echo "should_run=true" >> "$GITHUB_OUTPUT"
echo "Relevant: direct git sync file changes"
exit 0
+1
View File
@@ -106,3 +106,4 @@ $NAV --root backend callees "X" # what does X call?
- Search for existing code to reuse before writing new code
- Follow established patterns in the codebase
- Keep changes focused — don't refactor beyond what's asked
- **Never attribute work to a specific customer, account, or "requested by a customer" in repo-tracked content** (PR descriptions, commit messages, code comments, docs). Describe changes by their technical motivation instead.
+165
View File
@@ -1,5 +1,170 @@
# Changelog
## [1.718.0](https://github.com/windmill-labs/windmill/compare/v1.717.1...v1.718.0) (2026-06-05)
### Features
* **flows:** opt-in to include the stopping step's result in early-stop errors ([#9446](https://github.com/windmill-labs/windmill/issues/9446)) ([f2f0812](https://github.com/windmill-labs/windmill/commit/f2f0812a04c9256cfc8eba5e0dcf38d71d971410))
* make C# dotnet target framework configurable via DOTNET_TARGET_FRAMEWORK ([#9454](https://github.com/windmill-labs/windmill/issues/9454)) ([9a609bf](https://github.com/windmill-labs/windmill/commit/9a609bf08ac1b6157dbdfb827fc01e771d71262e))
* sandboxed daemonless container runtime via '# sandbox &lt;image&gt;' ([#9453](https://github.com/windmill-labs/windmill/issues/9453)) ([1727271](https://github.com/windmill-labs/windmill/commit/1727271e197b34026efeaf1b6561bb404a440baa))
* **sandbox:** pull/extract images with crane instead of podman ([#9455](https://github.com/windmill-labs/windmill/issues/9455)) ([7590b28](https://github.com/windmill-labs/windmill/commit/7590b281085afd1fc2774e8fb37a4c0af3aedbad))
### Bug Fixes
* distinguish canceled jobs in runs ([#9452](https://github.com/windmill-labs/windmill/issues/9452)) ([9067787](https://github.com/windmill-labs/windmill/commit/90677872f6185eb0c81e0e84a426a54653818457))
## [1.717.1](https://github.com/windmill-labs/windmill/compare/v1.717.0...v1.717.1) (2026-06-04)
### Bug Fixes
* invalidate relative-import cache when imported script changes ([#9443](https://github.com/windmill-labs/windmill/issues/9443)) ([f595787](https://github.com/windmill-labs/windmill/commit/f595787409a3fcda9278bbcf2cfcc80092f16460))
## [1.717.0](https://github.com/windmill-labs/windmill/compare/v1.716.0...v1.717.0) (2026-06-04)
### Features
* let flow AI chat create and edit sticky notes ([#9412](https://github.com/windmill-labs/windmill/issues/9412)) ([e4e0984](https://github.com/windmill-labs/windmill/commit/e4e0984e55afd3c73f1c365cd0608493a9fd87ed))
### Bug Fixes
* **cli:** push whole raw app instead of treating frontend files as scripts ([#9442](https://github.com/windmill-labs/windmill/issues/9442)) ([b5a6a1e](https://github.com/windmill-labs/windmill/commit/b5a6a1eeab663c2d6aaec2c89eab7a550cb0bb6b))
* read latest db draft for scripts/flows in global mode read tool ([#9441](https://github.com/windmill-labs/windmill/issues/9441)) ([819ba5e](https://github.com/windmill-labs/windmill/commit/819ba5e150ec9f5199919fbea50874fc156d0189))
## [1.716.0](https://github.com/windmill-labs/windmill/compare/v1.715.0...v1.716.0) (2026-06-03)
### Features
* add metadata generation model setting ([#9418](https://github.com/windmill-labs/windmill/issues/9418)) ([cf5fefb](https://github.com/windmill-labs/windmill/commit/cf5fefb521479170b9dc64b884630c4dac789931))
* auto-generate AI session names ([#9399](https://github.com/windmill-labs/windmill/issues/9399)) ([26b7270](https://github.com/windmill-labs/windmill/commit/26b727041830c9b741668a9ab73e2eb90c7cec74))
* support $f/ and $u/ import path aliases for scripts ([#9378](https://github.com/windmill-labs/windmill/issues/9378)) ([220cd35](https://github.com/windmill-labs/windmill/commit/220cd35cf799c42ebf588bc97a6d8e6f4e97c2e3))
* use metadata model for small AI tasks ([#9431](https://github.com/windmill-labs/windmill/issues/9431)) ([79178f6](https://github.com/windmill-labs/windmill/commit/79178f6f5a7c606a2e05677c6efcbdd84c608325))
### Bug Fixes
* **apps:** relock no longer reverts raw app to a stale version ([#9432](https://github.com/windmill-labs/windmill/issues/9432)) ([073857a](https://github.com/windmill-labs/windmill/commit/073857ac0a9ed54bdeac8f373f7c855fe34eb0ac))
* **security:** scope variable and resource value caches by caller identity ([#9427](https://github.com/windmill-labs/windmill/issues/9427)) ([0ba128a](https://github.com/windmill-labs/windmill/commit/0ba128afe797bd016da60563949ac3abbbfe1978))
## [1.715.0](https://github.com/windmill-labs/windmill/compare/v1.714.1...v1.715.0) (2026-06-03)
### Features
* **frontend:** add rebuild dependency map button to workspace settings ([#9424](https://github.com/windmill-labs/windmill/issues/9424)) ([3b2e748](https://github.com/windmill-labs/windmill/commit/3b2e748daf0a8ec4447c30423068df803f3f9ca2))
### Bug Fixes
* **auth:** filter script/flow listings by token scope (GHSA-2ppx-66jv-wpw5) ([#9426](https://github.com/windmill-labs/windmill/issues/9426)) ([7edf3f0](https://github.com/windmill-labs/windmill/commit/7edf3f02122e20fde1e95e0252e7bda641075326))
* **backend:** authorize single-job read endpoints by job/flow visibility ([#9416](https://github.com/windmill-labs/windmill/issues/9416)) ([89a7a37](https://github.com/windmill-labs/windmill/commit/89a7a377764086911db18252f2478f42f0e1e3ea))
* **mcp:** resolve MCP resource token via caller RLS + SSRF-guard url ([#9428](https://github.com/windmill-labs/windmill/issues/9428)) ([8053266](https://github.com/windmill-labs/windmill/commit/8053266f88bd4c94fc86278412df5a0beeed5e77))
* **nsjail:** precompile python stdlib + raise download rlimit_as ([#9429](https://github.com/windmill-labs/windmill/issues/9429)) ([7031744](https://github.com/windmill-labs/windmill/commit/7031744a199f0bf8b8e35043afa959977e5ecdbd))
* omit temperature for gpt-5+ and o-series models on all providers ([#9422](https://github.com/windmill-labs/windmill/issues/9422)) ([11d1ad9](https://github.com/windmill-labs/windmill/commit/11d1ad9a872d2ec2f14cde35708c84a0c7bdc172))
## [1.714.1](https://github.com/windmill-labs/windmill/compare/v1.714.0...v1.714.1) (2026-06-02)
### Bug Fixes
* **backend:** route //native TypeScript previews to native workers (WIN-2007) ([#9407](https://github.com/windmill-labs/windmill/issues/9407)) ([73edebc](https://github.com/windmill-labs/windmill/commit/73edebc833a981488a8ea116f4f13c020a011a6f))
* **nsjail:** raise python download fd limit for --compile-bytecode (WIN-2009) ([#9414](https://github.com/windmill-labs/windmill/issues/9414)) ([9e6559a](https://github.com/windmill-labs/windmill/commit/9e6559a6f688cc8d982277b19920219ea6d0fd8e))
* **triggers:** prevent Zoom challenge handler from being used as a signing oracle ([#9413](https://github.com/windmill-labs/windmill/issues/9413)) ([ab2a15b](https://github.com/windmill-labs/windmill/commit/ab2a15b2a859096eabde718bf6e60289ae187118))
## [1.714.0](https://github.com/windmill-labs/windmill/compare/v1.713.1...v1.714.0) (2026-06-02)
### Features
* add global ai chat test tools ([#9391](https://github.com/windmill-labs/windmill/issues/9391)) ([5c20d6b](https://github.com/windmill-labs/windmill/commit/5c20d6b4f79f2ccc1987ce7fdaf74e6b8f697846))
* add workspace datatable tools to global AI chat mode ([#9395](https://github.com/windmill-labs/windmill/issues/9395)) ([943ef6e](https://github.com/windmill-labs/windmill/commit/943ef6eb2089f4b744cfa7945ce47f7f3b361ec7))
* **flow-ai:** constrain flow-group colors to the NoteColor palette ([#9343](https://github.com/windmill-labs/windmill/issues/9343)) ([e4213c1](https://github.com/windmill-labs/windmill/commit/e4213c1ab8c448f492f372580f5c9df37e33fffc))
* **frontend:** surface local drafts in drawer editors with an unsaved-changes banner ([#9335](https://github.com/windmill-labs/windmill/issues/9335)) ([075faab](https://github.com/windmill-labs/windmill/commit/075faabf3bba16a10a02ae3973008e5a13473085))
* handle CTRL_BREAK_EVENT for graceful shutdown on Windows ([#9400](https://github.com/windmill-labs/windmill/issues/9400)) ([2e14456](https://github.com/windmill-labs/windmill/commit/2e1445616a412c5112ad2247b4087c7ddc218845))
* refine ask-user-question chat display and keyboard nav ([#9392](https://github.com/windmill-labs/windmill/issues/9392)) ([1275487](https://github.com/windmill-labs/windmill/commit/1275487f028d4c74a9eeb18981ed05c225505be0))
* sessions page with isolated AI chat + flow editor ([#9034](https://github.com/windmill-labs/windmill/issues/9034)) ([eadeac2](https://github.com/windmill-labs/windmill/commit/eadeac248bd022c2796cfe638eb617c6143b8fc4))
### Bug Fixes
* **cli:** make encryption key push non-interactive-safe + add --skip-reencrypt-on-key-change ([#9402](https://github.com/windmill-labs/windmill/issues/9402)) ([e356bb1](https://github.com/windmill-labs/windmill/commit/e356bb1f5df92eca3fbb0ca2114b9f4c32d4c496))
* **cli:** stop git-sync promotion deploys from dropping triggers/schedules ([#9403](https://github.com/windmill-labs/windmill/issues/9403)) ([24e3ef2](https://github.com/windmill-labs/windmill/commit/24e3ef27be8498fb820c228a52febf6a0a91b487))
* **frontend:** align Monaco editor font size with text-xs ([#9161](https://github.com/windmill-labs/windmill/issues/9161)) ([de76668](https://github.com/windmill-labs/windmill/commit/de76668c10c04abe8771a8ca7bba7b2259819a1c))
* resolve username rename failing on apps with runnable deps ([#9401](https://github.com/windmill-labs/windmill/issues/9401)) ([e8ad53d](https://github.com/windmill-labs/windmill/commit/e8ad53dae92597f5a1a8b76f38a7d8c24f578a47))
### Performance Improvements
* **python:** add --compile-bytecode to uv pip install ([#9393](https://github.com/windmill-labs/windmill/issues/9393)) ([c19441b](https://github.com/windmill-labs/windmill/commit/c19441bc8cb2da064e4ad44d77dc04ab8bbb22ec))
## [1.713.1](https://github.com/windmill-labs/windmill/compare/v1.713.0...v1.713.1) (2026-06-01)
### Bug Fixes
* **api:** handle multi-version scripts when removing granular ACL ([#9388](https://github.com/windmill-labs/windmill/issues/9388)) ([9d9c503](https://github.com/windmill-labs/windmill/commit/9d9c5038ce8b0016320a670c434ef9063cb40441))
## [1.713.0](https://github.com/windmill-labs/windmill/compare/v1.712.0...v1.713.0) (2026-05-31)
### Features
* **flows:** preserve step/subflow worker tags under a custom-tagged flow ([#9375](https://github.com/windmill-labs/windmill/issues/9375)) ([f0301b1](https://github.com/windmill-labs/windmill/commit/f0301b1605cee5fba4024803555333e6fa5c40ee))
* **oauth:** support per-provider sandbox URLs ([#9358](https://github.com/windmill-labs/windmill/issues/9358)) ([2bf11dc](https://github.com/windmill-labs/windmill/commit/2bf11dcb15540c538ea2ac3cf70dcbe589060b4e))
### Bug Fixes
* **ai:** validate token_url for SSRF in OAuth credentials flow ([#9385](https://github.com/windmill-labs/windmill/issues/9385)) ([4b06881](https://github.com/windmill-labs/windmill/commit/4b06881918b76c5a411cc70b318e46efcc1393a7))
* **api:** authorize and harden log-file reading endpoints ([#9368](https://github.com/windmill-labs/windmill/issues/9368)) ([bb90f4c](https://github.com/windmill-labs/windmill/commit/bb90f4ce83a0e60af219b11c12ab4fe1d13f47a4))
* **apps:** make public apps opt into cross-origin isolation via wm_coep (GIT-884) ([#9374](https://github.com/windmill-labs/windmill/issues/9374)) ([2c0c2c4](https://github.com/windmill-labs/windmill/commit/2c0c2c467f163cd24c14c7be2db07af9cf2ce020))
* **auth:** enforce monotonic privilege on user token lifecycle endpoints ([#9371](https://github.com/windmill-labs/windmill/issues/9371)) ([2ddf93d](https://github.com/windmill-labs/windmill/commit/2ddf93de96622b2a1b2b6f59398a7a1f59360efd))
* batch encryption-key rotation into one git-sync job ([#9355](https://github.com/windmill-labs/windmill/issues/9355)) ([04a0897](https://github.com/windmill-labs/windmill/commit/04a08976aec4ba9b0516350316df303e9f96bfd3))
* **cli:** preserve user drafts on sync push and permissioned-as ([#9381](https://github.com/windmill-labs/windmill/issues/9381)) ([b0c3b01](https://github.com/windmill-labs/windmill/commit/b0c3b01d31b0ab3a6566e1f5fec60e3e230cfadb))
* **frontend:** sanitize user markdown to prevent stored XSS ([#9386](https://github.com/windmill-labs/windmill/issues/9386)) ([def01b8](https://github.com/windmill-labs/windmill/commit/def01b8ff6f331cc36ce02b947adc31c766042c4))
* **security:** re-pin cached hub scripts to CVE-patched versions (+ HUB_BASE_URL override for cache mode) ([#9387](https://github.com/windmill-labs/windmill/issues/9387)) ([edf340c](https://github.com/windmill-labs/windmill/commit/edf340c4d4f18b16b142cb7deb67afa586f10946))
## [1.712.0](https://github.com/windmill-labs/windmill/compare/v1.711.0...v1.712.0) (2026-05-28)
### Features
* add deepseek fim support ([#9365](https://github.com/windmill-labs/windmill/issues/9365)) ([2553fbf](https://github.com/windmill-labs/windmill/commit/2553fbfe31417bd985e7994eac695bf918f97ce2))
* deploy raw apps from global chat ([#9349](https://github.com/windmill-labs/windmill/issues/9349)) ([dec58e6](https://github.com/windmill-labs/windmill/commit/dec58e6c4f55062b42a752c43c89ef05903e713a))
* inject active editor into global chat ([#9361](https://github.com/windmill-labs/windmill/issues/9361)) ([9e7eaf3](https://github.com/windmill-labs/windmill/commit/9e7eaf36847ad3a004ec84e8b7d4784771b7b451))
* **queue:** duration-weighted fairness admission ([#9334](https://github.com/windmill-labs/windmill/issues/9334)) ([045d120](https://github.com/windmill-labs/windmill/commit/045d12043e7c99830ef90bc0da798c94e2094711))
* warn when custom instance db is shared across workspaces ([#9359](https://github.com/windmill-labs/windmill/issues/9359)) ([a9e5140](https://github.com/windmill-labs/windmill/commit/a9e514099585e5ee72df21bd551a223cceb20fb0))
### Bug Fixes
* **cli:** redact encryption_key diff in stdout by default ([#9347](https://github.com/windmill-labs/windmill/issues/9347)) ([88056f8](https://github.com/windmill-labs/windmill/commit/88056f8d4c91c1d14d85a08851ecf0bd97e2260d))
* **cli:** stop re-prompting on wmill refresh prompts ([#9357](https://github.com/windmill-labs/windmill/issues/9357)) ([c2b5ba8](https://github.com/windmill-labs/windmill/commit/c2b5ba8871abbbcff6de69c90e2f09fee70586c1))
* **frontend:** close other sidebar menus when hovering Help ([#9354](https://github.com/windmill-labs/windmill/issues/9354)) ([da882c5](https://github.com/windmill-labs/windmill/commit/da882c54b21e3eaf2c1d1abccd0996b243d96dce))
* **frontend:** prevent duplicate asset node ids crashing flow graph ([#9367](https://github.com/windmill-labs/windmill/issues/9367)) ([9a659b6](https://github.com/windmill-labs/windmill/commit/9a659b636d713ee8fdfbdad41c58bb3d7c79e0d9))
* **frontend:** prevent MultiSelect crash on undefined value ([#9364](https://github.com/windmill-labs/windmill/issues/9364)) ([aea0061](https://github.com/windmill-labs/windmill/commit/aea00611c41379be2afdad0eedd608c9537d03f7))
* **git-sync:** publish fork branch on only_create_branch from the CLI ([#9366](https://github.com/windmill-labs/windmill/issues/9366)) ([2fdc51e](https://github.com/windmill-labs/windmill/commit/2fdc51e62985fc755884436130bdd58e294247c8))
* infer script arg schema when deploying via AI chat ([#9356](https://github.com/windmill-labs/windmill/issues/9356)) ([4efc372](https://github.com/windmill-labs/windmill/commit/4efc37212a98571214aba135b0fbb10dc263fd4f))
* **monitor:** cleanup stale server_heartbeat background_task_state rows ([#9338](https://github.com/windmill-labs/windmill/issues/9338)) ([59ab038](https://github.com/windmill-labs/windmill/commit/59ab038d7718d8a4c25efa5928f42e1393ebbf40))
## [1.711.0](https://github.com/windmill-labs/windmill/compare/v1.710.1...v1.711.0) (2026-05-26)
### Features
* **cli:** add object-storage commands and flow test-step ([#9326](https://github.com/windmill-labs/windmill/issues/9326)) ([36f574f](https://github.com/windmill-labs/windmill/commit/36f574ff951198a4d40ee068a27d74c41ce32154))
### Bug Fixes
* **cli:** handle __flow suffix when deriving the flow's Windmill path ([#9333](https://github.com/windmill-labs/windmill/issues/9333)) ([6f77034](https://github.com/windmill-labs/windmill/commit/6f770346fb330997a836c39fba347df4c088a83c))
* **queue:** duration-weighted workspace fairness signal ([#9329](https://github.com/windmill-labs/windmill/issues/9329)) ([42d2121](https://github.com/windmill-labs/windmill/commit/42d2121af925de50f549ecb72ffb5132f5c41079))
## [1.710.1](https://github.com/windmill-labs/windmill/compare/v1.710.0...v1.710.1) (2026-05-26)
+19 -5
View File
@@ -66,6 +66,7 @@ RUN npm ci
COPY frontend .
RUN mkdir /backend
COPY /backend/windmill-api/openapi.yaml /backend/windmill-api/openapi.yaml
COPY /backend/oauth_connect.json /backend/oauth_connect.json
COPY /openflow.openapi.yaml /openflow.openapi.yaml
COPY /backend/windmill-api/build_openapi.sh /backend/windmill-api/build_openapi.sh
COPY /system_prompts/auto-generated /system_prompts/auto-generated
@@ -232,11 +233,14 @@ ENV PATH="${PATH}:/usr/local/go/bin"
ENV GO_PATH=/usr/local/go/bin/go
# Install UV
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.24/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
RUN curl --proto '=https' --tlsv1.2 -LsSf https://github.com/astral-sh/uv/releases/download/0.9.25/uv-installer.sh | sh && mv /root/.local/bin/uv /usr/local/bin/uv
# Preinstall python runtimes to temp build location (will copy with world-writable perms later)
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install 3.11
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install $LATEST_STABLE_PY
# --compile-bytecode precompiles the stdlib to .pyc so jobs don't recompile it on every run
# under the read-only nsjail runtime mount (uv >= 0.9.25). The copy below MUST preserve
# timestamps or Python's mtime-based .pyc invalidation discards these compiled files.
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install 3.11 --compile-bytecode
RUN UV_CACHE_DIR=/tmp/build_cache/uv UV_PYTHON_INSTALL_DIR=/tmp/build_cache/py_runtime uv python install $LATEST_STABLE_PY --compile-bytecode
RUN curl -sL https://deb.nodesource.com/setup_20.x | bash -
@@ -258,7 +262,7 @@ RUN export GOCACHE=/tmp/build_cache/go && \
# chmod a+rw adds read+write WITHOUT removing execute bits (755->777, 644->666)
# Note: uv python install only creates py_runtime, not uv cache - we create uv/go dirs for runtime
RUN mkdir -p /tmp/windmill/cache && \
cp -r /tmp/build_cache/* /tmp/windmill/cache/ && \
cp -r --preserve=timestamps /tmp/build_cache/* /tmp/windmill/cache/ && \
chmod -R a+rw /tmp/windmill/cache && \
rm -rf /tmp/build_cache && \
mkdir -p -m 777 /tmp/windmill/cache/uv /tmp/windmill/cache/go /tmp/windmill/cache/rustup /tmp/windmill/cache/cargo
@@ -299,10 +303,20 @@ ENV CARGO_HOME="/tmp/windmill/cache/cargo"
ENV LD_LIBRARY_PATH="."
# nsjail runtime deps and binary
RUN apt-get update && apt-get install -y libprotobuf-dev libnl-route-3-dev \
RUN apt-get update && apt-get install -y --no-install-recommends libprotobuf32 libnl-route-3-200 libnl-3-200 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
COPY --from=nsjail /nsjail/nsjail /bin/nsjail
# crane: pulls + flattens images for the sandboxed container runtime (`# sandbox <image>`).
# Single static binary — no daemon/store/root needed. See docs/docker-v2-runtime.md.
ARG CRANE_VERSION=v0.20.6
RUN arch="$(dpkg --print-architecture)"; \
case "$arch" in amd64) crane_arch=x86_64 ;; arm64) crane_arch=arm64 ;; *) echo >&2 "error: unsupported arch '$arch' for crane"; exit 1 ;; esac; \
wget -O /tmp/crane.tgz "https://github.com/google/go-containerregistry/releases/download/${CRANE_VERSION}/go-containerregistry_Linux_${crane_arch}.tar.gz" \
&& tar -xzf /tmp/crane.tgz -C /usr/local/bin crane \
&& rm /tmp/crane.tgz \
&& chmod +x /usr/local/bin/crane
WORKDIR ${APP}
RUN ln -s ${APP}/windmill /usr/local/bin/windmill
+25
View File
@@ -86,6 +86,31 @@ Global prompts should exercise workspace-level drafting behavior:
Keep deterministic validation focused on the draft contract: required draft type/path, required content snippets, forbidden draft paths, and forbidden mutating tools such as deploy/delete unless the case explicitly asks for them.
Datatable cases should set `skipJudge: true` and validate through tool-use
(`requiredToolsUsed` / `forbiddenToolsUsed`) and SQL-argument assertions
(`toolCallArgs` with `stringIncludesAnyOf`, e.g. `['select']`, `['create table']`,
`['update', 'insert into']`). Two reasons the judge is unreliable here:
- `list_datatables`, `get_datatable_table_schema`, and `exec_datatable_sql`
produce no drafts, and the global judge only sees the drafts artifact — it
scores a no-draft conversational answer as empty (same as the
`askUserQuestion` cases).
- Even a case that *does* produce a draft (a script reading the data table via
`wmill.datatable()` at runtime) is mis-judged: the judge has no datatable SDK
reference and penalizes correct `wmill.datatable()` usage as wrong. Verify the
SDK call deterministically instead — `requiredDrafts.valueIncludes: ['wmill.datatable(']`
plus forbidding `exec_datatable_sql` (keeping chat-time SQL distinct from
runtime SDK use).
`stringIncludesAnyOf` is existential over calls (at least one matching call), so a
mutation case still passes when the model mixes its UPDATE/INSERT with
verification SELECTs. The in-memory engine (`datatableSqlEngine.ts`) is stateful
within a case — writes persist, so a model that re-queries to verify its
CREATE/UPDATE sees the change and does not loop. But the engine is best-effort
(SELECT returns all rows of the referenced/first table with no WHERE/projection),
so still never assert specific returned row values. Seed data via
`workspace.datatables` in the `initial` fixture (see README).
## Deterministic validation
Use deterministic validation only for hard failures such as:
+39 -8
View File
@@ -56,7 +56,7 @@ bun run cli -- run flow flow-test4-order-processing-loop --model opus
bun run cli -- run flow flow-test0-sum-two-numbers --models haiku,opus,4o
bun run cli -- run flow flow-test0-sum-two-numbers --runs 3 --verbose
bun run cli -- run flow --record
GEMINI_API_KEY=... bun run cli -- run app app-test1-counter-create --model gemini-pro
GEMINI_API_KEY=... bun run cli -- run app app-test1-counter-create --model gemini-3-flash-preview
WMILL_AI_EVAL_BACKEND_URL=http://127.0.0.1:8000 bun run cli -- run flow --backend-validation preview
bun run cli -- run global global-test1-script-create
bun run cli -- run cli bun-hello-script
@@ -88,15 +88,16 @@ Today:
- `sonnet`
- `opus`
- `4o`
- `gemini-flash`
- `gemini-pro`
- `gpt-5.5`
- `gemini-3-flash-preview`
- `gemini-3.1-pro-preview`
- `deepseek-v4-flash`
- `deepseek-v4-pro`
Notes:
- the command also prints accepted alias spellings such as `gpt-4o`, `claude-opus-4.6`, and `claude-haiku-4.5`
- frontend modes (`flow`, `script`, `app`, `global`) can use Anthropic, OpenAI, and Gemini-backed aliases
- the command also prints accepted alias spellings such as `gpt-4o`, `gpt-55`, `claude-opus-4.6`, and `claude-haiku-4.5`
- frontend modes (`flow`, `script`, `app`, `global`) can use Anthropic, OpenAI, Gemini, and DeepSeek-backed aliases
- `cli` mode always uses the Anthropic agent SDK, so only Anthropic aliases are valid there
- the judge model is separate and currently defaults to `claude-sonnet-4-6`
@@ -142,6 +143,32 @@ For `global` mode, `validate` can express draft-level requirements such as:
- required or forbidden draft counts
- forbidden draft paths
Global initial fixtures can also seed `liveEditorDrafts` with `type`,
`storagePath`, `effectivePath`, and `value` fields. These drafts emulate the
currently open script, flow, or raw app editor so cases can test prompts that
refer to "this" or the "current" item.
Global (and flow) initial fixtures can seed `workspace.datatables` so the
`list_datatables`, `get_datatable_table_schema`, and `exec_datatable_sql` tools
return seeded data during evals. Each entry is
`{ datatable_name, schemas: { <schema>: { <table>: { columns, rows? } } } }`.
SQL runs through a small in-memory engine (`datatableSqlEngine.ts`), not a real
database. Writes are **stateful within a case**: `CREATE`/`DROP`/`INSERT`/`UPDATE`/
`DELETE` mutate the seeded datatable in place, so a later `list_datatables`,
`get_datatable_table_schema`, `SELECT`, or `information_schema` query reflects them
— this is what stops a model from looping when it re-queries to verify a write.
The engine is best-effort: `SELECT` returns all rows of the referenced (or first)
table with no WHERE filtering/projection/joins, `WHERE` on UPDATE/DELETE supports
`col = value` predicates joined by `AND`, and anything unparseable is a no-op
success. So validate datatable cases through tool-use and SQL-argument assertions
(`requiredToolsUsed`, `stringIncludesAnyOf`) — not through exact returned row
values. An empty/absent `datatables` seed makes `list_datatables` return `[]`,
which is what the "no datatable configured" blocking cases rely on.
Set `WMILL_AI_EVAL_DISABLE_ACTIVE_EDITOR_CONTEXT=1` to run those cases with
the old behavior where the live editor is only discoverable through
`list_workspace_items`.
App fixtures can also include an optional `datatables.json` file at the fixture root.
For `flow` mode, an `initial` fixture can also include a benchmark workspace catalog of
@@ -189,11 +216,15 @@ If `--record` is used, the CLI also appends one compact JSON line to:
Each recorded line contains:
- run metadata (`createdAt`, `gitSha`, `mode`, `runModel`, `judgeModel`)
- suite totals (`caseCount`, `attemptCount`, `passedAttempts`, `passRate`, `averageDurationMs`, `averageJudgeScore`)
- average token usage (`averageTokenUsagePerAttempt`)
- per-case metrics under `cases[]` (`averageDurationMs`, `averageJudgeScore`, `averageTokenUsagePerAttempt`, pass rate)
- suite totals (`caseCount`, `attemptCount`, `passedAttempts`, `passRate`, `averageDurationMs`, `averagePassedDurationMs`, `averageJudgeScore`)
- average token usage (`averageTokenUsagePerAttempt`, `averageTokenUsagePerPassedAttempt`)
- per-case metrics under `cases[]` (`averageDurationMs`, `averagePassedDurationMs`, `averageJudgeScore`, `averageTokenUsagePerAttempt`, `averageTokenUsagePerPassedAttempt`, pass rate)
- `failedCaseIds`
The CLI headline duration and token averages use passed attempts only.
All-attempt averages are still recorded to make failures auditable without
letting failed attempts skew success cost comparisons.
Example:
- summary: `ai_evals/results/2026-04-09T09-40-33.051Z__flow.json`
@@ -7,8 +7,12 @@ import {
prepareGlobalSystemMessage,
prepareGlobalUserMessage,
} from "../../../../../frontend/src/lib/components/copilot/chat/global/core";
import { globalDraftStore } from "../../../../../frontend/src/lib/components/copilot/chat/global/draftStore.svelte";
import {
clearGlobalDrafts,
listGlobalDrafts,
} from "../../../../../frontend/src/lib/components/copilot/chat/global/userDraftAdapter";
import type { Tool as ProductionTool } from "../../../../../frontend/src/lib/components/copilot/chat/shared";
import { UserDraft } from "../../../../../frontend/src/lib/userDraft.svelte";
import type { ModeRunContext } from "../../../../core/types";
import type { GlobalDraftState } from "../../../../core/validators";
import type { WindmillBackendSettings } from "../../../../core/windmillBackendSettings";
@@ -24,6 +28,21 @@ const MUTATING_GLOBAL_TOOLS = new Set([
"deploy_workspace_item",
"delete_workspace_item",
]);
const DISABLE_ACTIVE_EDITOR_CONTEXT_ENV =
"WMILL_AI_EVAL_DISABLE_ACTIVE_EDITOR_CONTEXT";
const LIVE_EDITOR_ITEM_KINDS = {
script: "script",
flow: "flow",
app: "raw_app",
} as const;
export interface GlobalLiveEditorDraftFixture {
type: keyof typeof LIVE_EDITOR_ITEM_KINDS;
storagePath?: string;
effectivePath?: string;
value?: unknown;
}
export interface GlobalEvalResult {
success: boolean;
@@ -38,6 +57,7 @@ export interface GlobalEvalResult {
export interface GlobalEvalOptions {
workspaceFixtures?: BenchmarkWorkspaceRunnables;
liveEditorDrafts?: GlobalLiveEditorDraftFixture[];
model?: string;
maxIterations?: number;
provider?: AIProvider;
@@ -55,19 +75,26 @@ export async function runGlobalEval(
options.workspaceRoot ??
(await mkdtemp(join(tmpdir(), "wmill-frontend-global-benchmark-")));
globalDraftStore.clearDrafts(workspaceRoot);
clearGlobalDrafts(workspaceRoot);
registerBenchmarkWorkspaceRunnables(workspaceRoot, options.workspaceFixtures ?? {});
seedLiveEditorDrafts(workspaceRoot, options.liveEditorDrafts ?? []);
try {
const model = options.model ?? "claude-haiku-4-5-20251001";
const injectActiveEditorContext =
process.env[DISABLE_ACTIVE_EDITOR_CONTEXT_ENV] !== "1";
const rawResult = await runEval({
userPrompt,
systemMessage: prepareGlobalSystemMessage(),
userMessage: prepareGlobalUserMessage(userPrompt),
userMessage: prepareGlobalUserMessage(
userPrompt,
[],
injectActiveEditorContext ? { workspace: workspaceRoot } : {},
),
tools: getGlobalEvalTools(),
helpers: {},
apiKey,
getOutput: () => ({ drafts: globalDraftStore.listDrafts(workspaceRoot) }),
getOutput: () => ({ drafts: listGlobalDrafts(workspaceRoot) }),
onAssistantMessageStart: options.runContext?.onAssistantMessageStart,
onAssistantToken: options.runContext?.onAssistantChunk,
onAssistantMessageEnd: options.runContext?.onAssistantMessageEnd,
@@ -94,7 +121,8 @@ export async function runGlobalEval(
tokenUsage: rawResult.tokenUsage,
};
} finally {
globalDraftStore.clearDrafts(workspaceRoot);
clearGlobalDrafts(workspaceRoot);
clearLiveEditorDrafts(workspaceRoot, options.liveEditorDrafts ?? []);
unregisterBenchmarkWorkspaceRunnables(workspaceRoot);
if (!options.workspaceRoot) {
await rm(workspaceRoot, { recursive: true, force: true });
@@ -102,6 +130,36 @@ export async function runGlobalEval(
}
}
function seedLiveEditorDrafts(
workspace: string,
fixtures: GlobalLiveEditorDraftFixture[],
): void {
for (const fixture of fixtures) {
const itemKind = LIVE_EDITOR_ITEM_KINDS[fixture.type];
const storagePath = fixture.storagePath ?? fixture.effectivePath ?? "";
if (fixture.value !== undefined) {
UserDraft.save(itemKind, storagePath, fixture.value, { workspace });
}
UserDraft.setLiveEditorDraft({
workspace,
itemKind,
storagePath,
effectivePath: fixture.effectivePath ?? fixture.storagePath,
});
}
}
function clearLiveEditorDrafts(
workspace: string,
fixtures: GlobalLiveEditorDraftFixture[],
): void {
for (const fixture of fixtures) {
const itemKind = LIVE_EDITOR_ITEM_KINDS[fixture.type];
const storagePath = fixture.storagePath ?? fixture.effectivePath ?? "";
UserDraft.clearLiveEditorDraft(itemKind, { workspace, storagePath });
}
}
function getGlobalEvalTools(): ProductionTool<{}>[] {
return (globalTools as ProductionTool<{}>[]).map((tool) => {
if (!MUTATING_GLOBAL_TOOLS.has(tool.def.function.name)) {
@@ -21,9 +21,9 @@ describe("proxy helpers", () => {
describe("resolveEvalModelProvider", () => {
it("infers googleai from Gemini model ids", () => {
expect(resolveEvalModelProvider("gemini-2.5-flash")).toEqual({
expect(resolveEvalModelProvider("gemini-3-flash-preview")).toEqual({
provider: "googleai",
model: "gemini-2.5-flash",
model: "gemini-3-flash-preview",
});
});
@@ -35,9 +35,11 @@ describe("resolveEvalModelProvider", () => {
});
it("preserves an explicit provider", () => {
expect(resolveEvalModelProvider("gemini-2.5-pro", "googleai")).toEqual({
expect(
resolveEvalModelProvider("gemini-3.1-pro-preview", "googleai"),
).toEqual({
provider: "googleai",
model: "gemini-2.5-pro",
model: "gemini-3.1-pro-preview",
});
});
});
@@ -0,0 +1,262 @@
import { describe, expect, it } from 'bun:test'
import { applyDatatableSql, type BenchmarkDatatableSeed } from './datatableSqlEngine'
function makeDatatable(): BenchmarkDatatableSeed {
return {
datatable_name: 'main',
schemas: {
public: {
orders: {
columns: { id: 'int4', customer_id: 'int4', total: 'numeric', status: 'text' },
rows: [
{ id: 1, customer_id: 1, total: 42.5, status: 'shipped' },
{ id: 2, customer_id: 2, total: 19.99, status: 'pending' },
{ id: 3, customer_id: 1, total: 88, status: 'shipped' }
]
},
customers: {
columns: { id: 'int4', name: 'text' },
rows: [{ id: 1, name: 'Alice' }]
}
}
}
}
}
describe('SELECT', () => {
it('returns the referenced table rows', () => {
const dt = makeDatatable()
expect(applyDatatableSql(dt, 'SELECT id, name FROM customers').rows).toEqual([
{ id: 1, name: 'Alice' }
])
})
it('falls back to the first table when no known table is referenced', () => {
const dt = makeDatatable()
expect(applyDatatableSql(dt, 'select 1').rows).toHaveLength(3)
})
it('resolves a schema-qualified table', () => {
const dt = makeDatatable()
expect(applyDatatableSql(dt, 'SELECT * FROM public.customers').rows).toEqual([
{ id: 1, name: 'Alice' }
])
})
})
describe('CREATE TABLE', () => {
it('adds a table with parsed columns, skipping table constraints and FK clauses', () => {
const dt = makeDatatable()
const result = applyDatatableSql(
dt,
'CREATE TABLE public.refunds (\n order_id int4 NOT NULL REFERENCES public.orders(id),\n amount numeric(10,2),\n PRIMARY KEY (order_id)\n)'
)
expect(result.rows).toEqual([])
expect(dt.schemas.public.refunds).toEqual({
columns: { order_id: 'int4', amount: 'numeric(10,2)' },
rows: []
})
})
it('defaults an unqualified table to the public schema', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'CREATE TABLE notes (id int4, body text)')
expect(dt.schemas.public.notes.columns).toEqual({ id: 'int4', body: 'text' })
})
it('is a no-op for an existing table with IF NOT EXISTS', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'CREATE TABLE IF NOT EXISTS public.orders (x int4)')
expect(Object.keys(dt.schemas.public.orders.columns)).toContain('status')
})
})
describe('DROP TABLE', () => {
it('removes the table', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'DROP TABLE IF EXISTS public.customers')
expect(dt.schemas.public.customers).toBeUndefined()
})
})
describe('INSERT', () => {
it('appends a row using an explicit column list', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "INSERT INTO customers (id, name) VALUES (2, 'Bob')")
expect(dt.schemas.public.customers.rows).toContainEqual({ id: 2, name: 'Bob' })
})
it('infers columns from the table when none are given, and appends multiple tuples', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "INSERT INTO customers VALUES (2, 'Bob'), (3, 'Carol')")
expect(dt.schemas.public.customers.rows).toHaveLength(3)
})
it('returns the inserted rows when RETURNING is present', () => {
const dt = makeDatatable()
const result = applyDatatableSql(
dt,
"INSERT INTO customers (id, name) VALUES (2, 'Bob') RETURNING *"
)
expect(result.rows).toEqual([{ id: 2, name: 'Bob' }])
})
})
describe('UPDATE', () => {
it('updates only the rows matching an equality WHERE', () => {
const dt = makeDatatable()
const result = applyDatatableSql(
dt,
"UPDATE public.orders SET status = 'shipped' WHERE id = 2"
)
expect(result.rows).toEqual([])
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 2)?.status).toBe('shipped')
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 1)?.status).toBe('shipped')
})
it('strips a Postgres cast in the WHERE value', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "UPDATE orders SET status = 'done' WHERE id = 2::int4")
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 2)?.status).toBe('done')
})
it('matches multiple AND predicates including a numeric literal', () => {
const dt = makeDatatable()
applyDatatableSql(
dt,
"UPDATE orders SET status = 'done' WHERE customer_id = 2 AND total = 19.99"
)
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 2)?.status).toBe('done')
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 1)?.status).toBe('shipped')
})
it('updates every row when there is no WHERE', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "UPDATE orders SET status = 'archived'")
expect(dt.schemas.public.orders.rows?.every((r) => r.status === 'archived')).toBe(true)
})
it('returns the affected rows when RETURNING is present', () => {
const dt = makeDatatable()
const result = applyDatatableSql(
dt,
"UPDATE orders SET status = 'shipped' WHERE id = 2 RETURNING *"
)
expect(result.rows).toHaveLength(1)
expect(result.rows[0]).toMatchObject({ id: 2, status: 'shipped' })
})
it('affects no rows when the WHERE clause cannot be parsed', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "UPDATE orders SET status = 'x' WHERE total > 20")
expect(dt.schemas.public.orders.rows?.some((r) => r.status === 'x')).toBe(false)
})
})
describe('DELETE', () => {
it('removes only the matching rows', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'DELETE FROM orders WHERE id = 2')
expect(dt.schemas.public.orders.rows?.map((r) => r.id)).toEqual([1, 3])
})
it('returns the removed rows when RETURNING is present', () => {
const dt = makeDatatable()
const result = applyDatatableSql(dt, 'DELETE FROM orders WHERE id = 2 RETURNING *')
expect(result.rows).toEqual([{ id: 2, customer_id: 2, total: 19.99, status: 'pending' }])
})
})
describe('writes are reflected by later reads', () => {
it('UPDATE then SELECT sees the new value (the verify-loop fix)', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "UPDATE orders SET status = 'shipped' WHERE id = 2")
const seen = applyDatatableSql(dt, 'SELECT * FROM orders').rows
expect(seen.find((r) => r.id === 2)?.status).toBe('shipped')
})
it('INSERT then SELECT sees the new row', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "INSERT INTO customers (id, name) VALUES (9, 'Zed')")
const seen = applyDatatableSql(dt, 'SELECT * FROM customers').rows
expect(seen).toContainEqual({ id: 9, name: 'Zed' })
})
it('CREATE then SELECT on the new table returns its (empty) rows', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'CREATE TABLE public.refunds (order_id int4, amount numeric)')
expect(applyDatatableSql(dt, 'SELECT * FROM refunds').rows).toEqual([])
})
})
describe('system-catalog queries reflect the current tables/columns', () => {
it('lists current tables (including a freshly created one) via information_schema.tables', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'CREATE TABLE public.refunds (order_id int4)')
const rows = applyDatatableSql(
dt,
"SELECT table_name FROM information_schema.tables WHERE table_name = 'refunds'"
).rows
expect(rows.map((r) => r.table_name)).toContain('refunds')
})
it('does not list a dropped table', () => {
const dt = makeDatatable()
applyDatatableSql(dt, 'DROP TABLE public.customers')
const rows = applyDatatableSql(dt, 'SELECT table_name FROM information_schema.tables').rows
expect(rows.map((r) => r.table_name)).not.toContain('customers')
})
it('reports columns via information_schema.columns', () => {
const dt = makeDatatable()
const rows = applyDatatableSql(
dt,
"SELECT column_name FROM information_schema.columns WHERE table_name = 'orders'"
).rows
expect(rows.map((r) => r.column_name)).toContain('status')
})
})
describe('parser robustness (string/paren-aware splitting)', () => {
it('does not treat the word "returning" inside a string value as a RETURNING clause', () => {
const dt = makeDatatable()
const result = applyDatatableSql(
dt,
"INSERT INTO customers (id, name) VALUES (5, 'is returning soon')"
)
expect(result.rows).toEqual([])
expect(dt.schemas.public.customers.rows).toContainEqual({ id: 5, name: 'is returning soon' })
})
it('does not split on the word "where" inside a SET string value', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "UPDATE orders SET status = 'ship where ordered' WHERE id = 2")
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 2)?.status).toBe('ship where ordered')
expect(dt.schemas.public.orders.rows?.find((r) => r.id === 1)?.status).toBe('shipped')
})
it('keeps INSERT tuples intact when a value contains a function call', () => {
const dt = makeDatatable()
applyDatatableSql(dt, "INSERT INTO customers (id, name) VALUES (6, coalesce(NULL, 'x'))")
expect(dt.schemas.public.customers.rows).toHaveLength(2)
expect(dt.schemas.public.customers.rows?.[1]).toMatchObject({ id: 6 })
})
it('CREATE TABLE ignores a trailing semicolon-separated statement', () => {
const dt = makeDatatable()
applyDatatableSql(
dt,
'CREATE TABLE public.refunds (id int4, amount numeric); INSERT INTO refunds VALUES (1, 5)'
)
expect(dt.schemas.public.refunds.columns).toEqual({ id: 'int4', amount: 'numeric' })
expect(dt.schemas.public.refunds.rows).toEqual([])
})
})
describe('unparseable statements are a safe no-op', () => {
it('returns [] and does not throw', () => {
const dt = makeDatatable()
expect(applyDatatableSql(dt, 'VACUUM ANALYZE').rows).toEqual([])
expect(applyDatatableSql(dt, 'GRANT SELECT ON orders TO someone').rows).toEqual([])
})
})
@@ -0,0 +1,541 @@
/**
* A deliberately small, best-effort SQL engine for the benchmark datatable mock.
*
* This is NOT a real SQL implementation — it exists only so that writes a model
* issues during an eval (`CREATE TABLE`, `INSERT`, `UPDATE`, `DELETE`, `DROP`)
* become visible to its later reads (`list_datatables`, `get_datatable_table_schema`,
* `SELECT`). Without that, a model that re-queries to verify a write sees stale
* seed data, concludes the write failed, and loops until it exhausts its turns.
*
* It parses only the common statement shapes models produce. Anything it cannot
* parse is a no-op success (it never throws) — behavioral evals assert that the
* right statement was issued, not its exact data effects. Notable limits:
* - `SELECT` returns all rows of the referenced (or first) table — no WHERE
* filtering, projection, joins, or aggregation.
* - `WHERE` supports `col = value` predicates joined by `AND` only; an
* unparseable WHERE on UPDATE/DELETE affects zero rows (never the whole table).
*/
/** One seeded datatable table: its columns (col -> compact_type) and optional rows. */
export interface BenchmarkDatatableTableSeed {
columns: Record<string, string>
rows?: Record<string, unknown>[]
}
/** A seeded datatable: `datatable_name` plus a `schema -> table -> seed` map. */
export interface BenchmarkDatatableSeed {
datatable_name: string
schemas: {
[schema: string]: {
[table: string]: BenchmarkDatatableTableSeed
}
}
}
export interface DatatableSqlResult {
rows: Record<string, unknown>[]
}
const DEFAULT_SCHEMA = 'public'
type ParsedRef = { schema: string; table: string }
type Predicate = { column: string; value: unknown }
/**
* Apply one SQL statement to `datatable` IN PLACE and return the result rows.
* SELECT returns the referenced/first table's rows; a mutation returns its
* affected rows when it has a RETURNING clause, otherwise `[]`.
*/
export function applyDatatableSql(
datatable: BenchmarkDatatableSeed,
sql: string
): DatatableSqlResult {
const statement = stripTrailingSemicolon(sql.trim())
if (/^\s*(with|select)\b/i.test(statement)) {
return { rows: selectRows(datatable, statement) }
}
if (/^\s*create\s+table\b/i.test(statement)) {
return { rows: applyCreateTable(datatable, statement) }
}
if (/^\s*drop\s+table\b/i.test(statement)) {
return { rows: applyDropTable(datatable, statement) }
}
if (/^\s*insert\s+into\b/i.test(statement)) {
return { rows: applyInsert(datatable, statement) }
}
if (/^\s*update\b/i.test(statement)) {
return { rows: applyUpdate(datatable, statement) }
}
if (/^\s*delete\s+from\b/i.test(statement)) {
return { rows: applyDelete(datatable, statement) }
}
return { rows: [] }
}
// ============= Reads =============
function selectRows(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const fromRef = sql.match(/\bfrom\s+([a-zA-Z_"][\w."]*)/i)?.[1]
if (fromRef) {
const catalog = catalogRows(datatable, fromRef)
if (catalog) {
return catalog
}
}
const table = fromRef ? resolveTable(datatable, fromRef) : undefined
const seed = table ?? firstTable(datatable)
return seed?.rows ?? []
}
/**
* Synthesize rows for a system-catalog query so a model verifying a `CREATE`/`DROP`
* via `information_schema.tables` / `.columns` (or `pg_tables`) sees the current
* tables/columns instead of fallback data. WHERE is not applied, so the model gets
* the full set and finds (or no longer finds) the table it just changed.
* Returns `undefined` for non-catalog refs so normal table resolution proceeds.
*/
function catalogRows(
datatable: BenchmarkDatatableSeed,
ref: string
): Record<string, unknown>[] | undefined {
const normalized = ref.toLowerCase().replace(/"/g, '')
const name = normalized.split('.').pop()
const isCatalog = normalized.includes('information_schema.') || normalized.startsWith('pg_')
if (!isCatalog) {
return undefined
}
const tables = allTables(datatable)
if (name === 'tables' || name === 'pg_tables') {
return tables.map(({ schema, table }) => ({
table_schema: schema,
table_name: table,
schemaname: schema,
tablename: table
}))
}
if (name === 'columns') {
return tables.flatMap(({ schema, table, seed }) =>
Object.entries(seed.columns).map(([column, type]) => ({
table_schema: schema,
table_name: table,
column_name: column,
data_type: type
}))
)
}
return undefined
}
function allTables(
datatable: BenchmarkDatatableSeed
): { schema: string; table: string; seed: BenchmarkDatatableTableSeed }[] {
return Object.entries(datatable.schemas).flatMap(([schema, tables]) =>
Object.entries(tables).map(([table, seed]) => ({ schema, table, seed }))
)
}
// ============= DDL =============
function applyCreateTable(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const head = sql.match(
/^\s*create\s+table\s+(?:if\s+not\s+exists\s+)?([a-zA-Z_"][\w."]*)/i
)
// The first top-level paren group is the column-definition list; using it (rather
// than a greedy `(...)` capture) ignores any trailing `;`-separated statement.
const columnText = extractParenGroups(sql)[0]
if (!head || columnText === undefined) {
return []
}
const { schema, table } = parseRef(head[1])
const existing = datatable.schemas[schema]?.[table]
if (existing) {
return []
}
const columns: Record<string, string> = {}
for (const rawDef of splitTopLevel(columnText)) {
const def = rawDef.trim()
if (!def || isTableConstraint(def)) {
continue
}
const tokens = def.split(/\s+/)
const column = unquoteIdentifier(tokens[0])
if (!column) {
continue
}
columns[column] = tokens[1] ?? 'text'
}
if (!datatable.schemas[schema]) {
datatable.schemas[schema] = {}
}
datatable.schemas[schema][table] = { columns, rows: [] }
return []
}
function applyDropTable(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const match = sql.match(
/^\s*drop\s+table\s+(?:if\s+exists\s+)?([a-zA-Z_"][\w."]*)/i
)
if (!match) {
return []
}
const { schema, table } = parseRef(match[1])
if (datatable.schemas[schema]?.[table]) {
delete datatable.schemas[schema][table]
}
return []
}
// ============= DML =============
function applyInsert(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const { body, returning } = splitOffReturning(sql)
const match = body.match(
/^\s*insert\s+into\s+([a-zA-Z_"][\w."]*)\s*(?:\(([^)]*)\))?\s*values\s*([\s\S]+)$/i
)
if (!match) {
return []
}
const table = resolveTable(datatable, match[1])
if (!table) {
return []
}
const columns = match[2]
? splitTopLevel(match[2]).map((entry) => unquoteIdentifier(entry.trim()))
: Object.keys(table.columns)
const inserted: Record<string, unknown>[] = []
for (const tuple of extractParenGroups(match[3])) {
const values = splitTopLevel(tuple).map((entry) => parseValue(entry))
const row: Record<string, unknown> = {}
columns.forEach((column, index) => {
row[column] = values[index]
})
inserted.push(row)
}
table.rows ??= []
table.rows.push(...inserted)
return returning ? inserted : []
}
function applyUpdate(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const { body, returning } = splitOffReturning(sql)
const match = body.match(/^\s*update\s+([a-zA-Z_"][\w."]*)\s+set\s+([\s\S]+)$/i)
if (!match) {
return []
}
const table = resolveTable(datatable, match[1])
if (!table) {
return []
}
let assignmentText = match[2]
let whereText: string | undefined
const whereMatch = maskForClauseScan(assignmentText).match(/\swhere\s/i)
if (whereMatch && whereMatch.index !== undefined) {
whereText = assignmentText.slice(whereMatch.index + whereMatch[0].length)
assignmentText = assignmentText.slice(0, whereMatch.index)
}
const predicates = parsePredicates(whereText)
if (predicates === null) {
return []
}
const assignments: Record<string, unknown> = {}
for (const entry of splitTopLevel(assignmentText)) {
const pair = entry.match(/^\s*([a-zA-Z_"][\w."]*)\s*=\s*([\s\S]+?)\s*$/)
if (pair) {
assignments[lastIdentifier(pair[1])] = parseValue(pair[2])
}
}
const affected = (table.rows ?? []).filter((row) => rowMatches(row, predicates))
for (const row of affected) {
Object.assign(row, assignments)
}
return returning ? affected : []
}
function applyDelete(
datatable: BenchmarkDatatableSeed,
sql: string
): Record<string, unknown>[] {
const { body, returning } = splitOffReturning(sql)
const match = body.match(/^\s*delete\s+from\s+([a-zA-Z_"][\w."]*)\s*([\s\S]*)$/i)
if (!match) {
return []
}
const table = resolveTable(datatable, match[1])
if (!table) {
return []
}
const whereText = match[2].replace(/^\s*where\s+/i, '').trim() || undefined
const predicates = parsePredicates(whereText)
if (predicates === null) {
return []
}
const rows = table.rows ?? []
const removed = rows.filter((row) => rowMatches(row, predicates))
table.rows = rows.filter((row) => !rowMatches(row, predicates))
return returning ? removed : []
}
// ============= Parsing helpers =============
function resolveTable(
datatable: BenchmarkDatatableSeed,
ref: string
): BenchmarkDatatableTableSeed | undefined {
const { schema, table } = parseRef(ref)
const direct = datatable.schemas[schema]?.[table]
if (direct) {
return direct
}
// Bare table name: fall back to searching every schema for a matching table.
if (!ref.includes('.')) {
for (const tables of Object.values(datatable.schemas)) {
if (tables[table]) {
return tables[table]
}
}
}
return undefined
}
function firstTable(
datatable: BenchmarkDatatableSeed
): BenchmarkDatatableTableSeed | undefined {
for (const tables of Object.values(datatable.schemas)) {
for (const seed of Object.values(tables)) {
return seed
}
}
return undefined
}
function parseRef(ref: string): ParsedRef {
const parts = ref.split('.').map(unquoteIdentifier)
if (parts.length >= 2) {
return { schema: parts[parts.length - 2], table: parts[parts.length - 1] }
}
return { schema: DEFAULT_SCHEMA, table: parts[0] }
}
/** A WHERE clause with no parseable form returns `null`; absent WHERE returns `[]` (match all). */
function parsePredicates(whereText: string | undefined): Predicate[] | null {
if (whereText === undefined || whereText.trim() === '') {
return []
}
const predicates: Predicate[] = []
for (const part of whereText.split(/\s+and\s+/i)) {
const match = part.match(/^\s*([a-zA-Z_"][\w."]*)\s*=\s*([\s\S]+?)\s*$/)
if (!match) {
return null
}
predicates.push({ column: lastIdentifier(match[1]), value: parseValue(match[2]) })
}
return predicates
}
function rowMatches(row: Record<string, unknown>, predicates: Predicate[]): boolean {
return predicates.every((predicate) => looseEquals(row[predicate.column], predicate.value))
}
function looseEquals(left: unknown, right: unknown): boolean {
if (left === null || left === undefined) {
return right === null || right === undefined
}
if (typeof left === 'number' && typeof right === 'number') {
return left === right
}
return String(left) === String(right)
}
function parseValue(raw: string): unknown {
// Drop a trailing Postgres cast (e.g. `2::int4`) before interpreting the literal.
const token = raw.trim().replace(/::\s*[a-zA-Z_][\w]*(\([^)]*\))?\s*$/, '').trim()
const stringMatch = token.match(/^'([\s\S]*)'$/)
if (stringMatch) {
return stringMatch[1].replace(/''/g, "'")
}
if (/^-?\d+(\.\d+)?$/.test(token)) {
return Number(token)
}
if (/^true$/i.test(token)) {
return true
}
if (/^false$/i.test(token)) {
return false
}
if (/^null$/i.test(token)) {
return null
}
return token
}
function splitOffReturning(sql: string): { body: string; returning: boolean } {
const match = maskForClauseScan(sql).match(/\sreturning\s/i)
if (!match || match.index === undefined) {
return { body: sql, returning: false }
}
return { body: sql.slice(0, match.index), returning: true }
}
/**
* A same-length copy of `sql` with the contents of single-quoted strings and
* parenthesized groups blanked to spaces, so a top-level keyword scan
* (WHERE / RETURNING) cannot match inside a string literal or a subquery. Index
* positions in the result map 1:1 back onto the original.
*/
function maskForClauseScan(sql: string): string {
let masked = ''
let depth = 0
let inString = false
for (let i = 0; i < sql.length; i++) {
const char = sql[i]
if (inString) {
if (char === "'") {
if (sql[i + 1] === "'") {
masked += ' '
i++
continue
}
inString = false
}
masked += ' '
continue
}
if (char === "'") {
inString = true
masked += ' '
} else if (char === '(') {
depth++
masked += ' '
} else if (char === ')') {
depth = Math.max(0, depth - 1)
masked += ' '
} else {
masked += depth > 0 ? ' ' : char
}
}
return masked
}
/**
* Inner text of each top-level `( ... )` group in `input`, honoring nested parens
* (e.g. `now()`, `numeric(10,2)`) and single-quoted strings. Used for the CREATE
* column-definition group and INSERT value tuples.
*/
function extractParenGroups(input: string): string[] {
const groups: string[] = []
let depth = 0
let inString = false
let current = ''
for (let i = 0; i < input.length; i++) {
const char = input[i]
if (inString) {
current += char
if (char === "'") {
if (input[i + 1] === "'") {
current += input[++i]
} else {
inString = false
}
}
continue
}
if (char === "'") {
inString = true
current += char
} else if (char === '(') {
depth++
if (depth === 1) {
current = ''
} else {
current += char
}
} else if (char === ')') {
depth = Math.max(0, depth - 1)
if (depth === 0) {
groups.push(current)
current = ''
} else {
current += char
}
} else if (depth > 0) {
current += char
}
}
return groups
}
/** Split on commas that are not inside parentheses or single-quoted strings. */
function splitTopLevel(input: string): string[] {
const parts: string[] = []
let depth = 0
let inString = false
let current = ''
for (let i = 0; i < input.length; i++) {
const char = input[i]
if (inString) {
current += char
if (char === "'") {
if (input[i + 1] === "'") {
current += input[++i]
} else {
inString = false
}
}
continue
}
if (char === "'") {
inString = true
current += char
} else if (char === '(') {
depth++
current += char
} else if (char === ')') {
depth = Math.max(0, depth - 1)
current += char
} else if (char === ',' && depth === 0) {
parts.push(current)
current = ''
} else {
current += char
}
}
if (current.trim() !== '') {
parts.push(current)
}
return parts
}
function isTableConstraint(def: string): boolean {
return /^(primary\s+key|foreign\s+key|constraint|unique|check|exclude|like)\b/i.test(def)
}
function unquoteIdentifier(identifier: string): string {
const trimmed = identifier.trim()
const quoted = trimmed.match(/^"([\s\S]*)"$/)
return quoted ? quoted[1] : trimmed
}
/** For a qualified reference like `orders.id`, keep only the final identifier. */
function lastIdentifier(reference: string): string {
const parts = reference.split('.')
return unquoteIdentifier(parts[parts.length - 1])
}
function stripTrailingSemicolon(sql: string): string {
return sql.replace(/;\s*$/, '')
}
+108 -2
View File
@@ -1,7 +1,14 @@
import { randomUUID } from 'node:crypto'
import type { CompletedJob, Flow, Script } from '../../../frontend/src/lib/gen'
import type { ScriptLang } from '../../../frontend/src/lib/gen/types.gen'
import type {
DataTableTables,
DataTableTableSchema,
ScriptLang
} from '../../../frontend/src/lib/gen/types.gen'
import { buildScriptLintResult } from './core/script/preview'
import { applyDatatableSql, type BenchmarkDatatableSeed } from './datatableSqlEngine'
export type { BenchmarkDatatableSeed, BenchmarkDatatableTableSeed } from './datatableSqlEngine'
const BENCHMARK_TIMESTAMP = '1970-01-01T00:00:00.000Z'
@@ -25,6 +32,7 @@ export interface BenchmarkWorkspaceFlow {
export interface BenchmarkWorkspaceRunnables {
scripts?: BenchmarkWorkspaceScript[]
flows?: BenchmarkWorkspaceFlow[]
datatables?: BenchmarkDatatableSeed[]
}
type BenchmarkCompletedJob = CompletedJob & { type: 'CompletedJob' }
@@ -48,7 +56,12 @@ export function registerBenchmarkWorkspaceRunnables(
runnables: BenchmarkWorkspaceRunnables
): void {
benchmarkWorkspaces.add(workspace)
benchmarkWorkspaceRunnables.set(workspace, runnables)
// Datatables are mutated in place by exec_datatable_sql (a write must be visible
// to later reads), so store an isolated deep copy — never mutate the caller's seed.
benchmarkWorkspaceRunnables.set(workspace, {
...runnables,
datatables: runnables.datatables ? structuredClone(runnables.datatables) : undefined
})
}
export function unregisterBenchmarkWorkspace(workspace: string): void {
@@ -161,6 +174,99 @@ export function getBenchmarkCompletedJob(
return structuredClone(entry.job)
}
// ============= Datatables (best-effort in-memory SQL) =============
/**
* Project the seeded datatables down to the `list_datatable_tables` response:
* `datatable_name` + `schema -> table_names`, with no column detail.
* Returns `null` for a non-benchmark workspace so callers can fall through to
* the real backend; an empty seed yields `[]`.
*/
export function listBenchmarkDatatables(workspace: string): DataTableTables[] | null {
const runnables = benchmarkWorkspaceRunnables.get(workspace)
if (!runnables) {
return null
}
return (runnables.datatables ?? []).map((datatable) => ({
datatable_name: datatable.datatable_name,
schemas: Object.fromEntries(
Object.entries(datatable.schemas).map(([schema, tables]) => [schema, Object.keys(tables)])
)
}))
}
export function getBenchmarkDatatableSchema(input: {
workspace: string
datatableName: string
schemaName: string
tableName: string
}): DataTableTableSchema {
const runnables = benchmarkWorkspaceRunnables.get(input.workspace)
const datatable = (runnables?.datatables ?? []).find(
(entry) => entry.datatable_name === input.datatableName
)
if (!datatable) {
// Message MUST match the production `isDatatableNotConfiguredError` regex
// (/datatable\s+\S+\s+not found/i in datatableTools.ts) so the
// get_datatable_table_schema not-configured mapping is actually exercised.
throw new Error(`datatable "${input.datatableName}" not found`)
}
const table = datatable.schemas?.[input.schemaName]?.[input.tableName]
if (!table) {
throw new Error(
`table "${input.schemaName}.${input.tableName}" not found in datatable "${input.datatableName}"`
)
}
return {
datatable_name: input.datatableName,
schema_name: input.schemaName,
table_name: input.tableName,
columns: table.columns
}
}
/**
* Execute SQL against a seeded datatable through the best-effort in-memory engine
* (`applyDatatableSql`). Writes (CREATE/INSERT/UPDATE/DELETE/DROP) mutate the
* stored datatable in place so a later list/schema/SELECT reflects them; SELECT
* (and RETURNING) yield rows, other statements yield `[]`. Creates a benchmark
* completed job and returns its id, like `runBenchmarkScriptPreview`.
*/
export function runBenchmarkDatatableSql(input: {
workspace: string
datatableName: string
sql: string
}): string {
const runnables = benchmarkWorkspaceRunnables.get(input.workspace)
const datatable = (runnables?.datatables ?? []).find(
(entry) => entry.datatable_name === input.datatableName
)
const rows = datatable ? applyDatatableSql(datatable, input.sql).rows : []
return createBenchmarkCompletedJob({
workspace: input.workspace,
jobKind: 'preview',
success: true,
args: { database: `datatable://${input.datatableName}` },
result: rows
})
}
/**
* Mirror `JobService.getCompletedJobResultMaybe` for benchmark workspaces — the
* shape `pollJobResult` consumes. The job is created synchronously before
* polling, so it is always present and completed.
*/
export function getBenchmarkCompletedJobResultMaybe(input: {
workspace: string
id: string
}): { success: boolean; completed: boolean; result: unknown } {
const job = getBenchmarkCompletedJob(input.workspace, input.id)
if (!job) {
throw new Error(`Job "${input.id}" not found in benchmark workspace`)
}
return { success: job.success, completed: true, result: job.result }
}
export function runBenchmarkScriptPreview(input: {
workspace: string
requestBody: {
@@ -0,0 +1,175 @@
import { afterEach, beforeEach, describe, expect, it } from 'bun:test'
import {
getBenchmarkCompletedJobResultMaybe,
getBenchmarkDatatableSchema,
listBenchmarkDatatables,
registerBenchmarkWorkspaceRunnables,
resetBenchmarkMockBackend,
runBenchmarkDatatableSql,
type BenchmarkWorkspaceRunnables
} from './mockBackend'
const WORKSPACE = 'benchmark-datatable-ws'
// Mirrors the production `isDatatableNotConfiguredError` regex in
// datatableTools.ts. The schema mock's "not configured" message MUST match it,
// otherwise the not-configured mapping in get_datatable_table_schema is silently
// untested.
const NOT_CONFIGURED_RE = /datatable\s+\S+\s+not found/i
const SEED: BenchmarkWorkspaceRunnables = {
datatables: [
{
datatable_name: 'main',
schemas: {
public: {
orders: {
columns: { id: 'int', total: 'numeric' },
rows: [
{ id: 1, total: 10 },
{ id: 2, total: 20 }
]
},
customers: {
columns: { id: 'int', name: 'text' },
rows: [{ id: 1, name: 'alice' }]
}
}
}
}
]
}
beforeEach(() => resetBenchmarkMockBackend())
afterEach(() => resetBenchmarkMockBackend())
describe('listBenchmarkDatatables', () => {
it('returns null for a non-benchmark workspace (caller falls through to real backend)', () => {
expect(listBenchmarkDatatables('unregistered')).toBeNull()
})
it('returns [] for a registered workspace with no datatables seed', () => {
registerBenchmarkWorkspaceRunnables(WORKSPACE, {})
expect(listBenchmarkDatatables(WORKSPACE)).toEqual([])
})
it('projects seeded datatables to schema -> table names only (no columns)', () => {
registerBenchmarkWorkspaceRunnables(WORKSPACE, SEED)
expect(listBenchmarkDatatables(WORKSPACE)).toEqual([
{ datatable_name: 'main', schemas: { public: ['orders', 'customers'] } }
])
})
})
describe('getBenchmarkDatatableSchema', () => {
beforeEach(() => registerBenchmarkWorkspaceRunnables(WORKSPACE, SEED))
it('returns the columns for a seeded table', () => {
expect(
getBenchmarkDatatableSchema({
workspace: WORKSPACE,
datatableName: 'main',
schemaName: 'public',
tableName: 'orders'
})
).toEqual({
datatable_name: 'main',
schema_name: 'public',
table_name: 'orders',
columns: { id: 'int', total: 'numeric' }
})
})
it('throws a not-configured error matching the production regex for an unknown datatable', () => {
let error: Error | undefined
try {
getBenchmarkDatatableSchema({
workspace: WORKSPACE,
datatableName: 'ghost',
schemaName: 'public',
tableName: 'orders'
})
} catch (e) {
error = e as Error
}
expect(error).toBeDefined()
expect(error!.message).toMatch(NOT_CONFIGURED_RE)
})
it('throws a table-not-found error that does NOT match the datatable-not-configured regex', () => {
// The datatable IS configured; only the table is missing. Production maps
// this to a generic "error getting schema", not the blocking message.
let error: Error | undefined
try {
getBenchmarkDatatableSchema({
workspace: WORKSPACE,
datatableName: 'main',
schemaName: 'public',
tableName: 'ghost'
})
} catch (e) {
error = e as Error
}
expect(error).toBeDefined()
expect(error!.message).not.toMatch(NOT_CONFIGURED_RE)
})
})
describe('runBenchmarkDatatableSql + getBenchmarkCompletedJobResultMaybe', () => {
beforeEach(() => registerBenchmarkWorkspaceRunnables(WORKSPACE, SEED))
function exec(sql: string): { success: boolean; completed: boolean; result: unknown } {
const jobId = runBenchmarkDatatableSql({ workspace: WORKSPACE, datatableName: 'main', sql })
return getBenchmarkCompletedJobResultMaybe({ workspace: WORKSPACE, id: jobId })
}
it('returns the canned rows of the table named in a SELECT FROM clause', () => {
expect(exec('SELECT * FROM customers')).toEqual({
success: true,
completed: true,
result: [{ id: 1, name: 'alice' }]
})
})
it('falls back to the first seeded table when the SELECT references no known table', () => {
expect(exec('select 1').result).toEqual([
{ id: 1, total: 10 },
{ id: 2, total: 20 }
])
})
it('returns [] success for DDL and DML statements without RETURNING', () => {
expect(exec('CREATE TABLE foo (id int)').result).toEqual([])
expect(exec('INSERT INTO orders VALUES (3, 30)').result).toEqual([])
expect(exec('update orders set total = 0').result).toEqual([])
})
it('reflects a write in a later SELECT, isolated from the shared seed', () => {
exec('UPDATE orders SET total = 999 WHERE id = 1')
expect((exec('SELECT * FROM orders').result as Record<string, unknown>[])).toContainEqual({
id: 1,
total: 999
})
// Registration deep-clones the seed, so the shared SEED const stays pristine.
expect(SEED.datatables![0].schemas.public.orders.rows).toContainEqual({ id: 1, total: 10 })
})
it('reflects a CREATE in list_datatables and get_datatable_table_schema', () => {
exec('CREATE TABLE public.refunds (order_id int4, amount numeric)')
expect(listBenchmarkDatatables(WORKSPACE)?.[0].schemas.public).toContain('refunds')
expect(
getBenchmarkDatatableSchema({
workspace: WORKSPACE,
datatableName: 'main',
schemaName: 'public',
tableName: 'refunds'
}).columns
).toEqual({ order_id: 'int4', amount: 'numeric' })
})
it('throws for an unknown job id', () => {
expect(() =>
getBenchmarkCompletedJobResultMaybe({ workspace: WORKSPACE, id: 'does-not-exist' })
).toThrow()
})
})
@@ -34,15 +34,19 @@ vi.mock('$lib/gen', async () => {
const actual = await vi.importActual<any>('$lib/gen')
const {
getBenchmarkCompletedJob,
getBenchmarkCompletedJobResultMaybe,
getBenchmarkDatatableSchema,
getBenchmarkFlowByPath,
getBenchmarkScriptByHash,
getBenchmarkScriptByPath,
hasBenchmarkWorkspace,
listBenchmarkDatatables,
listBenchmarkFlows,
listBenchmarkScripts,
createBenchmarkHttpTrigger,
createBenchmarkSchedule,
previewBenchmarkSchedule,
runBenchmarkDatatableSql,
runBenchmarkFlowByPath,
runBenchmarkScriptPreview
} = await import('./mockBackend')
@@ -79,6 +83,16 @@ vi.mock('$lib/gen', async () => {
}
return actual.ScriptService.getScriptByPath(data)
},
getScriptByPathWithDraft: async (data: { workspace: string; path: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
const script = getBenchmarkScriptByPath(data.workspace, data.path)
if (!script) {
throw new Error(`Script "${data.path}" not found in benchmark workspace`)
}
return script
}
return actual.ScriptService.getScriptByPathWithDraft(data)
},
getScriptByHash: async (data: { workspace: string; hash: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
const script = getBenchmarkScriptByHash(data.workspace, data.hash)
@@ -108,6 +122,26 @@ vi.mock('$lib/gen', async () => {
return flow
}
return actual.FlowService.getFlowByPath(data)
},
getFlowByPathWithDraft: async (data: { workspace: string; path: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
const flow = getBenchmarkFlowByPath(data.workspace, data.path)
if (!flow) {
throw new Error(`Flow "${data.path}" not found in benchmark workspace`)
}
return flow
}
return actual.FlowService.getFlowByPathWithDraft(data)
},
getFlowLatestVersion: async (data: { workspace: string; path: string }) => {
if (hasBenchmarkWorkspace(data.workspace)) {
const flow = getBenchmarkFlowByPath(data.workspace, data.path)
if (!flow) {
throw new Error(`Flow "${data.path}" not found in benchmark workspace`)
}
return { id: 1 }
}
return actual.FlowService.getFlowLatestVersion(data)
}
}),
JobService: wrapService(actual.JobService, {
@@ -119,13 +153,27 @@ vi.mock('$lib/gen', async () => {
args?: Record<string, unknown>
path?: string
}
}) =>
hasBenchmarkWorkspace(data.workspace)
? runBenchmarkScriptPreview({
workspace: data.workspace,
requestBody: data.requestBody ?? {}
})
: actual.JobService.runScriptPreview(data),
}) => {
if (!hasBenchmarkWorkspace(data.workspace)) {
return actual.JobService.runScriptPreview(data)
}
const requestBody = data.requestBody ?? {}
const database = requestBody.args?.database
// Datatable SQL runs as a `postgresql` preview against `datatable://<name>`.
// Execute it through the canned-SQL mock instead of linting it as a script.
if (
requestBody.language === 'postgresql' &&
typeof database === 'string' &&
database.startsWith('datatable://')
) {
return runBenchmarkDatatableSql({
workspace: data.workspace,
datatableName: database.slice('datatable://'.length),
sql: requestBody.content ?? ''
})
}
return runBenchmarkScriptPreview({ workspace: data.workspace, requestBody })
},
runFlowByPath: async (data: {
workspace: string
path: string
@@ -147,7 +195,31 @@ vi.mock('$lib/gen', async () => {
return job
}
return actual.JobService.getJob(data)
}
},
getCompletedJobResultMaybe: async (data: { workspace: string; id: string }) =>
hasBenchmarkWorkspace(data.workspace)
? getBenchmarkCompletedJobResultMaybe({ workspace: data.workspace, id: data.id })
: actual.JobService.getCompletedJobResultMaybe(data)
}),
WorkspaceService: wrapService(actual.WorkspaceService, {
listDataTableTables: async (data: { workspace: string }) =>
hasBenchmarkWorkspace(data.workspace)
? (listBenchmarkDatatables(data.workspace) ?? [])
: actual.WorkspaceService.listDataTableTables(data),
getDataTableTableSchema: async (data: {
workspace: string
datatableName: string
schemaName: string
tableName: string
}) =>
hasBenchmarkWorkspace(data.workspace)
? getBenchmarkDatatableSchema({
workspace: data.workspace,
datatableName: data.datatableName,
schemaName: data.schemaName,
tableName: data.tableName
})
: actual.WorkspaceService.getDataTableTableSchema(data)
}),
ScheduleService: wrapService(actual.ScheduleService, {
existsSchedule: async (data: { workspace: string; path: string }) =>
+11
View File
@@ -8,6 +8,9 @@
args:
a: 4
b: 5
toolExpect:
requiredToolsUsed:
- test_run_flow
judgeChecklist:
- "the flow takes `a` and `b` as inputs"
- "the main step is named `sum_numbers`"
@@ -25,6 +28,9 @@
args:
a: 2
b: 3
toolExpect:
requiredToolsUsed:
- test_run_flow
judgeChecklist:
- "the flow takes `a` and `b` as inputs"
- "the main step is named `sum_numbers`"
@@ -42,6 +48,9 @@
args:
a: 7
b: 8
toolExpect:
requiredToolsUsed:
- test_run_flow
judgeChecklist:
- "the parent flow takes `a` and `b` as inputs"
- "the main step is named `call_add_numbers`"
@@ -426,6 +435,7 @@
- return_schedule_status
toolExpect:
requiredToolsUsed:
- test_run_flow
- create_schedule
toolCallArgs:
- tool: create_schedule
@@ -453,6 +463,7 @@
- webhook_response
toolExpect:
requiredToolsUsed:
- test_run_flow
- create_trigger
toolCallArgs:
- tool: create_trigger
+734
View File
@@ -87,3 +87,737 @@
- the flow accepts numeric inputs a and b
- the flow returns the sum of a and b
- the result stays as an AI draft and is not deployed or saved to the workspace
- id: global-test4-multi-artifact-notification-job
prompt: |-
Set up a draft stale-trial notification job.
Create a Bun script at `f/evals/global/check_stale_trials` that accepts `max_age_days`, uses mocked inline trial account data, and returns the stale trial account IDs.
Also create a weekday 09:00 UTC schedule at `f/evals/global/check_stale_trials_weekday` for that script with `max_age_days` set to 14.
Add an HTTP POST trigger at `f/evals/global/check_stale_trials_manual` with route path `evals/check-stale-trials` that runs the same script manually.
Leave everything as AI drafts only; do not deploy or save anything to the workspace.
runtime:
maxTurns: 12
validate:
draftCountExactly: 3
requiredDrafts:
- type: script
path: f/evals/global/check_stale_trials
language: bun
valueIncludes:
- max_age_days
- trial
- type: schedule
path: f/evals/global/check_stale_trials_weekday
valueIncludes:
- f/evals/global/check_stale_trials
- UTC
- "14"
- type: trigger
triggerKind: http
path: f/evals/global/check_stale_trials_manual
valueIncludes:
- evals/check-stale-trials
- f/evals/global/check_stale_trials
toolExpect:
requiredToolsUsed:
- write_script
- write_schedule
- write_trigger
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates a Bun script draft for stale trial accounts
- creates a weekday 09:00 UTC schedule draft for the script with max_age_days set to 14
- creates an HTTP POST trigger draft with route path evals/check-stale-trials for the same script
- leaves all artifacts as drafts only and does not deploy
- id: global-test5-existing-flow-inline-code-edit
prompt: |-
Update the existing flow at `f/evals/global/process_invoice`.
Only change the `calculate_total` inline code so it applies 8% tax and returns an object containing `subtotal`, `tax`, and `total`.
Leave the updated flow as an AI draft only; do not deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/process_invoice_flow.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
path: f/evals/global/process_invoice
valueIncludes:
- calculate_total
- tax
- total
toolExpect:
requiredToolsUsed:
- read_workspace_item
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- reads the existing process_invoice flow before editing it
- updates the calculate_total inline code to apply 8% tax
- returns subtotal, tax, and total from the updated flow logic
- leaves the result as an AI draft only
- id: global-test6-secret-variable-draft
prompt: |-
Create a secret variable draft at `f/evals/global/slack_bot_token`.
Use the placeholder value `xoxb-redacted-test-token` and description `Slack bot token for eval notifications`.
Do not create any resource or deploy anything.
runtime:
maxTurns: 6
validate:
draftCountExactly: 1
requiredDrafts:
- type: variable
path: f/evals/global/slack_bot_token
valueIncludes:
- Slack bot token
- "true"
forbiddenDrafts:
- type: resource
path: f/evals/global/slack_bot_token
toolExpect:
requiredToolsUsed:
- write_variable
forbiddenToolsUsed:
- write_resource
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: write_variable
field: value
stringStartsWithAnyOf:
- xoxb-redacted-test-token
skipJudge: true
judgeChecklist:
- creates exactly one secret variable draft at f/evals/global/slack_bot_token
- uses the requested placeholder value and description
- does not create a resource or deploy anything
- id: global-test7-ambiguous-app-asks-question
prompt: |-
Create a new raw app for triaging support tickets.
runtime:
maxTurns: 4
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- askUserQuestion
forbiddenToolsUsed:
- init_app
- write_app_file
- write_app_runnable
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
- id: global-test8-human-script-infer-path-language
prompt: |-
I need a small helper that formats a customer-facing welcome line.
It should take a person's name and return "Welcome aboard, <name>!".
Please just stage it as a draft for now.
runtime:
maxTurns: 8
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
valueIncludes:
- Welcome aboard
- name
toolExpect:
requiredToolsUsed:
- write_script
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates a single script draft for a welcome-line helper
- accepts a person's name as input
- returns a message containing Welcome aboard, the provided name, and an exclamation mark
- chooses a reasonable workspace path and script language without needing the user to specify them
- leaves the result as an AI draft only
- id: global-test9-human-weekday-trial-job
prompt: |-
Can you set up a draft daily job that checks a few hard-coded trial accounts and returns the ones whose trial has ended?
It should run every weekday morning around 9 in UTC with a 30 day cutoff.
Keep it as draft work only.
runtime:
maxTurns: 10
validate:
draftCountExactly: 2
requiredDrafts:
- type: script
pathIncludes:
- trial
valueIncludes:
- trial
- "30"
- type: schedule
pathIncludes:
- trial
valueIncludes:
- UTC
toolExpect:
requiredToolsUsed:
- write_script
- write_schedule
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates a script draft that checks hard-coded trial accounts
- returns the accounts whose trial has ended based on a 30 day cutoff
- creates a schedule draft for weekday mornings around 09:00 UTC
- links the schedule to the generated script
- leaves both artifacts as drafts only
- id: global-test10-human-secret-variable
prompt: |-
I need a placeholder Slack bot token stored securely for future notification work.
Use xoxb-redacted-test-token and note that it is for eval notifications.
Only prepare a draft.
runtime:
maxTurns: 6
validate:
draftCountExactly: 1
requiredDrafts:
- type: variable
pathIncludes:
- slack
valueIncludes:
- eval notifications
- "true"
toolExpect:
requiredToolsUsed:
- write_variable
forbiddenToolsUsed:
- write_resource
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: write_variable
field: value
stringStartsWithAnyOf:
- xoxb-redacted-test-token
skipJudge: true
judgeChecklist:
- creates a single secret variable draft for the Slack bot token placeholder
- uses the requested placeholder value
- includes a note or description that it is for eval notifications
- does not create a resource or deploy anything
- id: global-test11-human-existing-flow-informal-edit
prompt: |-
There is an invoice processing flow in this workspace.
Can you adjust its total calculation so it adds 8% tax and returns subtotal, tax, and total?
Keep the change as a draft.
initial: ai_evals/fixtures/frontend/global/initial/process_invoice_flow.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
pathIncludes:
- invoice
valueIncludes:
- calculate_total
- tax
- total
toolExpect:
requiredToolsUsed:
- read_workspace_item
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- finds and edits the existing invoice processing flow without the user providing its exact path
- updates the total calculation to apply 8% tax
- returns subtotal, tax, and total from the updated flow logic
- leaves the result as an AI draft only
- id: global-test12-current-live-script-edit
prompt: |-
The script I have open formats greetings.
Can you update this script so it uppercases the name before greeting them and ends with an exclamation mark?
Keep it as draft work.
initial: ai_evals/fixtures/frontend/global/initial/current_greeting_live_script.json
runtime:
maxTurns: 8
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
path: f/evals/global/current_greeting
language: bun
valueIncludes:
- toUpperCase
- "!"
forbiddenDrafts:
- type: script
path: f/evals/global/format_greeting
- type: script
path: f/evals/global/format_greeting_archive
toolExpect:
requiredToolsUsed:
- read_workspace_item
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- resolves "this script" to the active live editor script instead of another similarly named workspace script
- updates the greeting logic to uppercase the provided name
- returns a greeting ending with an exclamation mark
- leaves the result as a draft only
- id: global-test13-current-live-flow-edit
prompt: |-
I have the invoice flow open.
In the current flow, update the total calculation to add 8% tax and return subtotal, tax, and total.
Keep the change as a draft.
initial: ai_evals/fixtures/frontend/global/initial/current_invoice_live_flow.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
path: f/evals/global/current_invoice_flow
valueIncludes:
- calculate_total
- tax
- total
forbiddenDrafts:
- type: flow
path: f/evals/global/process_invoice
- type: flow
path: f/evals/global/process_refund
toolExpect:
requiredToolsUsed:
- read_workspace_item
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- resolves "current flow" to the active live editor flow
- does not edit the similarly named deployed invoice or refund flows
- updates the calculate_total logic to apply 8% tax
- returns subtotal, tax, and total from the updated flow logic
- leaves the result as a draft only
- id: global-test14-current-without-live-editor-asks-question
prompt: |-
Please update this script so it returns `ok`.
Keep it as a draft.
runtime:
maxTurns: 4
validate:
draftCountExactly: 0
toolExpect:
forbiddenToolsUsed:
- write_script
- edit_script
- write_flow
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- asks which script to update when the user refers to "this script" without selected or active editor context
- does not guess a path or create a new script draft
- id: global-test15-human-postgres-resource
prompt: |-
I'm wiring the eval reporting database into this workspace.
Can you stage a Postgres connection for it in the shared evals/global folder?
Use host `reports-db.internal`, port 5432, database `evals_reporting`, user `report_reader`, and password `pg-redacted-reporting-password`.
Keep the credentials safe.
This is just draft work for now.
runtime:
maxTurns: 10
validate:
draftCountExactly: 2
requiredDrafts:
- type: variable
pathStartsWith: f/evals/global/
pathIncludes:
- evals
- global
- report
- password
valueIncludes:
- "true"
- report
- type: resource
pathStartsWith: f/evals/global/
pathIncludes:
- evals
- global
- report
valueIncludes:
- postgres
- reports-db.internal
- "5432"
- evals_reporting
- report_reader
- "$var:"
valueExcludes:
- pg-redacted-reporting-password
toolExpect:
requiredToolsUsed:
- write_variable
- search_resource_types
- write_resource
forbiddenToolsUsed:
- write_schedule
- write_trigger
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: write_variable
field: value
stringStartsWithAnyOf:
- pg-redacted-reporting-password
skipJudge: true
judgeChecklist:
- creates a Postgres resource draft for the eval reporting database
- creates a secret variable draft for the database password
- puts the drafts in sensible eval/global reporting-related paths
- uses the requested host, port, database, and user
- references the secret variable from the resource instead of embedding the password
- leaves the work as a draft only
- id: global-test16-human-visible-variable
prompt: |-
We keep reusing a 30 day trial cutoff in eval notification jobs.
Can you stage that as a normal workspace variable in the shared evals/global folder, with a short description so people know what it controls?
It is not a secret.
runtime:
maxTurns: 6
validate:
draftCountExactly: 1
requiredDrafts:
- type: variable
pathStartsWith: f/evals/global/
pathIncludes:
- evals
- global
- trial
valueIncludes:
- "30"
- "false"
- trial
toolExpect:
requiredToolsUsed:
- write_variable
forbiddenToolsUsed:
- write_resource
- write_schedule
- write_trigger
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates exactly one non-secret variable draft for the trial cutoff
- stores the value 30
- chooses a sensible eval/global path related to trials or notifications
- includes a useful description of what the value controls
- does not create resources, schedules, triggers, or deployed workspace changes
- id: global-test17-human-schedule-existing-helper
prompt: |-
The workspace already has a report digest helper.
Can you stage a weekday 8:30 AM UTC run for it with `dry_run` turned on?
I only want the schedule draft for review.
initial: ai_evals/fixtures/frontend/global/initial/report_digest_script.json
runtime:
maxTurns: 8
validate:
draftCountExactly: 1
requiredDrafts:
- type: schedule
pathIncludes:
- digest
valueIncludes:
- f/evals/global/send_report_digest
- UTC
- dry_run
- "true"
toolExpect:
requiredToolsUsed:
- list_workspace_items
- write_schedule
forbiddenToolsUsed:
- write_script
- write_flow
- write_resource
- write_variable
- write_trigger
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- finds the existing report digest helper rather than creating a new script or flow
- creates one schedule draft for that helper
- schedules it for weekdays around 08:30 UTC
- passes dry_run as true
- leaves only the schedule draft for review
- id: global-test18-human-slack-resource-with-secret
prompt: |-
I'm preparing Slack notifications for eval failures.
Can you stage a Slack connection in the shared evals/global folder?
The bot token is `xoxb-redacted-test-token`; keep it safe.
Don't deploy anything yet.
runtime:
maxTurns: 8
validate:
draftCountExactly: 2
requiredDrafts:
- type: variable
pathStartsWith: f/evals/global/
pathIncludes:
- evals
- global
- slack
- token
valueIncludes:
- "true"
- type: resource
pathStartsWith: f/evals/global/
pathIncludes:
- evals
- global
- slack
valueIncludes:
- slack
- "$var:"
valueExcludes:
- xoxb-redacted-test-token
toolExpect:
requiredToolsUsed:
- write_variable
- search_resource_types
- write_resource
forbiddenToolsUsed:
- write_schedule
- write_trigger
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: write_variable
field: value
stringStartsWithAnyOf:
- xoxb-redacted-test-token
skipJudge: true
judgeChecklist:
- creates a secret variable draft for the Slack bot token placeholder
- creates a Slack resource draft that references the secret variable instead of embedding the token
- keeps both drafts under a sensible eval/global Slack-related path
- does not create schedules, triggers, or deployed workspace changes
- id: global-test19-datatable-not-configured-asks-to-set-up
prompt: |-
Here are two newsletter signups: alice@example.com and bob@example.com.
Save them into a workspace data table for me.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- list_datatables
forbiddenToolsUsed:
- exec_datatable_sql
- write_script
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- checks which data tables exist in the workspace before acting
- recognizes that no data table is configured in this workspace
- explains that a data table must first be set up by the user in the workspace settings (Data Tables) and is not created via SQL
- does not run SQL, write a script, or invent a data table to work around the missing configuration
- tells the user to configure a data table and then try again
- id: global-test20-datatable-no-hallucinated-main
prompt: |-
Pull the latest rows from the orders table in our data table so I can see recent orders.
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- list_datatables
forbiddenToolsUsed:
- exec_datatable_sql
- write_script
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- checks which data tables exist in the workspace before querying
- recognizes that no data table is configured in this workspace
- does not assume a data table named "main" (or any other name) exists
- does not run SQL against a guessed data table or fabricate order rows
- tells the user they need to set up a data table in the workspace settings first
- id: global-test21-datatable-list-summarize
prompt: |-
What tables do we have in our workspace data table? Just give me the list.
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- list_datatables
forbiddenToolsUsed:
- get_datatable_table_schema
- exec_datatable_sql
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- lists the tables available in the workspace data table (orders and customers)
- answers from the data table listing rather than fabricating table names
- does not fetch column details or run SQL just to produce a table list
- id: global-test22-datatable-inspect-columns
prompt: |-
What columns does the orders table have in our workspace data table?
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- get_datatable_table_schema
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
skipJudge: true
judgeChecklist:
- inspects the orders table schema in the workspace data table
- reports the orders columns (such as id, customer_id, total, status, created_at)
- answers from the retrieved schema rather than guessing the columns
- id: global-test23-datatable-query-select
prompt: |-
Show me the orders in our workspace data table, including their status and total.
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- exec_datatable_sql
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: exec_datatable_sql
field: sql
stringIncludesAnyOf:
- select
skipJudge: true
judgeChecklist:
- runs a SELECT query against the orders table in the workspace data table
- reports the orders returned by the query back to the user instead of fabricating data
- does not tell the user to set up a data table, since one already exists
- id: global-test24-datatable-create-table
prompt: |-
Add a new table called refunds to our workspace data table, with an order id and a refund amount.
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
maxTurns: 6
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- exec_datatable_sql
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: exec_datatable_sql
field: sql
stringIncludesAnyOf:
- create table
skipJudge: true
judgeChecklist:
- creates the refunds table with a plain CREATE TABLE statement on the data table
- includes an order id and a refund amount column
- treats creating the table as a normal SQL statement and does not claim a separate registration step is needed
- does not write a script to create the table
- id: global-test25-datatable-mutate-rows
prompt: |-
Mark order number 2 as shipped in our workspace data table.
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
# Headroom for inspect-schema -> UPDATE -> verify; the in-memory engine now
# persists the write, so verification confirms on the first try (no loop).
maxTurns: 12
validate:
draftCountExactly: 0
toolExpect:
requiredToolsUsed:
- exec_datatable_sql
forbiddenToolsUsed:
- write_script
- deploy_workspace_item
- delete_workspace_item
toolCallArgs:
- tool: exec_datatable_sql
field: sql
stringIncludesAnyOf:
- update
- insert into
skipJudge: true
judgeChecklist:
- runs an UPDATE on the orders table setting the status of order id 2 to shipped
- targets only order number 2 rather than rewriting the whole table
- confirms the change back to the user
- id: global-test26-datatable-script-sdk
prompt: |-
Write a script that reads our workspace data table and returns the total revenue across all orders.
Leave it as an AI draft only; do not deploy or save it.
initial: ai_evals/fixtures/frontend/global/initial/datatable_orders_seed.json
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: script
valueIncludes:
- wmill.datatable(
toolExpect:
requiredToolsUsed:
- get_instructions
- write_script
forbiddenToolsUsed:
- exec_datatable_sql
- deploy_workspace_item
- delete_workspace_item
# The judge has no datatable SDK reference and wrongly penalizes correct
# wmill.datatable() tagged-template usage, so rely on the deterministic checks:
# required get_instructions + write_script, forbidden exec_datatable_sql, and a
# draft that contains wmill.datatable(.
skipJudge: true
judgeChecklist:
- writes a script (not a chat-time SQL execution) that reads the workspace data table at runtime
- uses the wmill.datatable() SDK to query the orders table and sum the order totals
- returns the total revenue from the script
- leaves the result as an AI draft and does not deploy or save it
+5
View File
@@ -5,6 +5,9 @@
Keep it simple and do not add external dependencies.
initial: ai_evals/fixtures/frontend/script/initial/test1_empty_bun.json
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
toolExpect:
requiredToolsUsed:
- test_run_script
judgeChecklist:
- uses the existing `name` input
- returns a plain greeting string
@@ -20,6 +23,7 @@
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
toolExpect:
requiredToolsUsed:
- test_run_script
- create_schedule
toolCallArgs:
- tool: create_schedule
@@ -44,6 +48,7 @@
expected: ai_evals/fixtures/frontend/script/expected/test1_greet_user.json
toolExpect:
requiredToolsUsed:
- test_run_script
- create_trigger
toolCallArgs:
- tool: create_trigger
+7 -3
View File
@@ -211,7 +211,7 @@ async function handleRun(input: {
const summaries: Array<{
label: string;
passRate: number;
averageDurationMs: number;
averagePassedDurationMs: number | null;
}> = [];
for (const [index, model] of models.entries()) {
@@ -259,7 +259,7 @@ async function handleRun(input: {
summaries.push({
label: `${model.id} (${runModel})`,
passRate: result.passRate,
averageDurationMs: result.averageDurationMs,
averagePassedDurationMs: result.averagePassedDurationMs ?? null,
});
}
@@ -267,7 +267,7 @@ async function handleRun(input: {
process.stdout.write("\nModel summary\n");
for (const summary of summaries) {
process.stdout.write(
`- ${summary.label}: ${formatPercent(summary.passRate)} | ${Math.round(summary.averageDurationMs)}ms\n`,
`- ${summary.label}: ${formatPercent(summary.passRate)} | passed avg ${formatNullableDuration(summary.averagePassedDurationMs)}\n`,
);
}
}
@@ -351,6 +351,10 @@ function formatPercent(value: number): string {
return `${(value * 100).toFixed(1)}%`;
}
function formatNullableDuration(value: number | null): string {
return value === null ? "n/a" : `${Math.round(value)}ms`;
}
void main().catch((error) => {
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${message}\n`);
+44 -1
View File
@@ -14,6 +14,21 @@ describe("loadCases", () => {
},
},
});
expect(caseEntry?.toolExpect).toEqual({
requiredToolsUsed: ["test_run_flow"],
});
});
it("loads script and flow test tool expectations", async () => {
const scriptCases = await loadCases("script");
const flowCases = await loadCases("flow");
expect(scriptCases.find((entry) => entry.id === "script-test1-greet-user")?.toolExpect).toEqual({
requiredToolsUsed: ["test_run_script"],
});
expect(flowCases.find((entry) => entry.id === "flow-test0-sum-two-numbers")?.toolExpect).toEqual({
requiredToolsUsed: ["test_run_flow"],
});
});
it("loads the workspace-flow preference benchmark case", async () => {
@@ -203,6 +218,34 @@ describe("loadCases", () => {
});
});
it("loads global active-editor eval cases", async () => {
const globalCases = await loadCases("global");
const scriptCase = globalCases.find(
(entry) => entry.id === "global-test12-current-live-script-edit"
);
const flowCase = globalCases.find(
(entry) => entry.id === "global-test13-current-live-flow-edit"
);
expect(scriptCase?.initialPath).toContain(
"ai_evals/fixtures/frontend/global/initial/current_greeting_live_script.json"
);
expect(scriptCase?.toolExpect).toMatchObject({
requiredToolsUsed: ["read_workspace_item"],
});
expect(flowCase?.initialPath).toContain(
"ai_evals/fixtures/frontend/global/initial/current_invoice_live_flow.json"
);
expect(flowCase?.validate).toMatchObject({
requiredDrafts: [
{
type: "flow",
path: "f/evals/global/current_invoice_flow",
},
],
});
});
it("loads tool expectations for workspace mutation cases", async () => {
const scriptCases = await loadCases("script");
const caseEntry = scriptCases.find(
@@ -210,7 +253,7 @@ describe("loadCases", () => {
);
expect(caseEntry?.toolExpect).toEqual({
requiredToolsUsed: ["create_schedule"],
requiredToolsUsed: ["test_run_script", "create_schedule"],
toolCallArgs: [
{
tool: "create_schedule",
+17 -10
View File
@@ -2,15 +2,22 @@ import { describe, expect, it } from "bun:test";
import { resolveEvalModel } from "./models";
describe("resolveEvalModel", () => {
it("supports GPT-5.5 aliases for frontend evals", () => {
expect(resolveEvalModel("flow", "gpt-5.5").frontend).toEqual({
provider: "openai",
model: "gpt-5.5",
});
expect(resolveEvalModel("app", "gpt-55").frontend).toEqual({
provider: "openai",
model: "gpt-5.5",
});
expect(resolveEvalModel("script", "5.5").frontend).toEqual({
provider: "openai",
model: "gpt-5.5",
});
});
it("supports Gemini aliases for frontend evals", () => {
expect(resolveEvalModel("flow", "gemini").frontend).toEqual({
provider: "googleai",
model: "gemini-2.5-flash",
});
expect(resolveEvalModel("app", "gemini-pro").frontend).toEqual({
provider: "googleai",
model: "gemini-2.5-pro",
});
expect(
resolveEvalModel("script", "gemini-3-flash-preview").frontend,
).toEqual({
@@ -37,8 +44,8 @@ describe("resolveEvalModel", () => {
});
it("rejects Gemini aliases for cli evals", () => {
expect(() => resolveEvalModel("cli", "gemini")).toThrow(
"Model gemini-flash is not supported for cli mode",
expect(() => resolveEvalModel("cli", "gemini-3-flash-preview")).toThrow(
"Model gemini-3-flash-preview is not supported for cli mode",
);
});
});
+5 -14
View File
@@ -88,21 +88,12 @@ export const EVAL_MODELS: EvalModelSpec[] = [
},
},
{
id: "gemini-flash",
label: "Gemini 2.5 Flash",
aliases: ["gemini", "gemini-flash", "gemini-2.5-flash"],
id: "gpt-5.5",
label: "GPT-5.5",
aliases: ["gpt-5.5", "gpt-55", "5.5"],
frontend: {
provider: "googleai",
model: "gemini-2.5-flash",
},
},
{
id: "gemini-pro",
label: "Gemini 2.5 Pro",
aliases: ["gemini-pro", "gemini-2.5-pro"],
frontend: {
provider: "googleai",
model: "gemini-2.5-pro",
provider: "openai",
model: "gpt-5.5",
},
},
{
+242
View File
@@ -0,0 +1,242 @@
import { mkdtemp, readFile, rm } from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { describe, expect, it } from "bun:test";
import {
appendHistoryRecord,
buildRunResult,
formatRunSummary,
} from "./results";
import type { BenchmarkCaseResult } from "./types";
function caseResult(
attempts: BenchmarkCaseResult["attempts"],
): BenchmarkCaseResult {
return {
id: "case-1",
prompt: "Do the thing",
attempts,
};
}
describe("benchmark results", () => {
it("keeps success cost metrics separate from failed attempts", () => {
const result = buildRunResult({
mode: "global",
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
caseResults: [
caseResult([
{
attempt: 1,
passed: true,
durationMs: 1000,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: { prompt: 100, completion: 20, total: 120 },
},
{
attempt: 2,
passed: false,
durationMs: 100,
assistantMessageCount: 1,
toolCallCount: 0,
toolsUsed: [],
skillsInvoked: [],
checks: [{ name: "edited", passed: false }],
judgeScore: 10,
judgeSummary: "missed",
error: "failed",
tokenUsage: { prompt: 10, completion: 5, total: 15 },
},
]),
],
});
expect(result.attemptCount).toBe(2);
expect(result.passedAttempts).toBe(1);
expect(result.passRate).toBe(0.5);
expect(result.averageDurationMs).toBe(550);
expect(result.averagePassedDurationMs).toBe(1000);
expect(result.totalTokenUsage).toEqual({
prompt: 110,
completion: 25,
total: 135,
});
expect(result.totalPassedTokenUsage).toEqual({
prompt: 100,
completion: 20,
total: 120,
});
expect(result.averageTokenUsagePerAttempt).toEqual({
prompt: 55,
completion: 12.5,
total: 67.5,
});
expect(result.averageTokenUsagePerPassedAttempt).toEqual({
prompt: 100,
completion: 20,
total: 120,
});
const summary = formatRunSummary(result);
expect(summary).toContain("Average duration (passed): 1000ms");
expect(summary).toContain("Average tokens (passed): 120 total");
expect(summary).toContain("Average duration (all attempts): 550ms");
});
it("reports passed averages as unavailable when no attempt passes", () => {
const result = buildRunResult({
mode: "global",
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
caseResults: [
caseResult([
{
attempt: 1,
passed: false,
durationMs: 100,
assistantMessageCount: 1,
toolCallCount: 0,
toolsUsed: [],
skillsInvoked: [],
checks: [{ name: "edited", passed: false }],
judgeScore: 10,
judgeSummary: "missed",
error: "failed",
tokenUsage: { prompt: 10, completion: 5, total: 15 },
},
]),
],
});
expect(result.averagePassedDurationMs).toBeNull();
expect(result.totalPassedTokenUsage).toBeNull();
expect(result.averageTokenUsagePerPassedAttempt).toBeNull();
expect(formatRunSummary(result)).toContain(
"Average duration (passed): n/a",
);
});
it("normalizes passed token averages by passed attempts", () => {
const result = buildRunResult({
mode: "global",
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
caseResults: [
caseResult([
{
attempt: 1,
passed: true,
durationMs: 1000,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: { prompt: 100, completion: 20, total: 120 },
},
{
attempt: 2,
passed: true,
durationMs: 1200,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: null,
},
]),
],
});
expect(result.passedAttempts).toBe(2);
expect(result.totalPassedTokenUsage).toEqual({
prompt: 100,
completion: 20,
total: 120,
});
expect(result.averageTokenUsagePerPassedAttempt).toEqual({
prompt: 50,
completion: 10,
total: 60,
});
});
it("records passed-attempt metrics in history", async () => {
const tempDir = await mkdtemp(join(tmpdir(), "windmill-ai-evals-"));
try {
const historyPath = join(tempDir, "history.jsonl");
const result = buildRunResult({
mode: "global",
runs: 1,
runModel: "model-under-test",
judgeModel: "judge-model",
caseResults: [
caseResult([
{
attempt: 1,
passed: true,
durationMs: 1000,
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["edit_script"],
skillsInvoked: [],
checks: [{ name: "edited", passed: true }],
judgeScore: 100,
judgeSummary: "ok",
error: null,
tokenUsage: { prompt: 100, completion: 20, total: 120 },
},
{
attempt: 2,
passed: false,
durationMs: 100,
assistantMessageCount: 1,
toolCallCount: 0,
toolsUsed: [],
skillsInvoked: [],
checks: [{ name: "edited", passed: false }],
judgeScore: 10,
judgeSummary: "missed",
error: "failed",
tokenUsage: { prompt: 10, completion: 5, total: 15 },
},
]),
],
});
await appendHistoryRecord(result, historyPath);
const record = JSON.parse(await readFile(historyPath, "utf8"));
expect(record.averageDurationMs).toBe(550);
expect(record.averagePassedDurationMs).toBe(1000);
expect(record.averageTokenUsagePerAttempt.total).toBe(67.5);
expect(record.averageTokenUsagePerPassedAttempt.total).toBe(120);
expect(record.cases[0].averageDurationMs).toBe(550);
expect(record.cases[0].averagePassedDurationMs).toBe(1000);
expect(record.cases[0].averageTokenUsagePerAttempt.total).toBe(67.5);
expect(record.cases[0].averageTokenUsagePerPassedAttempt.total).toBe(
120,
);
} finally {
await rm(tempDir, { recursive: true, force: true });
}
});
});
+114 -67
View File
@@ -4,12 +4,20 @@ import { execFileSync } from "node:child_process";
import { getAiEvalsRoot, getRepoRoot } from "./cases";
import type {
BenchmarkArtifactFile,
BenchmarkAttemptResult,
BenchmarkCaseResult,
BenchmarkRunResult,
BenchmarkTokenUsage,
EvalMode,
} from "./types";
type AttemptAggregate = {
attemptCount: number;
durationTotal: number;
tokenUsageAttemptCount: number;
tokenUsageTotal: BenchmarkTokenUsage | null;
};
export async function writeRunResult(
result: BenchmarkRunResult,
outputPath?: string,
@@ -77,36 +85,12 @@ export function buildRunResult(input: {
judgeModel: string | null;
caseResults: BenchmarkCaseResult[];
}): BenchmarkRunResult {
const attemptCount = input.caseResults.reduce(
(sum, entry) => sum + entry.attempts.length,
0,
);
const passedAttempts = input.caseResults.reduce(
(sum, entry) =>
sum + entry.attempts.filter((attempt) => attempt.passed).length,
0,
);
const durationTotal = input.caseResults.reduce(
(sum, entry) =>
sum +
entry.attempts.reduce((inner, attempt) => inner + attempt.durationMs, 0),
0,
);
const tokenUsageTotal = input.caseResults.reduce<BenchmarkTokenUsage | null>(
(sum, entry) => {
for (const attempt of entry.attempts) {
if (!attempt.tokenUsage) {
continue;
}
sum ??= { prompt: 0, completion: 0, total: 0 };
sum.prompt += attempt.tokenUsage.prompt;
sum.completion += attempt.tokenUsage.completion;
sum.total += attempt.tokenUsage.total;
}
return sum;
},
null,
);
const attempts = input.caseResults.flatMap((entry) => entry.attempts);
const passedAttemptResults = attempts.filter((attempt) => attempt.passed);
const attemptAggregate = aggregateAttempts(attempts);
const passedAttemptAggregate = aggregateAttempts(passedAttemptResults);
const attemptCount = attemptAggregate.attemptCount;
const passedAttempts = passedAttemptAggregate.attemptCount;
return {
version: 1,
@@ -120,16 +104,19 @@ export function buildRunResult(input: {
attemptCount,
passedAttempts,
passRate: attemptCount === 0 ? 0 : passedAttempts / attemptCount,
averageDurationMs: attemptCount === 0 ? 0 : durationTotal / attemptCount,
totalTokenUsage: tokenUsageTotal,
averageDurationMs:
attemptCount === 0 ? 0 : attemptAggregate.durationTotal / attemptCount,
averagePassedDurationMs: averageDuration(passedAttemptAggregate),
totalTokenUsage: attemptAggregate.tokenUsageTotal,
totalPassedTokenUsage: passedAttemptAggregate.tokenUsageTotal,
averageTokenUsagePerAttempt:
attemptCount === 0 || !tokenUsageTotal
attemptCount === 0
? null
: {
prompt: tokenUsageTotal.prompt / attemptCount,
completion: tokenUsageTotal.completion / attemptCount,
total: tokenUsageTotal.total / attemptCount,
},
: averageTokenUsage(attemptAggregate, attemptCount),
averageTokenUsagePerPassedAttempt: averageTokenUsage(
passedAttemptAggregate,
passedAttempts,
),
cases: input.caseResults,
};
}
@@ -138,9 +125,25 @@ export function formatRunSummary(result: BenchmarkRunResult): string {
const lines = [
`${result.mode} benchmark complete`,
`Pass rate: ${formatPercent(result.passRate)} (${result.passedAttempts}/${result.attemptCount})`,
`Average duration: ${Math.round(result.averageDurationMs)}ms`,
`Average duration (passed): ${formatNullableDuration(result.averagePassedDurationMs ?? null)}`,
];
if (result.averageTokenUsagePerPassedAttempt) {
lines.push(
`Average tokens (passed): ${formatTokenUsage(result.averageTokenUsagePerPassedAttempt)}`,
);
}
if (result.passedAttempts < result.attemptCount) {
lines.push(
`Average duration (all attempts): ${Math.round(result.averageDurationMs)}ms`,
);
if (result.averageTokenUsagePerAttempt) {
lines.push(
`Average tokens (all attempts): ${formatTokenUsage(result.averageTokenUsagePerAttempt)}`,
);
}
}
const failures = collectFailures(result);
if (failures.length > 0) {
lines.push("Failures:");
@@ -172,6 +175,60 @@ function collectFailures(result: BenchmarkRunResult): string[] {
return failures;
}
function aggregateAttempts(attempts: BenchmarkAttemptResult[]): AttemptAggregate {
const aggregate: AttemptAggregate = {
attemptCount: attempts.length,
durationTotal: 0,
tokenUsageAttemptCount: 0,
tokenUsageTotal: null,
};
for (const attempt of attempts) {
aggregate.durationTotal += attempt.durationMs;
if (!attempt.tokenUsage) {
continue;
}
aggregate.tokenUsageAttemptCount += 1;
aggregate.tokenUsageTotal ??= { prompt: 0, completion: 0, total: 0 };
aggregate.tokenUsageTotal.prompt += attempt.tokenUsage.prompt;
aggregate.tokenUsageTotal.completion += attempt.tokenUsage.completion;
aggregate.tokenUsageTotal.total += attempt.tokenUsage.total;
}
return aggregate;
}
function averageDuration(aggregate: AttemptAggregate): number | null {
return aggregate.attemptCount === 0
? null
: aggregate.durationTotal / aggregate.attemptCount;
}
function averageTokenUsage(
aggregate: AttemptAggregate,
denominator: number,
): BenchmarkTokenUsage | null {
if (denominator === 0 || !aggregate.tokenUsageTotal) {
return null;
}
return {
prompt: aggregate.tokenUsageTotal.prompt / denominator,
completion: aggregate.tokenUsageTotal.completion / denominator,
total: aggregate.tokenUsageTotal.total / denominator,
};
}
function formatNullableDuration(value: number | null): string {
return value === null ? "n/a" : `${Math.round(value)}ms`;
}
function formatTokenUsage(value: BenchmarkTokenUsage): string {
const total = Math.round(value.total);
const prompt = Math.round(value.prompt);
const completion = Math.round(value.completion);
return `${total} total (${prompt} prompt, ${completion} completion)`;
}
function defaultFileName(mode: EvalMode): string {
return `${new Date().toISOString().replaceAll(":", "-")}__${mode}.json`;
}
@@ -252,12 +309,15 @@ function toHistoryRecord(result: BenchmarkRunResult) {
passedAttempts: result.passedAttempts,
passRate: result.passRate,
averageDurationMs: result.averageDurationMs,
averagePassedDurationMs: result.averagePassedDurationMs ?? null,
averageJudgeScore:
judgeScores.length === 0
? null
: judgeScores.reduce((sum, score) => sum + score, 0) /
judgeScores.length,
averageTokenUsagePerAttempt: result.averageTokenUsagePerAttempt ?? null,
averageTokenUsagePerPassedAttempt:
result.averageTokenUsagePerPassedAttempt ?? null,
failedCaseIds: Array.from(
new Set(
result.cases
@@ -268,31 +328,15 @@ function toHistoryRecord(result: BenchmarkRunResult) {
),
),
cases: result.cases.map((caseResult) => {
const attemptCount = caseResult.attempts.length;
const passedAttempts = caseResult.attempts.filter(
(attempt) => attempt.passed,
).length;
const totalDurationMs = caseResult.attempts.reduce(
(sum, attempt) => sum + attempt.durationMs,
0,
const attemptAggregate = aggregateAttempts(caseResult.attempts);
const passedAttemptAggregate = aggregateAttempts(
caseResult.attempts.filter((attempt) => attempt.passed),
);
const attemptCount = attemptAggregate.attemptCount;
const passedAttempts = passedAttemptAggregate.attemptCount;
const judgeScores = caseResult.attempts.flatMap((attempt) =>
typeof attempt.judgeScore === "number" ? [attempt.judgeScore] : [],
);
const totalTokenUsage =
caseResult.attempts.reduce<BenchmarkTokenUsage | null>(
(sum, attempt) => {
if (!attempt.tokenUsage) {
return sum;
}
sum ??= { prompt: 0, completion: 0, total: 0 };
sum.prompt += attempt.tokenUsage.prompt;
sum.completion += attempt.tokenUsage.completion;
sum.total += attempt.tokenUsage.total;
return sum;
},
null,
);
return {
id: caseResult.id,
@@ -300,20 +344,23 @@ function toHistoryRecord(result: BenchmarkRunResult) {
passedAttempts,
passRate: attemptCount === 0 ? 0 : passedAttempts / attemptCount,
averageDurationMs:
attemptCount === 0 ? 0 : totalDurationMs / attemptCount,
attemptCount === 0
? 0
: attemptAggregate.durationTotal / attemptCount,
averagePassedDurationMs: averageDuration(passedAttemptAggregate),
averageJudgeScore:
judgeScores.length === 0
? null
: judgeScores.reduce((sum, score) => sum + score, 0) /
judgeScores.length,
averageTokenUsagePerAttempt:
attemptCount === 0 || !totalTokenUsage
attemptCount === 0
? null
: {
prompt: totalTokenUsage.prompt / attemptCount,
completion: totalTokenUsage.completion / attemptCount,
total: totalTokenUsage.total / attemptCount,
},
: averageTokenUsage(attemptAggregate, attemptCount),
averageTokenUsagePerPassedAttempt: averageTokenUsage(
passedAttemptAggregate,
passedAttempts,
),
};
}),
};
+15 -1
View File
@@ -110,7 +110,9 @@ export interface AppValidationSpec {
export interface GlobalDraftRequirement {
type: string;
path: string;
path?: string;
pathIncludes?: string[];
pathStartsWith?: string;
triggerKind?: string;
language?: string;
summaryIncludes?: string[];
@@ -153,6 +155,15 @@ export interface ToolCallArgumentRule {
field: string;
stringStartsWithAnyOf?: string[];
stringMustNotStartWithAnyOf?: string[];
/**
* Case-insensitive "contains", existential over calls: at least one recorded
* call to `tool` must have `field` containing one of these substrings. Other
* calls to the same tool may do anything. Use instead of `stringStartsWithAnyOf`
* (which is universal over calls) when the meaningful token can appear anywhere
* in the value and the model may make additional, unrelated calls to the same
* tool — e.g. SQL where a mutation is mixed with verification SELECTs.
*/
stringIncludesAnyOf?: string[];
}
export interface ToolValidationSpec {
@@ -324,8 +335,11 @@ export interface BenchmarkRunResult {
passedAttempts: number;
passRate: number;
averageDurationMs: number;
averagePassedDurationMs?: number | null;
totalTokenUsage?: BenchmarkTokenUsage | null;
totalPassedTokenUsage?: BenchmarkTokenUsage | null;
averageTokenUsagePerAttempt?: BenchmarkTokenUsage | null;
averageTokenUsagePerPassedAttempt?: BenchmarkTokenUsage | null;
artifactsPath?: string | null;
cases: BenchmarkCaseResult[];
}
+168
View File
@@ -140,6 +140,111 @@ describe("validateToolExpectations", () => {
details: "tools used: write_script, deploy_workspace_item",
});
});
it("accepts a stringIncludesAnyOf substring regardless of case or position", () => {
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["exec_datatable_sql"],
toolCallDetails: [
{
name: "exec_datatable_sql",
arguments: {
sql: "WITH recent AS (SELECT * FROM orders) SELECT count(*) FROM recent",
},
},
],
skillsInvoked: [],
},
toolExpect: {
requiredToolsUsed: ["exec_datatable_sql"],
toolCallArgs: [
{
tool: "exec_datatable_sql",
field: "sql",
stringIncludesAnyOf: ["select"],
},
],
},
});
expect(checks.every((check) => check.passed)).toBe(true);
});
it("accepts stringIncludesAnyOf when only one of several calls matches", () => {
// Existential: a mutation mixed with verification SELECTs still passes.
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 2,
toolsUsed: ["exec_datatable_sql"],
toolCallDetails: [
{
name: "exec_datatable_sql",
arguments: { sql: "UPDATE orders SET status = 'shipped' WHERE id = 2" },
},
{
name: "exec_datatable_sql",
arguments: { sql: "SELECT * FROM orders WHERE id = 2" },
},
],
skillsInvoked: [],
},
toolExpect: {
toolCallArgs: [
{
tool: "exec_datatable_sql",
field: "sql",
stringIncludesAnyOf: ["insert into", "update"],
},
],
},
});
expect(checks.every((check) => check.passed)).toBe(true);
});
it("rejects stringIncludesAnyOf when no call matches any substring", () => {
const checks = validateToolExpectations({
run: {
success: true,
actual: {},
assistantMessageCount: 1,
toolCallCount: 1,
toolsUsed: ["exec_datatable_sql"],
toolCallDetails: [
{
name: "exec_datatable_sql",
arguments: {
sql: "DROP TABLE orders",
},
},
],
skillsInvoked: [],
},
toolExpect: {
toolCallArgs: [
{
tool: "exec_datatable_sql",
field: "sql",
stringIncludesAnyOf: ["insert into", "update"],
},
],
},
});
expect(checks).toContainEqual({
name: "exec_datatable_sql.sql includes a required substring",
passed: false,
details:
'accepted substrings: insert into, update; values: "DROP TABLE orders"',
});
});
});
describe("validateGlobalState", () => {
@@ -195,6 +300,69 @@ describe("validateGlobalState", () => {
});
});
it("accepts a required script draft without an exact path", () => {
const checks = validateGlobalState({
actual: {
drafts: [
{
type: "script",
path: "f/team_tools/friendly_greeting",
language: "bun",
summary: "Friendly greeting helper",
value:
"export async function main(name: string) {\n return `Hello, ${name}!`\n}\n",
isDraft: true,
},
],
},
validate: {
draftCountExactly: 1,
requiredDrafts: [
{
type: "script",
pathIncludes: ["greeting"],
language: "bun",
summaryIncludes: ["Friendly"],
valueIncludes: ["Hello"],
},
],
},
});
expect(checks.every((check) => check.passed)).toBe(true);
});
it("reports flexible global draft path filters when no draft matches", () => {
const checks = validateGlobalState({
actual: {
drafts: [
{
type: "script",
path: "f/team_tools/friendly_greeting",
language: "bun",
value:
"export async function main(name: string) {\n return `Hello, ${name}!`\n}\n",
isDraft: true,
},
],
},
validate: {
requiredDrafts: [
{
type: "script",
pathIncludes: ["invoice"],
},
],
},
});
expect(checks).toContainEqual({
name: "global includes script draft (path includes invoice)",
passed: false,
details: "drafts: script:f/team_tools/friendly_greeting",
});
});
it("does not require a TypeScript entrypoint for non-TypeScript script drafts", () => {
const checks = validateGlobalState({
actual: {
+119 -15
View File
@@ -222,6 +222,25 @@ export function validateToolExpectations(input: {
)
);
}
if (rule.stringIncludesAnyOf && rule.stringIncludesAnyOf.length > 0) {
// Existential: at least one call must contain one of the substrings.
// Other calls to the same tool may do anything — this suits SQL, where a
// model mixes the requested statement (e.g. an UPDATE) with verification
// SELECTs that would otherwise fail an "all calls" check.
const needles = rule.stringIncludesAnyOf.map((needle) => needle.toLowerCase());
const hasMatch = values.some(
(value) =>
typeof value === "string" && needles.some((needle) => value.toLowerCase().includes(needle))
);
checks.push(
check(
`${rule.tool}.${rule.field} includes a required substring`,
hasMatch,
`accepted substrings: ${rule.stringIncludesAnyOf.join(", ")}; values: ${summarizeToolValues(values)}`
)
);
}
}
return checks;
@@ -315,10 +334,11 @@ export function validateGlobalState(input: {
}
for (const required of validate.requiredDrafts ?? []) {
const draft = findGlobalDraft(drafts, required.type, required.path, required.triggerKind);
const requirementLabel = formatGlobalDraftRequirement(required);
const draft = findGlobalDraft(drafts, required);
checks.push(
check(
`global includes ${required.type} draft ${required.path}`,
`global includes ${requirementLabel}`,
Boolean(draft),
summarizeGlobalDrafts(drafts)
)
@@ -330,7 +350,7 @@ export function validateGlobalState(input: {
if (required.language !== undefined) {
checks.push(
check(
`${required.type} draft ${required.path} uses ${required.language}`,
`${requirementLabel} uses ${required.language}`,
draft.language === required.language,
`language=${draft.language ?? "(none)"}`
)
@@ -340,7 +360,7 @@ export function validateGlobalState(input: {
for (const snippet of required.summaryIncludes ?? []) {
checks.push(
check(
`${required.type} draft ${required.path} summary includes '${snippet}'`,
`${requirementLabel} summary includes '${snippet}'`,
normalizeText(draft.summary ?? "").includes(normalizeText(snippet)),
`summary=${draft.summary ?? ""}`
)
@@ -351,7 +371,7 @@ export function validateGlobalState(input: {
for (const snippet of required.valueIncludes ?? []) {
checks.push(
check(
`${required.type} draft ${required.path} value includes '${snippet}'`,
`${requirementLabel} value includes '${snippet}'`,
normalizeText(valueText).includes(normalizeText(snippet)),
truncateForDetails(valueText)
)
@@ -361,7 +381,7 @@ export function validateGlobalState(input: {
for (const snippet of required.valueExcludes ?? []) {
checks.push(
check(
`${required.type} draft ${required.path} value excludes '${snippet}'`,
`${requirementLabel} value excludes '${snippet}'`,
!normalizeText(valueText).includes(normalizeText(snippet)),
truncateForDetails(valueText)
)
@@ -373,7 +393,7 @@ export function validateGlobalState(input: {
checks.push(
check(
`global does not include ${forbidden.type} draft ${forbidden.path}`,
!findGlobalDraft(drafts, forbidden.type, forbidden.path, forbidden.triggerKind),
!findGlobalDraft(drafts, forbidden),
summarizeGlobalDrafts(drafts)
)
);
@@ -615,16 +635,100 @@ function summarizeProblems(problems: string[], limit = 5): string | undefined {
function findGlobalDraft(
drafts: GlobalDraft[],
type: string,
path: string,
triggerKind?: string
requirement: {
type: string;
path?: string;
pathIncludes?: string[];
pathStartsWith?: string;
triggerKind?: string;
summaryIncludes?: string[];
valueIncludes?: string[];
valueExcludes?: string[];
}
): GlobalDraft | undefined {
return drafts.find(
(draft) =>
draft.type === type &&
draft.path === path &&
(triggerKind === undefined || draft.triggerKind === triggerKind)
const candidates = drafts.filter((draft) =>
globalDraftMatchesLocator(draft, requirement)
);
return (
candidates.find((draft) => globalDraftMatchesContent(draft, requirement)) ??
candidates[0]
);
}
function globalDraftMatchesLocator(
draft: GlobalDraft,
requirement: {
type: string;
path?: string;
pathIncludes?: string[];
pathStartsWith?: string;
triggerKind?: string;
}
): boolean {
return (
draft.type === requirement.type &&
(requirement.path === undefined || draft.path === requirement.path) &&
(requirement.pathStartsWith === undefined ||
draft.path.startsWith(requirement.pathStartsWith)) &&
(requirement.pathIncludes ?? []).every((snippet) =>
normalizeText(draft.path).includes(normalizeText(snippet))
) &&
(requirement.triggerKind === undefined ||
draft.triggerKind === requirement.triggerKind)
);
}
function globalDraftMatchesContent(
draft: GlobalDraft,
requirement: {
summaryIncludes?: string[];
valueIncludes?: string[];
valueExcludes?: string[];
}
): boolean {
const summary = normalizeText(draft.summary ?? "");
const value = normalizeText(stringifyGlobalDraftValue(draft.value));
return (
(requirement.summaryIncludes ?? []).every((snippet) =>
summary.includes(normalizeText(snippet))
) &&
(requirement.valueIncludes ?? []).every((snippet) =>
value.includes(normalizeText(snippet))
) &&
(requirement.valueExcludes ?? []).every(
(snippet) => !value.includes(normalizeText(snippet))
)
);
}
function formatGlobalDraftRequirement(
requirement: {
type: string;
path?: string;
pathIncludes?: string[];
pathStartsWith?: string;
triggerKind?: string;
}
): string {
const typeLabel =
requirement.triggerKind === undefined
? requirement.type
: `${requirement.triggerKind} ${requirement.type}`;
if (requirement.path !== undefined) {
return `${typeLabel} draft ${requirement.path}`;
}
const filters = [
...(requirement.pathStartsWith === undefined
? []
: [`path starts with ${requirement.pathStartsWith}`]),
...(requirement.pathIncludes ?? []).map(
(snippet) => `path includes ${snippet}`
),
];
return filters.length === 0
? `${typeLabel} draft`
: `${typeLabel} draft (${filters.join(", ")})`;
}
function summarizeGlobalDrafts(drafts: GlobalDraft[]): string {
@@ -0,0 +1,66 @@
{
"workspace": {
"scripts": [
{
"path": "f/evals/global/format_greeting",
"summary": "Format a deployed greeting",
"description": "Returns a plain greeting for a provided name.",
"language": "bun",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"name": {
"type": "string"
}
},
"required": ["name"]
},
"content": "export async function main(name: string) {\n return `Hello, ${name}`\n}\n"
},
{
"path": "f/evals/global/format_greeting_archive",
"summary": "Archived greeting formatter",
"description": "Older greeting formatter kept for reference.",
"language": "bun",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"name": {
"type": "string"
}
},
"required": ["name"]
},
"content": "export async function main(name: string) {\n return `Hi, ${name}`\n}\n"
}
]
},
"liveEditorDrafts": [
{
"type": "script",
"storagePath": "f/evals/global/current_greeting",
"effectivePath": "f/evals/global/current_greeting",
"value": {
"path": "f/evals/global/current_greeting",
"summary": "Open greeting formatter",
"description": "Formats a greeting in the live editor.",
"language": "bun",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"name": {
"type": "string"
}
},
"required": ["name"]
},
"content": "export async function main(name: string) {\n return `Hello, ${name}`\n}\n",
"is_template": false,
"kind": "script"
}
}
]
}
@@ -0,0 +1,118 @@
{
"workspace": {
"flows": [
{
"path": "f/evals/global/process_invoice",
"summary": "Deployed invoice processor",
"description": "Calculates invoice totals from a subtotal.",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"subtotal": {
"type": "number"
}
},
"required": ["subtotal"]
},
"value": {
"modules": [
{
"id": "calculate_total",
"summary": "Calculate total from subtotal",
"value": {
"type": "rawscript",
"language": "bun",
"content": "export async function main(subtotal: number) {\n return { subtotal, total: subtotal }\n}\n",
"input_transforms": {
"subtotal": {
"type": "javascript",
"expr": "flow_input.subtotal"
}
}
}
}
]
}
},
{
"path": "f/evals/global/process_refund",
"summary": "Refund processor",
"description": "Calculates refund totals.",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"subtotal": {
"type": "number"
}
},
"required": ["subtotal"]
},
"value": {
"modules": [
{
"id": "calculate_total",
"summary": "Calculate refund total",
"value": {
"type": "rawscript",
"language": "bun",
"content": "export async function main(subtotal: number) {\n return { subtotal, total: subtotal }\n}\n",
"input_transforms": {
"subtotal": {
"type": "javascript",
"expr": "flow_input.subtotal"
}
}
}
}
]
}
}
]
},
"liveEditorDrafts": [
{
"type": "flow",
"storagePath": "f/evals/global/current_invoice_flow",
"effectivePath": "f/evals/global/current_invoice_flow",
"value": {
"path": "f/evals/global/current_invoice_flow",
"summary": "Open invoice processor",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"subtotal": {
"type": "number"
}
},
"required": ["subtotal"]
},
"value": {
"modules": [
{
"id": "calculate_total",
"summary": "Calculate total from subtotal",
"value": {
"type": "rawscript",
"language": "bun",
"content": "export async function main(subtotal: number) {\n return { subtotal, total: subtotal }\n}\n",
"input_transforms": {
"subtotal": {
"type": "javascript",
"expr": "flow_input.subtotal"
}
}
}
}
]
},
"edited_by": "",
"edited_at": "",
"archived": false,
"extra_perms": {}
}
}
]
}
@@ -0,0 +1,39 @@
{
"workspace": {
"datatables": [
{
"datatable_name": "main",
"schemas": {
"public": {
"orders": {
"columns": {
"id": "int4",
"customer_id": "int4",
"total": "numeric",
"status": "text",
"created_at": "timestamptz"
},
"rows": [
{ "id": 1, "customer_id": 1, "total": 42.5, "status": "shipped", "created_at": "2026-05-01T10:00:00Z" },
{ "id": 2, "customer_id": 2, "total": 19.99, "status": "pending", "created_at": "2026-05-02T11:30:00Z" },
{ "id": 3, "customer_id": 1, "total": 88, "status": "shipped", "created_at": "2026-05-03T09:15:00Z" }
]
},
"customers": {
"columns": {
"id": "int4",
"name": "text",
"email": "text",
"tier": "text"
},
"rows": [
{ "id": 1, "name": "Alice", "email": "alice@example.com", "tier": "gold" },
{ "id": 2, "name": "Bob", "email": "bob@example.com", "tier": "silver" }
]
}
}
}
}
]
}
}
@@ -0,0 +1,40 @@
{
"workspace": {
"flows": [
{
"path": "f/evals/global/process_invoice",
"summary": "Process an invoice subtotal",
"description": "Calculates invoice totals from a subtotal.",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"subtotal": {
"type": "number"
}
},
"required": ["subtotal"]
},
"value": {
"modules": [
{
"id": "calculate_total",
"summary": "Calculate total from subtotal",
"value": {
"type": "rawscript",
"language": "bun",
"content": "export async function main(subtotal: number) {\n return { subtotal, total: subtotal }\n}\n",
"input_transforms": {
"subtotal": {
"type": "javascript",
"expr": "flow_input.subtotal"
}
}
}
}
]
}
}
]
}
}
@@ -0,0 +1,23 @@
{
"workspace": {
"scripts": [
{
"path": "f/evals/global/send_report_digest",
"summary": "Build and send the eval report digest",
"description": "Returns a dry-run summary for eval report digest notifications.",
"language": "bun",
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"dry_run": {
"type": "boolean"
}
},
"required": ["dry_run"]
},
"content": "export async function main(dry_run: boolean) {\n return { dry_run, sent: !dry_run, message: dry_run ? 'Preview digest' : 'Digest sent' }\n}\n"
}
]
}
}
+7 -1
View File
@@ -1,5 +1,8 @@
import { readFile } from "node:fs/promises";
import { runGlobalEval } from "../adapters/frontend/core/global/globalEvalRunner";
import {
runGlobalEval,
type GlobalLiveEditorDraftFixture,
} from "../adapters/frontend/core/global/globalEvalRunner";
import type { BenchmarkWorkspaceRunnables } from "../adapters/frontend/mockBackend";
import type { FrontendEvalModelConfig } from "../core/models";
import type { BenchmarkArtifactFile, GlobalValidationSpec, ModeRunner } from "../core/types";
@@ -9,6 +12,7 @@ import { getFrontendApiKey } from "./frontendCommon";
export interface GlobalInitialFixture {
workspace?: BenchmarkWorkspaceRunnables;
liveEditorDrafts?: GlobalLiveEditorDraftFixture[];
}
export function createGlobalModeRunner(
@@ -31,6 +35,7 @@ export function createGlobalModeRunner(
getFrontendApiKey(modelConfig.provider),
{
workspaceFixtures: initial?.workspace,
liveEditorDrafts: initial?.liveEditorDrafts,
maxIterations: context.evalCase?.runtime?.maxTurns,
provider: modelConfig.provider,
model: modelConfig.model,
@@ -73,6 +78,7 @@ async function loadGlobalInitialFixture(path: string): Promise<GlobalInitialFixt
const parsed = JSON.parse(await readFile(path, "utf8")) as GlobalInitialFixture;
return {
workspace: parsed.workspace ?? {},
liveEditorDrafts: parsed.liveEditorDrafts ?? [],
};
}
@@ -46,11 +46,11 @@
]
},
"nullable": [
true,
true,
true,
true,
true,
false,
false,
false,
false,
false,
true,
true
]
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "WITH RECURSIVE chain(id, parent_job) AS (\n SELECT id, parent_job FROM v2_job WHERE id = $1 AND workspace_id = $2\n UNION ALL\n SELECT j.id, j.parent_job FROM v2_job j\n JOIN chain c ON j.id = c.parent_job AND j.workspace_id = $2\n )\n SELECT id AS \"id!\" FROM chain",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
}
],
"parameters": {
"Left": [
"Uuid",
"Text"
]
},
"nullable": [
false
]
},
"hash": "19513c4158267cc7fe10d999ad571052c112e6bbb3cf834f16176cbb7e1ac319"
}
@@ -0,0 +1,22 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO token\n (token_hash, token_prefix, token, email, label, expiration, super_admin, scopes, read_only)\n VALUES ($1, $2, $3, $4, $5, now() + ($6 || ' seconds')::interval, $7, $8, $9)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Varchar",
"Text",
"Bool",
"TextArray",
"Bool"
]
},
"nullable": []
},
"hash": "7f832370916794ab0e5645053688c24678f1519d49ee7263a86dba71d45b8e8c"
}
@@ -0,0 +1,26 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT ws.workspace_id AS \"workspace_id!\", entry->'catalog'->>'resource_path' AS dbname\n FROM workspace_settings ws\n CROSS JOIN LATERAL jsonb_each(\n CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'\n THEN ws.ducklake->'ducklakes'\n ELSE '{}'::jsonb END\n ) AS dl(k, entry)\n WHERE entry->'catalog'->>'resource_type' = 'instance'\n AND entry->'catalog'->>'resource_path' IS NOT NULL\n UNION ALL\n SELECT ws.workspace_id AS \"workspace_id!\", entry->'database'->>'resource_path' AS dbname\n FROM workspace_settings ws\n CROSS JOIN LATERAL jsonb_each(\n CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'\n THEN ws.datatable->'datatables'\n ELSE '{}'::jsonb END\n ) AS dt(k, entry)\n WHERE entry->'database'->>'resource_type' = 'instance'\n AND entry->'database'->>'resource_path' IS NOT NULL\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id!",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "dbname",
"type_info": "Text"
}
],
"parameters": {
"Left": []
},
"nullable": [
null,
null
]
},
"hash": "815d96aea4681490582b08630a30a168cc1191acaab96bed6a016c437059c2cd"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "DELETE FROM background_task_state\n WHERE name LIKE $1\n AND updated_at < NOW() - INTERVAL '7 days'",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text"
]
},
"nullable": []
},
"hash": "8597cd40f80e69edbf1bc7d7402baca32e33e871be454acb5175c11361fe1b0a"
}
@@ -0,0 +1,24 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM v2_job WHERE id = $1 AND workspace_id = $2 AND tag = ANY($3))",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"Uuid",
"Text",
"TextArray"
]
},
"nullable": [
null
]
},
"hash": "8e8933fc6648a88dc35cd81559a31d10678d6c68fc920c876914e71324d5e460"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE app SET versions = array_append(versions, $1::bigint) WHERE path = $2 AND workspace_id = $3 AND versions[array_upper(versions, 1)] = $1::bigint",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Int8",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "afb0762c88d9232b79090f2e5966e78437a5e4d3b5e2341ec5f7725a28870270"
}
@@ -0,0 +1,16 @@
{
"db_name": "PostgreSQL",
"query": "\n INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms, account)\n VALUES ($1, $2, $3, true, '', '{}'::jsonb, NULL)\n ON CONFLICT (workspace_id, path) DO UPDATE SET value = EXCLUDED.value\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Varchar",
"Varchar"
]
},
"nullable": []
},
"hash": "b876f26ce90e30c3510eacddb03d9dd26fac05d18183f2d623ac91ea3876dd5c"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "\n SELECT j.id, j.args\n FROM v2_job j\n JOIN v2_job_queue q ON j.id = q.id\n WHERE j.runnable_path = $1\n AND j.kind = 'deploymentcallback'\n AND j.workspace_id = 'test-workspace'\n ORDER BY j.created_at DESC\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Uuid"
},
{
"ordinal": 1,
"name": "args",
"type_info": "Jsonb"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true
]
},
"hash": "bf601a919de299e44e6e418b2e711e24909fc206b7fd48541483e6316fe002d5"
}
@@ -0,0 +1,23 @@
{
"db_name": "PostgreSQL",
"query": "SELECT EXISTS(SELECT 1 FROM v2_job WHERE id = ANY($1) AND workspace_id = $2)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "exists",
"type_info": "Bool"
}
],
"parameters": {
"Left": [
"UuidArray",
"Text"
]
},
"nullable": [
null
]
},
"hash": "ca5bb402834502432f3d7260fdd5b9fb568a4c77e2a91f55575a93461d5a7f50"
}
@@ -0,0 +1,60 @@
{
"db_name": "PostgreSQL",
"query": "SELECT tag, script_lang AS \"script_lang: ScriptLang\" FROM v2_job WHERE id = $1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "tag",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "script_lang: ScriptLang",
"type_info": {
"Custom": {
"name": "script_lang",
"kind": {
"Enum": [
"python3",
"deno",
"go",
"bash",
"postgresql",
"nativets",
"bun",
"mysql",
"bigquery",
"snowflake",
"graphql",
"powershell",
"mssql",
"php",
"bunnative",
"rust",
"ansible",
"csharp",
"oracledb",
"nu",
"java",
"duckdb",
"ruby",
"rlang"
]
}
}
}
}
],
"parameters": {
"Left": [
"Uuid"
]
},
"nullable": [
false,
true
]
},
"hash": "cce5e3e639faed8e42574730cc66f0322a83c01cc465742f54a21f8fe5f4f037"
}
@@ -0,0 +1,29 @@
{
"db_name": "PostgreSQL",
"query": "SELECT hash, content FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "hash",
"type_info": "Int8"
},
{
"ordinal": 1,
"name": "content",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false,
false
]
},
"hash": "e88e1009f5359e205a523a32e4e8e72605971a3f417cb2b032c4d43652aca056"
}
@@ -1,16 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE workspace_runnable_dependencies SET app_path = REGEXP_REPLACE(app_path,'u/' || $2 || '/(.*)','u/' || $1 || '/\\1') WHERE app_path LIKE ('u/' || $2 || '/%') AND workspace_id = $3",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Text",
"Text",
"Text"
]
},
"nullable": []
},
"hash": "f699cc3644aeb35a0588bbb3a6bf2dc0746d9f3a1bea104123188fe2921bc886"
}
+283 -299
View File
File diff suppressed because it is too large Load Diff
+3 -2
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.710.1"
version = "1.718.0"
authors.workspace = true
edition.workspace = true
@@ -87,7 +87,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal", "./parsers/windmill-parser-wasm"]
[workspace.package]
version = "1.710.1"
version = "1.718.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -251,6 +251,7 @@ windmill-object-store.workspace = true
windmill-git-sync.workspace = true
windmill-api = { workspace = true, default-features = false }
windmill-api-agent-workers = { workspace = true, optional = true }
windmill-api-scripts.workspace = true
windmill-api-settings.workspace = true
windmill-worker.workspace = true
windmill-indexer = { workspace = true, optional = true }
+172
View File
@@ -0,0 +1,172 @@
# Threat Model: Windmill Backend
## 1. System context
Windmill is an open-source (AGPLv3) developer platform for internal tools,
workflows, background jobs, API integrations, and UIs — a self-hostable
alternative to Retool / Pipedream / Airplane. The backend is a Rust workspace
(~60 crates: `windmill-api`, `windmill-worker`, `windmill-queue`,
`windmill-common`, a family of `windmill-trigger-*` crates, `windmill-mcp`,
`windmill-sandbox`, etc.) fronting a PostgreSQL database. A Svelte 5 frontend
(not in scope here, but referenced where stored-XSS threats originate) is
served by the same instance. The product ships in a Community Edition (CE,
public Docker images) and an Enterprise Edition (EE, `*_ee.rs` files gated by
`enterprise`/`private`/`license` cargo features).
The defining characteristic for threat modeling is that **Windmill executes
arbitrary user-supplied code** (Python, TypeScript via Bun/Deno, Go, Bash,
SQL, GraphQL, PowerShell, Rust, …) on its workers, and **stores the
credentials to every system its users connect to** (databases, cloud
accounts, SaaS APIs, OAuth tokens). It is therefore simultaneously an
arbitrary-code-execution engine and a credential vault — compromising one
instance can pivot into an organization's entire connected estate. Crucially,
the owner confirms `nsjail` is **off by default everywhere** (`ENABLE_NSJAIL`
is opt-in) and network isolation (`clone_newnet`) is separately gated: the
*only* job isolation present in a default install is PID-namespace `unshare`.
Filesystem and outbound-network isolation are therefore absent unless an
operator deliberately enables them, which makes "weak-by-default isolation" a
more accurate frame than "sandbox escape" for typical deployments. Cross-tenant
separation is enforced in software via workspace IDs, token scopes, folder
ACLs, and Postgres row-level security; on the managed offering, sensitive
customers can opt into dedicated DB / worker / namespace infrastructure, but
the shared tier relies entirely on that software boundary. Administrators are
strongly encouraged to use nsjail sandboxing and are reminded that if they don't,
their security model is that they trust their developers that write code ran on windmill
to not do anything TOO malicious on the workers. When the default
database secret backend is used, only per-workspace secret *variables* are
encrypted at rest — instance-level `global_settings` (OAuth client secrets,
SMTP, object-store keys, license) are stored plaintext, so a database read
yields the instance-wide credential set. Internet-facing instances are
typically exposed directly with no built-in rate limiting or WAF.
It is deployed self-hosted (Docker Compose, Kubernetes/Helm, bare metal), on
cloud providers, and as a Windmill-Labs-managed multi-tenant service. The API
server is internet-facing in most deployments; workers pull jobs from the
Postgres queue. The large public attack surface (a sprawling authenticated
HTTP API, unauthenticated public-app and webhook/trigger endpoints, outbound
HTTP from user code and proxies) combined with the high-value assets makes
authorization-enforcement bugs, SSRF, SQL injection, and sandbox escape the
dominant risk categories — a pattern strongly confirmed by the project's
published advisory history (73 GHSA advisories, several rated 9.9 critical).
## 2. Assets
| asset | description | sensitivity |
|---|---|---|
| Workspace encryption keys | Per-workspace key (`workspace_key`) used to encrypt secret variables (MagicCrypt256); decrypts all secrets in the workspace | critical |
| Secret variables | User secrets stored encrypted in `variable` (is_secret) | critical |
| Resource credentials | DB passwords, cloud creds, API keys, connection strings in `resource` JSONB | critical |
| OAuth / external-account tokens | Refresh/access tokens in `account`, MCP OAuth tables | critical |
| User password hashes | Argon2 hashes in `password` table | critical |
| API tokens & session cookies | Bearer tokens / cookies in `token`; superadmin & scoped tokens | critical |
| Instance global settings | License key, JWT secret, SUPERADMIN_SECRET, SMTP, object-store + secret-backend (Vault/KMS/SM) creds in `global_settings` | critical |
| Worker host & process integrity | The host that runs untrusted user code | critical |
| Cross-tenant / cross-workspace isolation | The software boundary separating workspaces, folders, and tenants | critical |
| Downstream connected systems | Windmill is a credential vault: stored creds reach external DBs, cloud accounts, SaaS | critical |
| Script / flow / app source | Customer IP & business logic in `script`, `flow`, `app`, `raw_app` | high |
| Job arguments, results & logs | `queue`/`completed_job` args+result, `job_logs`; routinely contain secrets | high |
| Object store / S3 data | Files uploaded/produced by jobs | high |
| Audit logs | `audit`/`audit_partitioned` action trail | high |
| Service availability | API server + worker fleet uptime | high |
| PII | User emails, group membership | medium |
## 3. Entry points & trust boundaries
| entry_point | description | trust_boundary | reachable_assets |
|---|---|---|---|
| EP1 Authenticated job-execution API | `jobs/run/preview`, `run/h/{hash}`, `run_flow/run_script` — runs user code on workers | authenticated user → arbitrary code on worker | Worker host, downstream systems, isolation, job args/results/logs |
| EP2 Unauthenticated public endpoints | `apps_u/*`, `jobs_u/getupdate*`, `scripts_u`, `settings_u`, `resources_u` (`public_app_layer.rs`) | unauth HTTP → app logic & job data | Job results, scripts, secrets, PII |
| EP3 HTTP-trigger & webhook ingestion | `/api/r/*`, GCP/Azure push, Slack callback, `capture_u/*` | untrusted webhook → job queue | Job execution integrity, worker host |
| EP4 Message-queue / native triggers | kafka, postgres, mqtt, websocket, nats, sqs, email triggers | external broker/message → job queue | Job execution integrity, availability |
| EP5 HTTP API authorization layer | Token/scope/RLS/folder-ACL enforcement across all workspaced routes (`windmill-api-auth`) | scoped token / low-priv user → other users' & workspaces' data | Scripts, job data, secrets, isolation |
| EP6 AI proxy & MCP endpoints | `ai/proxy/*`, `mcp` — resolve `$var:`/resources, proxy to LLM APIs, `X-Resource-Path` | authenticated user → outbound HTTP + secret resolution | Secrets, resource creds, internal network, downstream |
| EP7 Outbound HTTP from executors/resources | GraphQL/HTTP/Postgres executors, webhook delivery, `test_object_storage_config`, git clone, npm tarball fetch | user-controlled URL → server-side request | Cloud metadata, internal network, downstream creds |
| EP8 SQL query builders & contextual-var substitution | App DB query builder (`whereClause`/`tags`), Postgres-trigger `where_clause`, `%%WM_*%%` interpolation, `WM_INTERNAL_DB` | user input → raw SQL | Database, connected DBs |
| EP9 Worker sandbox | nsjail / unshare / dind / rootless podman isolating user code | user code → host & cross-tenant filesystem/network | Worker host, isolation, downstream |
| EP10 Worker code generation / wrappers | Entrypoint override, env-var names, workspace env interpolated into generated wrapper code | user-controlled identifier → executable code | Worker host, isolation |
| EP11 OAuth / OIDC / SAML / MCP-OAuth / logout | Login callbacks, MCP OAuth client registration, logout `rd` redirect | untrusted IdP / redirect input → session | Session tokens, accounts |
| EP12 Stored-content rendering | App builder HTML component, markdown, S3 download response headers | stored user content → admin browser (same origin) | Admin session, account takeover |
| EP13 Log/file reading & export endpoints | `service_logs`, `jobs_u/getupdate` log file read (symlinks), workspace/tarball export | authed/unauth request → arbitrary file or admin-only config | Arbitrary files, global settings |
| EP14 Secret-value & resource-value caches | In-memory caches in `windmill-store` keyed (historically un-keyed) by path | cache lookup crossing identity/folder boundary | Secret variables, resource creds |
| EP15 Deployment & runtime config | docker-compose defaults: dind, debugger (`REQUIRE_SIGNED_DEBUG_REQUESTS=false`), CORS `Any`, default admin/`changeme`, exposed Postgres, `SUPERADMIN_SECRET`, `ENABLE_NSJAIL=false`, privileged containers | operator/infra default → full instance | All assets |
| EP16 Supply chain | Cached hub scripts, GitHub workflow actions, vendored deps, Docker base image | build/update-time input → host & build integrity | Worker host, build integrity |
| EP17 Token lifecycle | Token create/rescope/refresh, script-issued JWTs | scoped caller → broader privilege | Tokens, accounts, isolation |
## 4. Threats
| id | threat | actor | surface | asset | impact | likelihood | status | controls | evidence |
|---|---|---|---|---|---|---|---|---|---|
| T1 | SQL injection in app/internal query builders and trigger clauses compromises the metadata DB and connected databases | remote_auth | EP8 | Database, downstream connected systems | critical | almost_certain | partially_mitigated | sqlx parameterized queries elsewhere; query-builder safety reviews | GHSA-225c-j3xq-g6x6, GHSA-78p7-jc72-gv66, GHSA-hvc7-f67h-jx3g, GHSA-wrrg-f89m-f84q, GHSA-79vf-3qwm-2w64, GHSA-55p6-fxj4-v983, GHSA-5g4v-49rj-r52r, GHSA-x6cq-7xr8-53x3, 2cf4bb180b |
| T2 | Server-side request forgery via proxies/executors reaches cloud metadata, internal network, and downstream credentials | remote_auth | EP6, EP7 | Cloud metadata, internal network, downstream connected systems, resource creds | critical | almost_certain | partially_mitigated | SSRF URL validation + redirect-following disabled added piecemeal; outbound network isolation (`clone_newnet`) is opt-in and off by default | GHSA-3ggp-h37f-5qfw, GHSA-98qq-g8rh-xhff, GHSA-hfw8-27mx-63jm, GHSA-3r59-qvvc-774j, GHSA-4pj9-w5jc-g8w7, GHSA-8hh3-jf25-78j5, GHSA-3pjm-4w7f-3r2w, GHSA-f44c-x9hq-h68r, GHSA-j4h4-f8fj-3m3c, 4b06881918, 96a8eb63d4, dbd3942ef3 |
| T3 | Broken authorization / IDOR lets a scoped token or low-privilege member read scripts, job data, and secrets across folders and workspaces | remote_auth | EP5, EP2, EP1 | Scripts, job data, secrets, isolation | critical | almost_certain | partially_mitigated | RLS, token scopes, folder ACLs, view-token HMAC (added incrementally); on managed, sensitive tenants can opt into dedicated DB/worker/namespace, but the shared tier IS the software boundary | GHSA-qfg7-x243-5hg4, GHSA-8x8x-88qc-qp4r, GHSA-2ppx-66jv-wpw5, GHSA-x3x7-g97v-mp59, GHSA-j276-g4h8-g6h5, GHSA-8mv7-hmrg-96xv, GHSA-x2wf-f962-7frq, GHSA-qc7c-gcw6-h4xp, GHSA-vxc5-w28p-m9xw, GHSA-2g34-wfvr-5qqj, GHSA-w7p6-wpxm-pp66, 7edf3f0212, 89a7a37776, ab11c7747a, 664edcdfb7 |
| T4 | Remote code execution by injecting attacker-controlled identifiers into generated worker wrappers | remote_auth | EP10 | Worker host, isolation, downstream | critical | likely | partially_mitigated | entrypoint/env-var-name validation added | GHSA-wxjq-w5pj-jqhx, GHSA-5f5q-2vg2-r2x4, GHSA-8q8j-mm3g-5c2q (CVE-2026-33881), bf93657fee, bd05bcadde, 22ec4da5f0 |
| T5 | Worker compromise & cross-tenant access via weak-by-default isolation (nsjail off by default → user code runs with only PID-ns `unshare`); sandbox escape where nsjail/dind/podman is enabled | remote_auth | EP9, EP15 | Worker host, isolation, downstream | critical | likely | unmitigated | nsjail off by default everywhere (`DISABLE_NSJAIL=true`); shipped compose gives PID-ns `unshare` only (`FAVOR_UNSHARE_PID=true`), bare installs get no isolation. Where nsjail enabled: read-only remounts, jail-tmp refusal, podman socket gating | GHSA-6qr8-xhg4-453q, GHSA-3vpp-vf62-wqp6, f8467f38c8, df5aec0f5d, f1b6746e0e |
| T6 | Disclosure of secrets, resource credentials, and workspace encryption keys across the authorization boundary (AI proxy, MCP, caches, export); database read additionally yields plaintext instance-level `global_settings` secrets | remote_auth | EP6, EP14, EP13 | Secret variables, encryption keys, resource creds, global settings | critical | likely | partially_mitigated | RLS on `$var:`, cache scoping by caller, admin checks on export; per-workspace secret *variables* encrypted at rest, but `global_settings` is plaintext under the default DB secret backend | GHSA-jwg4-v3cj-rvfm, GHSA-8m2p-2crh-9h3w, GHSA-6635-6fch-v8px, GHSA-437f-725p-7w84, GHSA-f27g-j463-q85w (CVE-2026-26964), GHSA-j679-v6vj-jfxc, GHSA-6vrr-fq33-qpfp, 0ba128afe7, 7836a4e733, ff8e39c69b |
| T7 | Full instance compromise from insecure deployment defaults (dind control, default admin/`changeme`, exposed Postgres, publicly readable SUPERADMIN_SECRET) | remote_unauth | EP15 | All assets | critical | likely | partially_mitigated | first-time-setup warning on default admin; docs recommend hardening | GHSA-3vpp-vf62-wqp6, GHSA-24fr-44f8-fqwg (CVE-2026-29059), GHSA-6q36-5p3h-766j |
| T8 | Unauthenticated RCE via the Debugger WebSocket in the default `windmill_extra` configuration | remote_unauth | EP15 | Worker host, all assets | critical | possible | unmitigated | `REQUIRE_SIGNED_DEBUG_REQUESTS` exists but defaults to false | GHSA-725h-99vx-9xr4 |
| T9 | Supply-chain compromise via cached hub scripts, GitHub workflow command injection, or vulnerable base-image deps | supply_chain | EP16 | Worker host, build integrity | critical | possible | partially_mitigated | hub-script re-pin to patched versions; HUB_BASE_URL override | GHSA-w2m9-q5f7-3gpq, edf340c4d4, GHSA-8rq7-w7g6-8wvr, GHSA-vch9-39v5-4wg7 (CVE-2024-37371) |
| T10 | Unauthenticated disclosure of job results, args, logs, and admin config via missing-authz public endpoints | remote_unauth | EP2, EP13 | Job results/args/logs, global settings, scripts | high | likely | partially_mitigated | anonymous-job checks, log-endpoint authz hardening | GHSA-qfg7-x243-5hg4, GHSA-v448-fmm4-52fp, 108a88a180, bb90f4ce83 |
| T11 | Stored XSS leading to admin/account takeover via app HTML component, markdown, or S3 download content-type | remote_auth | EP12 | Admin session, accounts | high | likely | partially_mitigated | DOMPurify markdown sanitization, `X-Content-Type-Options: nosniff` + CSP sandbox on downloads | GHSA-9c5c-hh3c-r9mc, GHSA-qxj7-hpx3-r892, GHSA-cf2x-rg8c-v63v, bb78b1c06d, 625b67dff0 |
| T12 | Webhook authentication bypass / signature replay forges trigger invocations and approvals | remote_unauth | EP3 | Job execution integrity, approvals | high | likely | partially_mitigated | HMAC verification on some triggers; signing-oracle fix | GHSA-jw8c-h45c-xpjw, GHSA-hh9x-rcf8-xjr2, GHSA-q9g3-q6fj-hc2x, GHSA-8jc4-wj2p-2vmp, ab2a15b2a8 |
| T13 | Path traversal / arbitrary file read via log-reading and MCP path endpoints (incl. symlink following) | remote_auth | EP13 | Arbitrary files on server, global settings | high | likely | partially_mitigated | traversal checks + no-symlink-follow added | GHSA-4hrf-mgvv-xp9x, bb90f4ce83, df451aa64f, ad5ec293b5, 5f2d3e6812 |
| T14 | Privilege escalation via token rescope/refresh, script-issued JWTs, or operator-permission gaps | remote_auth | EP17, EP5 | Tokens, isolation, accounts | high | likely | partially_mitigated | monotonic-privilege enforcement on token lifecycle; SECURITY DEFINER triggers | GHSA-p62p-67xp-v775, GHSA-vv9w-wx3c-q3x2, 2ddf93de96, 865ab70c89, 33fb08cf3d |
| T15 | Credential leakage via worker `/proc` environment and unmasked secrets in job logs | remote_auth | EP9, EP1 | DB creds, secrets, downstream | high | likely | partially_mitigated | Aho-Corasick secret masking in logs | GHSA-pmp9-9924-f9cx, 0885d8c986 |
| T16 | Denial of service via resource exhaustion: unbounded uploads, runaway jobs, queue flooding, or trigger-message storms | remote_auth | EP1, EP3, EP4 | Service availability, worker fleet | high | likely | risk_accepted | Per-job rlimits/timeouts exist; instance-wide DoS by an authenticated tenant is largely accepted on shared self-host (operator's job to add global quotas). Hard requirement only for managed multi-tenant | |
| T17 | Account/credential theft via unauthenticated MCP-OAuth client registration and open redirect on logout | remote_unauth | EP11 | Accounts, session tokens | high | possible | partially_mitigated | redirect-URI handling / registration hardening | GHSA-q9xg-f2v2-695g, GHSA-53xj-pvqf-wpm9, GHSA-rr8j-ffc4-pf7h, GHSA-6c5w-777m-8rv5 |
| T18 | Account takeover via missing rate limiting / brute force on auth endpoints | remote_unauth | EP11 | Accounts | medium | likely | unmitigated | none built-in; owner confirms instances are typically exposed directly with no app-level rate limiting or WAF | GHSA-cmv6-m7wc-c87p |
| T19 | Enterprise license bypass and account impersonation | remote_auth | EP5 | Global settings, accounts | medium | possible | unmitigated | license validation gated by `license` feature | GHSA-48j5-p323-4mpx, GHSA-pv35-65rq-w29h, GHSA-2qx7-634r-qj6r |
| T20 | Trigger spoofing: an actor with broker/queue access injects messages that execute jobs without app-level auth | adjacent_network | EP4 | Job execution integrity, downstream | medium | possible | risk_accepted | Owner confirms trust is delegated to broker ACLs by design; no app-level message authenticity check. Anyone able to publish to a subscribed topic/queue can cause job execution | |
| T21 | Data-in-transit interception/tampering from TLS-disabled defaults (DB `sslmode=disable`, HTTP-only Caddy) | adjacent_network | EP15 | DB creds, secrets, session tokens | medium | possible | unmitigated | docs recommend TLS; not default | |
| T22 | Repudiation / incident blind spots from gaps in audit coverage of sensitive actions | remote_auth | EP5 | Audit logs | medium | possible | partially_mitigated | `windmill-audit` records many actions | |
## 5. Deprioritized
| threat | reason |
|---|---|
| Physical access to the host / cold-boot key extraction | Out of scope; deployment-environment responsibility, not addressable in this codebase |
| Memory-safety RCE in the Rust backend itself | Rust's safety model makes this rare; no evidence in history. Note: `unsafe` FFI (duckdb) is a narrow exception folded into supply-chain/T9 |
| Client-side-only nuisance bugs (CSS, layout) with no security impact | No asset compromised |
| Insider with legitimate superadmin / DB-root access | Trusted role; mitigations are operational (least privilege, audit), not technical controls in scope |
| Spoofing of a fully-trusted upstream IdP that has itself been compromised | Out of model; Windmill trusts the configured IdP by design |
| Instance-wide DoS by an authenticated tenant on shared self-host (T16) | Risk accepted (owner): per-job rlimits/timeouts are in place; global concurrency/queue quotas are the operator's responsibility on self-host. Remains a hard requirement for the managed multi-tenant fleet |
| Job execution triggered by an actor with legitimate broker/queue publish access (T20) | Risk accepted (owner): trigger authenticity is delegated to broker ACLs by design; consuming from a configured source and acting on its messages is the intended behavior |
## 6. Open questions
Facts that drove the score changes above. Two were confirmed in code during
the interview (`[Code-verified]`); the rest remain `[Owner-states]` pending a
check.
- [Code-verified] nsjail is off by default in every configuration: `DISABLE_NSJAIL` defaults to `true` (`windmill-worker/src/worker.rs:346`), and `is_sandboxing_enabled()` requires `DISABLE_NSJAIL=false` or the `job_isolation` global setting = `nsjail_sandboxing` (`worker.rs:890`). PID-ns `unshare` is also off at the code level (`is_unshare_enabled()`, `worker.rs:903`); the shipped `docker-compose.yml` sets `FAVOR_UNSHARE_PID=true` (line 91), so the official compose gives PID-ns unshare only, nsjail off — a bare install gets no isolation at all. No separate `clone_newnet` flag exists; network isolation is an nsjail feature, so outbound network from user code is unrestricted by default. Affects: T2 controls/likelihood, T5 status (unmitigated), T8.
- [Code-verified] `global_settings` is plaintext at rest under the default DB backend: `set_value_in_global_settings` stores the raw JSON value with no encryption (`windmill-common/src/global_settings.rs:259`); the encrypting secret backend (`secret_backend/database.rs:66`) only encrypts per-workspace `variable` rows with `is_secret=true`. Instance-level SMTP/OAuth/AI/object-store secrets are therefore plaintext. Affects: T6 impact/controls, T7.
- [Owner-states] Internet-facing instances are typically exposed directly with no built-in rate limiting / WAF. Affects: T16, T18 likelihood. Verify by: confirm absence of a rate-limit layer in `windmill-api/src/lib.rs` middleware stack.
- [Owner-states] Managed offering provides an optional dedicated DB/worker/namespace tier for sensitive tenants; the shared tier relies solely on the software authz boundary. Affects: T3 controls. Verify by: deployment topology (not in this repo) — out-of-tree.
- [Owner-states] Per-job rlimits/timeouts exist; instance-wide DoS by an authed tenant is risk-accepted on shared self-host. Affects: T16 status. Verify by: locate the rlimit/timeout enforcement in the worker execution path and confirm there is no global queue/concurrency cap.
- [Owner-states] Message-queue trigger authenticity is delegated to broker ACLs only. Affects: T20 status. Verify by: review `windmill-trigger-{kafka,sqs,nats,mqtt,postgres}` consume paths for any payload authentication.
## 7. Provenance
- mode: bootstrap-then-interview
- date: 2026-06-05
- target: /home/rfiszel/windmill/backend @ 819ba5e150
- inputs: git-log mined + GitHub security advisories (gh api, 73 advisories) + CHANGELOG; seed: THREAT_MODEL.md (bootstrap pass)
- owner: Ruben Fiszel (Windmill core dev)
## 8. Recommended mitigations
| mitigation | threat_ids | closes_class | effort |
|---|---|---|---|
| Centralize a single audited query-builder that forbids string-interpolated SQL; ban `format!`-built queries via lint/CI | T1 | yes | M |
| Route all outbound requests through one SSRF-guarded HTTP client (allowlist/denylist of private+metadata ranges, redirects disabled, re-validated per hop) | T2 | yes | M |
| Enforce authorization centrally in middleware (scope + RLS + folder ACL) with deny-by-default and a per-route coverage test, instead of per-handler checks | T3, T10, T14, T22 | yes | L |
| Treat all user-supplied identifiers as data: pass via argv/env/structured params, never splice into generated wrapper source; validate against strict allowlists at the boundary | T4 | yes | M |
| Make `nsjail` + network-namespace isolation default-on / fail-closed (flip `ENABLE_NSJAIL` and `clone_newnet` defaults) and remove privileged/dind defaults from shipped compose; default-deny debugger | T2, T5, T7, T8 | partial | L |
| Encrypt `global_settings` at rest under the workspace/instance key even on the default DB secret backend, so a DB read no longer yields plaintext instance-wide credentials | T6, T7 | partial | M |
| Ship hardened defaults: random per-install secrets, no default admin password, Postgres not exposed, CORS locked to configured origin, TLS-on | T7, T18, T21 | partial | M |
| Resolve secrets/resources only with the caller's identity and scope every cache entry by (caller, scope); apply uniformly to AI proxy, MCP, and exports | T6 | yes | M |
| Output-encode/sanitize all stored content at render and force `nosniff` + restrictive CSP on every user-content response | T11 | yes | M |
| Verify webhook authenticity uniformly (constant-time HMAC + timestamp/nonce anti-replay) in a shared trigger-auth helper | T12 | yes | S |
| Canonicalize + confine all file-path inputs to a base dir and never follow symlinks in log/file readers | T13 | yes | S |
| Mask secrets at the log sink and keep secrets out of worker process env (`/proc`) — pass via files/pipes scrubbed after use | T15 | partial | M |
| Add global rate limiting and per-tenant resource/queue quotas at the edge | T16, T18 | partial | M |
| Pin and integrity-verify hub scripts and CI actions; SBOM + automated base-image CVE scanning in release | T9 | partial | M |
+1 -1
View File
@@ -1 +1 @@
31cda7cea811108e5004842ec5b5467ca48181cd
2c7964460327fab5e3a27c0f74b8d6f26ab7f79a
+44 -11
View File
@@ -154,28 +154,61 @@
"zoho": {
"auth_url": "https://accounts.zoho.com/oauth/v2/auth",
"token_url": "https://accounts.zoho.com/oauth/v2/token",
"scopes": [
"ZohoAssist.sessionapi.ALL"
],
"scopes": ["ZohoAssist.sessionapi.ALL"],
"extra_params": {
"access_type": "offline"
}
},
"snowflake_oauth": {},
"snowflake_oauth": {
"connect_config_template": {
"display_name": "Snowflake",
"label": "Snowflake Account Identifier",
"placeholder": "<orgname>-<account_name>",
"help_url": "https://docs.snowflake.com/en/user-guide/admin-account-identifier#using-an-account-name-as-an-identifier",
"auth_url": "https://{instance}.snowflakecomputing.com/oauth/authorize",
"token_url": "https://{instance}.snowflakecomputing.com/oauth/token-request",
"req_body_auth": false,
"extra_params_key": "account_identifier",
"resource_mapping": { "account_identifier": "{instance}" }
}
},
"apify": {
"auth_url": "https://console.apify.com/authorize/oauth",
"token_url": "https://console-backend.apify.com/oauth/apps/token",
"scopes": [
"profile",
"full_api_access"
],
"scopes": ["profile", "full_api_access"],
"extra_params": {}
},
"docusign": {
"auth_url": "https://account.docusign.com/oauth/auth",
"token_url": "https://account.docusign.com/oauth/token",
"scopes": [
"signature"
]
"scopes": ["signature"],
"sandbox": {
"auth_url": "https://account-d.docusign.com/oauth/auth",
"token_url": "https://account-d.docusign.com/oauth/token"
}
},
"salesforce": {
"auth_url": "https://login.salesforce.com/services/oauth2/authorize",
"token_url": "https://login.salesforce.com/services/oauth2/token",
"scopes": ["api", "refresh_token", "offline_access"],
"sandbox": {
"auth_url": "https://test.salesforce.com/services/oauth2/authorize",
"token_url": "https://test.salesforce.com/services/oauth2/token"
}
},
"servicenow": {
"connect_config_template": {
"display_name": "ServiceNow",
"label": "ServiceNow Instance",
"placeholder": "<instance> (e.g. dev12345)",
"help_url": "https://www.servicenow.com/docs/bundle/zurich-platform-security/page/administer/security/concept/c_OAuthApplications.html",
"auth_url": "https://{instance}.service-now.com/oauth_auth.do",
"token_url": "https://{instance}.service-now.com/oauth_token.do",
"req_body_auth": true,
"strip_suffix": ".service-now.com",
"resource_mapping": {
"instance_url": "https://{instance}.service-now.com"
}
}
}
}
@@ -129,7 +129,10 @@ impl Visit for ImportsFinder {
/// See also: [`parse_relative_imports`] for resolved absolute paths.
pub fn parse_expr_for_imports(code: &str, skip_type_only: bool) -> anyhow::Result<Vec<String>> {
let cm: Lrc<SourceMap> = Default::default();
let fm = cm.new_source_file(FileName::Custom("main.d.ts".into()).into(), code.to_string());
let fm = cm.new_source_file(
FileName::Custom("main.d.ts".into()).into(),
code.to_string(),
);
let mut tss = TsSyntax::default();
tss.disallow_ambiguous_jsx_like;
tss.tsx = true;
+24 -24
View File
@@ -6183,7 +6183,7 @@ checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
[[package]]
name = "windmill-common"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"aho-corasick",
"anyhow",
@@ -6263,7 +6263,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"proc-macro2",
"quote",
@@ -6275,7 +6275,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"convert_case",
"serde",
@@ -6284,7 +6284,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6296,7 +6296,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6308,7 +6308,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"gosyn",
@@ -6320,7 +6320,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6332,7 +6332,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6344,7 +6344,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -6355,7 +6355,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6366,7 +6366,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -6378,7 +6378,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-asset"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6389,7 +6389,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -6411,7 +6411,7 @@ dependencies = [
[[package]]
name = "windmill-parser-r"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde_json",
@@ -6423,7 +6423,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6437,7 +6437,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"convert_case",
@@ -6454,7 +6454,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6467,7 +6467,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql-asset"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde",
@@ -6479,7 +6479,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -6497,7 +6497,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts-asset"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde-wasm-bindgen",
@@ -6513,7 +6513,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wac"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"rustpython-ast",
@@ -6529,7 +6529,7 @@ dependencies = [
[[package]]
name = "windmill-parser-wasm"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"getrandom 0.2.17",
@@ -6561,7 +6561,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"serde",
@@ -6572,7 +6572,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.710.1"
version = "1.718.0"
dependencies = [
"anyhow",
"bitflags",
@@ -12,7 +12,7 @@ resolver = "2"
members = ["."]
[workspace.package]
version = "1.710.1"
version = "1.718.0"
edition = "2021"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
+42 -7
View File
@@ -57,11 +57,14 @@ use windmill_common::{
PREVIEW_TAGS_OVERRIDE_SETTING, REQUEST_SIZE_LIMIT_SETTING,
REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING, RESTART_COORDINATION_SETTING,
RETENTION_PERIOD_SECS_SETTING, RUBY_REPOS_SETTING, SAML_METADATA_SETTING,
SCIM_TOKEN_SETTING, SMTP_SETTING, STORE_AUDIT_LOGS_S3_SETTING, TEAMS_SETTING,
TIMEOUT_WAIT_RESULT_SETTING, UV_EXCLUDE_NEWER_SETTING, UV_INDEX_STRATEGY_SETTING,
UV_PYTHON_INSTALL_MIRROR_SETTING, WORKSPACE_FAIRNESS_DURATION_SECS_SETTING,
WORKSPACE_FAIRNESS_ENABLED_SETTING, WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING,
WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING, WORKSPACE_REGISTRIES_SETTING,
SANDBOX_IMAGE_CACHE_MAX_MB_SETTING, SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING,
SANDBOX_IMAGE_MAX_SIZE_MB_SETTING, SANDBOX_IMAGE_PULL_POLICY_SETTING,
SANDBOX_REGISTRY_AUTH_SETTING, SCIM_TOKEN_SETTING, SMTP_SETTING,
STORE_AUDIT_LOGS_S3_SETTING, TEAMS_SETTING, TIMEOUT_WAIT_RESULT_SETTING,
UV_EXCLUDE_NEWER_SETTING, UV_INDEX_STRATEGY_SETTING, UV_PYTHON_INSTALL_MIRROR_SETTING,
WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
WORKSPACE_FAIRNESS_MAX_PERCENT_SETTING, WORKSPACE_FAIRNESS_MIN_TOTAL_SETTING,
WORKSPACE_REGISTRIES_SETTING,
},
scripts::ScriptLang,
stats_oss::schedule_stats,
@@ -134,8 +137,11 @@ use crate::monitor::{
reload_job_default_timeout_setting, reload_job_isolation_setting, reload_jwt_secret_setting,
reload_license_key, reload_npm_config_registry_setting, reload_nsjail_tmp_backing_setting,
reload_nsjail_tmpfs_size_setting, reload_otel_tracing_proxy_setting,
reload_pip_index_url_setting, reload_retention_period_setting, reload_scim_token_setting,
reload_smtp_config, reload_store_audit_logs_s3_setting, reload_uv_exclude_newer_setting,
reload_pip_index_url_setting, reload_retention_period_setting,
reload_sandbox_image_cache_max_setting, reload_sandbox_image_default_registry_setting,
reload_sandbox_image_max_size_setting, reload_sandbox_image_pull_policy_setting,
reload_sandbox_registry_auth_setting, reload_scim_token_setting, reload_smtp_config,
reload_store_audit_logs_s3_setting, reload_uv_exclude_newer_setting,
reload_uv_index_strategy_setting, reload_uv_python_install_mirror_setting,
reload_worker_config, MonitorIteration,
};
@@ -258,6 +264,15 @@ pub fn main() -> anyhow::Result<()> {
}
async fn cache_hub_scripts(file_path: Option<String>) -> anyhow::Result<()> {
// The `cache` CLI mode never connects to the DB, so HUB_BASE_URL keeps its
// compiled default. Allow overriding it via env so the prebuild cache step can
// be pointed at a private/staging hub (e.g. a local proxy for testing).
if let Ok(hub_base_url) = std::env::var("HUB_BASE_URL") {
if !hub_base_url.is_empty() {
tracing::info!("Overriding hub base url from env: {hub_base_url}");
windmill_common::HUB_BASE_URL.store(std::sync::Arc::new(hub_base_url));
}
}
let file_path = file_path.unwrap_or("./hubPaths.json".to_string());
let mut file = File::open(&file_path)
.await
@@ -567,6 +582,7 @@ fn print_help() {
println!(" RUN_UPDATE_CA_CERTIFICATE_AT_START = false Run system CA update at startup");
println!(" RUN_UPDATE_CA_CERTIFICATE_PATH = /usr/sbin/update-ca-certificates Path to CA update tool");
println!(" SYNC_CACHED_RT = false Sync cached resource types to admins workspace on server start");
println!(" HUB_BASE_URL = https://hub.windmill.dev Hub to fetch scripts from in `cache` mode (server/worker use the DB setting instead)");
println!();
println!("Notes:");
println!("- Advanced and less commonly used settings are managed via the database and are omitted here.");
@@ -1654,6 +1670,12 @@ async fn process_notify_event(
match *source_type {
"script" => {
windmill_common::DEPLOYED_SCRIPT_HASH_CACHE.remove(&key);
// Evict the relative-import latest-hash cache so a redeployed
// imported script flips the content cache to its new version
// across all replicas within a poll interval (see #6769). Keyed
// by the bare path, matching this event's payload.
windmill_api_scripts::scripts::RAW_SCRIPT_LATEST_HASH_CACHE
.remove(&format!("{workspace_id}:{path}"));
if *kind == "preprocessor" {
match sqlx::query_scalar::<_, i64>(
"SELECT fv.id
@@ -1811,6 +1833,19 @@ async fn process_notify_event(
JOB_ISOLATION_SETTING => reload_job_isolation_setting(conn).await,
NSJAIL_TMPFS_SIZE_MB_SETTING => reload_nsjail_tmpfs_size_setting(conn).await,
NSJAIL_TMP_BACKING_SETTING => reload_nsjail_tmp_backing_setting(conn).await,
SANDBOX_IMAGE_MAX_SIZE_MB_SETTING => {
reload_sandbox_image_max_size_setting(conn).await
}
SANDBOX_IMAGE_CACHE_MAX_MB_SETTING => {
reload_sandbox_image_cache_max_setting(conn).await
}
SANDBOX_IMAGE_PULL_POLICY_SETTING => {
reload_sandbox_image_pull_policy_setting(conn).await
}
SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING => {
reload_sandbox_image_default_registry_setting(conn).await
}
SANDBOX_REGISTRY_AUTH_SETTING => reload_sandbox_registry_auth_setting(conn).await,
#[cfg(feature = "parquet")]
OBJECT_STORE_CONFIG_SETTING => {
if !disable_s3_store {
+93 -3
View File
@@ -66,7 +66,9 @@ use windmill_common::{
OTEL_SETTING, OTEL_TRACING_PROXY_SETTING, PIP_INDEX_URL_SETTING,
POWERSHELL_REPO_PAT_SETTING, POWERSHELL_REPO_URL_SETTING, PREVIEW_TAGS_OVERRIDE_SETTING,
REQUEST_SIZE_LIMIT_SETTING, REQUIRE_PREEXISTING_USER_FOR_OAUTH_SETTING,
RETENTION_PERIOD_SECS_SETTING, SAML_METADATA_SETTING, SCIM_TOKEN_SETTING,
RETENTION_PERIOD_SECS_SETTING, SAML_METADATA_SETTING, SANDBOX_IMAGE_CACHE_MAX_MB_SETTING,
SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING, SANDBOX_IMAGE_MAX_SIZE_MB_SETTING,
SANDBOX_IMAGE_PULL_POLICY_SETTING, SANDBOX_REGISTRY_AUTH_SETTING, SCIM_TOKEN_SETTING,
STORE_AUDIT_LOGS_S3_SETTING, TIMEOUT_WAIT_RESULT_SETTING, UV_EXCLUDE_NEWER_SETTING,
UV_INDEX_STRATEGY_SETTING, UV_PYTHON_INSTALL_MIRROR_SETTING,
WORKSPACE_FAIRNESS_DURATION_SECS_SETTING, WORKSPACE_FAIRNESS_ENABLED_SETTING,
@@ -112,8 +114,10 @@ use windmill_worker::{
JOB_DEFAULT_TIMEOUT, JOB_ISOLATION, KEEP_JOB_DIR, MAVEN_REPOS, MAVEN_SETTINGS_XML,
NO_DEFAULT_MAVEN, NPMRC, NPM_CONFIG_REGISTRY, NSJAIL_AVAILABLE, NSJAIL_TMPFS_SIZE_MB,
NSJAIL_TMP_BACKING, NUGET_CONFIG, OTEL_TRACING_PROXY_SETTINGS, PIP_EXTRA_INDEX_URL,
PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, UNSHARE_PATH, UV_EXCLUDE_NEWER,
UV_INDEX_STRATEGY, UV_PYTHON_INSTALL_MIRROR, WORKSPACE_REGISTRIES,
PIP_INDEX_URL, POWERSHELL_REPO_PAT, POWERSHELL_REPO_URL, SANDBOX_IMAGE_CACHE_MAX_MB,
SANDBOX_IMAGE_DEFAULT_REGISTRY, SANDBOX_IMAGE_MAX_SIZE_MB, SANDBOX_IMAGE_PULL_POLICY,
SANDBOX_REGISTRY_AUTH, UNSHARE_PATH, UV_EXCLUDE_NEWER, UV_INDEX_STRATEGY,
UV_PYTHON_INSTALL_MIRROR, WORKSPACE_REGISTRIES,
};
#[cfg(feature = "parquet")]
@@ -407,6 +411,11 @@ pub async fn initial_load(
reload_job_isolation_setting(&conn).await;
reload_nsjail_tmpfs_size_setting(&conn).await;
reload_nsjail_tmp_backing_setting(&conn).await;
reload_sandbox_image_max_size_setting(&conn).await;
reload_sandbox_image_cache_max_setting(&conn).await;
reload_sandbox_image_pull_policy_setting(&conn).await;
reload_sandbox_image_default_registry_setting(&conn).await;
reload_sandbox_registry_auth_setting(&conn).await;
reload_extra_pip_index_url_setting(&conn).await;
reload_pip_index_url_setting(&conn).await;
reload_uv_index_strategy_setting(&conn).await;
@@ -2045,6 +2054,66 @@ pub async fn reload_nsjail_tmp_backing_setting(conn: &Connection) {
.await;
}
pub async fn reload_sandbox_image_max_size_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
SANDBOX_IMAGE_MAX_SIZE_MB_SETTING,
"SANDBOX_IMAGE_MAX_SIZE_MB",
SANDBOX_IMAGE_MAX_SIZE_MB.clone(),
)
.await;
}
pub async fn reload_sandbox_image_cache_max_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
SANDBOX_IMAGE_CACHE_MAX_MB_SETTING,
"SANDBOX_IMAGE_CACHE_MAX_MB",
SANDBOX_IMAGE_CACHE_MAX_MB.clone(),
)
.await;
}
pub async fn reload_sandbox_image_pull_policy_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
SANDBOX_IMAGE_PULL_POLICY_SETTING,
"SANDBOX_IMAGE_PULL_POLICY",
SANDBOX_IMAGE_PULL_POLICY.clone(),
)
.await;
}
pub async fn reload_sandbox_image_default_registry_setting(conn: &Connection) {
reload_option_setting_with_tracing(
conn,
SANDBOX_IMAGE_DEFAULT_REGISTRY_SETTING,
"SANDBOX_IMAGE_DEFAULT_REGISTRY",
SANDBOX_IMAGE_DEFAULT_REGISTRY.clone(),
)
.await;
}
pub async fn reload_sandbox_registry_auth_setting(conn: &Connection) {
// Secret-aware: the value is a raw docker/podman auth.json with credentials, so
// it must never be logged. Load directly (the generic reload_option_setting path
// logs the value via load_option_setting_value) and only log a redacted message.
let q =
match load_value_from_global_settings_with_conn(conn, SANDBOX_REGISTRY_AUTH_SETTING, true)
.await
{
Ok(q) => q,
Err(e) => {
tracing::error!("Error reloading setting SANDBOX_REGISTRY_AUTH: {e:?}");
return;
}
};
let value = q.and_then(|q| serde_json::from_value::<String>(q).ok());
let configured = value.as_ref().is_some_and(|v| !v.trim().is_empty());
*SANDBOX_REGISTRY_AUTH.write().await = value;
tracing::info!("Loaded setting SANDBOX_REGISTRY_AUTH (redacted), configured={configured}");
}
pub async fn reload_job_isolation_setting(conn: &Connection) {
let value =
match load_value_from_global_settings_with_conn(conn, JOB_ISOLATION_SETTING, true).await {
@@ -2705,6 +2774,26 @@ pub async fn monitor_db(
}
};
// run every hour (120 iterations * 30s = 3600s)
let cleanup_stale_server_heartbeats_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(120) {
if let Some(db) = conn.as_sql() {
match windmill_api::cleanup_stale_server_heartbeats(db).await {
Ok(count) if count > 0 => {
tracing::info!(
"Deleted {} stale server_heartbeat background_task_state rows",
count
);
}
Err(e) => {
tracing::error!("Error cleaning up stale server_heartbeat rows: {:?}", e);
}
_ => {}
}
}
}
};
// run every hour (120 iterations * 30s = 3600s)
let manage_audit_partitions_f = async {
if server_mode && iteration.is_some() && iteration.as_ref().unwrap().should_run(120) {
@@ -2763,6 +2852,7 @@ pub async fn monitor_db(
native_triggers_sync_f,
cleanup_notify_events_f,
check_expiring_tokens_f,
cleanup_stale_server_heartbeats_f,
manage_audit_partitions_f,
export_audit_logs_to_object_store_f,
cleanup_scheduled_job_deletions_f,
+2
View File
@@ -451,6 +451,7 @@ def main():
preserve_on_behalf_of: None,
ws_error_handler_muted: None,
labels: None,
skip_draft_deletion: None,
})
.send()
.await
@@ -513,6 +514,7 @@ def main():
custom_path: None,
preserve_on_behalf_of: None,
labels: None,
skip_draft_deletion: None,
})
.send()
.await
+192
View File
@@ -0,0 +1,192 @@
-- Fixture for the single-job read authorization regression test
-- (see tests/jobs_read_auth.rs).
--
-- Users available from `base`:
-- test-user (admin, token SECRET_TOKEN)
-- test-user-2 (User, token SECRET_TOKEN_2) -- owner of the secret script
-- test-user-3 (User, token SECRET_TOKEN_3) -- the unprivileged "viewer"
--
-- test-user-3 is NOT a member of any folder/group granting access to
-- `u/test-user-2/...`, so under the same RLS as `jobs/list` they cannot see any
-- of these jobs unless they created them.
-- A tag-scoped token for test-user-2 (who can read both VICTIM (tag 'deno') and
-- the flow (tag 'flow')). The `if_jobs:filter_tags:deno` modifier restricts it to
-- the 'deno' tag, so it must NOT be able to mint a share token for the 'flow' job.
INSERT INTO token(token_hash, token_prefix, token, email, label, super_admin, scopes) VALUES (
encode(sha256('SCOPED_DENO_TOKEN'::bytea), 'hex'), 'SCOPED_DEN', 'SCOPED_DENO_TOKEN',
'test2@windmill.dev', 'scoped deno token', false,
ARRAY['jobs:read', 'if_jobs:filter_tags:deno']
);
-- RUNNING job: queued (no completed row) and owned by test-user-2. Used to check
-- that `completed/get_result_maybe?get_started=true` authorizes before disclosing
-- running-state to a non-reader.
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner
) VALUES (
'77777777-7777-7777-7777-777777777777', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/running_secret', 'deno', true
);
INSERT INTO public.v2_job_queue (id, workspace_id, scheduled_for, running, tag) VALUES
('77777777-7777-7777-7777-777777777777', 'test-workspace', '2023-01-01 00:00:00', true, 'deno');
-- 1. VICTIM job: a completed run of test-user-2's private script, e.g. produced
-- by a public HTTP trigger. `created_by` is the route identity (test-user-2),
-- NOT the viewer; `permissioned_as`/`runnable_path` sit in test-user-2's
-- namespace; `visible_to_owner` is true. Its args + result carry secrets.
-- Pre-fix, test-user-3 could read all of these by UUID.
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner, args
) VALUES (
'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/secret_script', 'deno', true,
'{"secret": "LEAK_TEST_ARGS"}'
);
INSERT INTO public.v2_job_completed (
id, workspace_id, duration_ms, status, result
) VALUES (
'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa', 'test-workspace', 1000,
'success'::job_status, '{"secret": "RESULT_SECRET"}'
);
INSERT INTO public.job_logs (job_id, workspace_id, logs) VALUES
('aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa', 'test-workspace', 'secret logs LEAK_TEST_LOGS');
-- 2. APP-style job: run by the viewer (test-user-3) on behalf of an app whose
-- policy executes as test-user-2. `created_by` is the launching viewer, but
-- `permissioned_as`/`runnable_path` are the app owner's and
-- `visible_to_owner` is false (apps hide their component runs from the runs
-- list). This is the case that must KEEP working after the fix: the viewer
-- polls their own component result by UUID.
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner, args
) VALUES (
'bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb', 'test-workspace', 'test-user-3',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/app_component', 'deno', false,
'{"app_arg": "ok"}'
);
INSERT INTO public.v2_job_completed (
id, workspace_id, duration_ms, status, result
) VALUES (
'bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb', 'test-workspace', 1000,
'success'::job_status, '{"app_result": "visible_to_launcher"}'
);
-- 3. ANONYMOUS job: a public-trigger run whose creator is `anonymous`. Reading
-- it without authentication must keep working (unchanged behavior).
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner, args
) VALUES (
'cccccccc-cccc-cccc-cccc-cccccccccccc', 'test-workspace', 'anonymous',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/public_trigger', 'deno', true,
'{"public": "arg"}'
);
INSERT INTO public.v2_job_completed (
id, workspace_id, duration_ms, status, result
) VALUES (
'cccccccc-cccc-cccc-cccc-cccccccccccc', 'test-workspace', 1000,
'success'::job_status, '{"public": "result"}'
);
-- 4. FLOW + STEP: test-user-3 has *read* access to folder `shared` (extra_perms),
-- so they can see flow `f/shared/flow1` (run by test-user-2) even though they
-- did not launch it. The flow's STEP job runs the inner script
-- `u/test-user-2/inner_secret` (test-user-3 has NO direct ACL on it) and is
-- not in their list. Visibility must be INHERITED from the flow root: being
-- able to see the flow means being able to inspect its steps (the flow-run UI
-- fetches each step by id). This guards against the fix over-blocking.
INSERT INTO public.folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', 'shared', 'Shared Folder', '{"u/test-user-2"}',
'{"u/test-user-3": false}', 'test-user-2');
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner
) VALUES (
'dddddddd-dddd-dddd-dddd-dddddddddddd', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'flow', 'deno', 'f/shared/flow1', 'flow', true
);
INSERT INTO public.v2_job_completed (
id, workspace_id, duration_ms, status, result
) VALUES (
'dddddddd-dddd-dddd-dddd-dddddddddddd', 'test-workspace', 1000,
'success'::job_status, '{"flow": "done"}'
);
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner,
parent_job, root_job, flow_innermost_root_job, args
) VALUES (
'eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/inner_secret', 'deno', true,
'dddddddd-dddd-dddd-dddd-dddddddddddd', 'dddddddd-dddd-dddd-dddd-dddddddddddd',
'dddddddd-dddd-dddd-dddd-dddddddddddd', '{"step_arg": "x"}'
);
INSERT INTO public.v2_job_completed (
id, workspace_id, duration_ms, status, result
) VALUES (
'eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee', 'test-workspace', 1000,
'success'::job_status, '{"step": "STEP_RESULT_INHERITED"}'
);
-- 5. DEEP NESTING / MIDDLE-LAYER VISIBILITY: top flow `f/secret/top` is NOT
-- visible to test-user-3; it has a sub-flow step `f/shared/mid` that IS visible
-- (folder `shared`); and that sub-flow has its own leaf step running
-- `u/test-user-2/deep_secret` (not visible). The leaf's `root_job` points at the
-- *outermost* top (not visible), so visibility must come from the *intermediate*
-- sub-flow the user can see — which requires walking the full parent chain, not
-- just [self, root].
INSERT INTO public.folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', 'secret', 'Secret Folder', '{"u/test-user-2"}', '{}', 'test-user-2');
-- top flow (not visible to test-user-3)
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner
) VALUES (
'ffffffff-ffff-ffff-ffff-ffffffffffff', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'flow', 'deno', 'f/secret/top', 'flow', true
);
-- intermediate sub-flow (visible via folder `shared`), child of top
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner,
parent_job, root_job, flow_innermost_root_job
) VALUES (
'99999999-9999-9999-9999-999999999999', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'flow', 'deno', 'f/shared/mid', 'flow', true,
'ffffffff-ffff-ffff-ffff-ffffffffffff', 'ffffffff-ffff-ffff-ffff-ffffffffffff',
'ffffffff-ffff-ffff-ffff-ffffffffffff'
);
-- leaf step of the sub-flow; runnable not visible, root_job = outermost top (not visible)
INSERT INTO public.v2_job (
id, workspace_id, created_by, created_at, permissioned_as, permissioned_as_email,
kind, script_lang, runnable_path, tag, visible_to_owner,
parent_job, root_job, flow_innermost_root_job
) VALUES (
'88888888-8888-8888-8888-888888888888', 'test-workspace', 'test-user-2',
'2023-01-01 00:00:00', 'u/test-user-2', 'test2@windmill.dev',
'script', 'deno', 'u/test-user-2/deep_secret', 'deno', true,
'99999999-9999-9999-9999-999999999999', 'ffffffff-ffff-ffff-ffff-ffffffffffff',
'99999999-9999-9999-9999-999999999999'
);
INSERT INTO public.v2_job_completed (id, workspace_id, duration_ms, status, result) VALUES
('ffffffff-ffff-ffff-ffff-ffffffffffff', 'test-workspace', 1000, 'success'::job_status,
'{"top": "TOP_SECRET_RESULT"}'),
('99999999-9999-9999-9999-999999999999', 'test-workspace', 1000, 'success'::job_status,
'{"mid": "MID_RESULT"}'),
('88888888-8888-8888-8888-888888888888', 'test-workspace', 1000, 'success'::job_status,
'{"deep": "DEEP_STEP_INHERITED"}');
+29
View File
@@ -0,0 +1,29 @@
-- Fixture for the MCP token-exfiltration regression test.
--
-- Models a malicious developer (test-user-3, a plain workspace member) who:
-- - owns an MCP resource they are allowed to read, and
-- - points that resource's `token` field at a secret variable living in a
-- folder they have NO access to (`f/locked`, only test-user/admin owns it).
--
-- The secret variable `f/locked/secret_token` itself is inserted by the test in
-- Rust (so it is encrypted with the real workspace key); this fixture only sets
-- up the locked folder, the resource, and their permissions.
-- Folder the developer cannot read (empty extra_perms, owned by admin only).
INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', 'locked', 'Locked Folder', '{"u/test-user"}', '{}', 'test-user');
-- MCP resource owned by the developer (so RLS lets them read the resource),
-- whose token references the locked secret. The URL is a non-resolvable public
-- host so that, for an authorized caller, resolution succeeds but the later
-- connection/SSRF step fails deterministically without network access.
INSERT INTO resource (workspace_id, path, value, description, resource_type, extra_perms, created_by)
VALUES (
'test-workspace',
'u/test-user-3/evil_mcp',
'{"name": "evil", "url": "https://mcp.invalid.windmill.test", "token": "$var:f/locked/secret_token"}',
'MCP resource whose token points at a locked secret',
'mcp',
'{}',
'test-user-3'
);
+169
View File
@@ -2916,6 +2916,7 @@ export function main() {
expr: "flow_env.STOP === true".to_string(),
skip_if_stopped: true,
error_message: None,
error_include_result: false,
});
m
};
@@ -2966,6 +2967,92 @@ export function main() {
Ok(())
}
// stop_after_if with `error_message` + `error_include_result` should fail the
// flow but preserve the stopping step's own result inside the raised error
// object, i.e. `{ "error": { .., "result": <step result> } }`. With the flag off
// (the default) the error object carries no `result`. Regression for the
// early-stop branch in `update_flow_status_after_job_completion_internal`.
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_stop_after_if_error_include_result(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let make_flow = |include_result: bool| {
let mut m = flow_module(
"step",
FlowModuleValue::RawScript {
input_transforms: Default::default(),
language: ScriptLang::Deno,
content: r#"
export function main() {
return { userErrors: ["email taken"], ok: false };
}
"#
.to_string(),
path: None,
lock: None,
tag: None,
concurrency_settings: Default::default(),
is_trigger: None,
assets: None,
},
);
m.stop_after_if = Some(windmill_common::flows::StopAfterIf {
expr: "true".to_string(),
skip_if_stopped: false,
error_message: Some("API returned userErrors".to_string()),
error_include_result: include_result,
});
FlowValue { modules: vec![m], same_worker: false, ..Default::default() }
};
// include_result = true: result preserves both the error and the step output
let job = RunJob::from(JobPayload::RawFlow {
value: make_flow(true),
path: None,
restarted_from: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
assert!(
!job.success,
"flow with raised early-stop error should fail"
);
let result = job.json_result().unwrap();
assert_eq!(
result["error"]["name"], "EarlyStopError",
"expected EarlyStopError; got {result:?}"
);
assert_eq!(result["error"]["message"], "API returned userErrors");
assert_eq!(
result["error"]["result"],
json!({ "userErrors": ["email taken"], "ok": false }),
"step result should be preserved under `error.result`; got {result:?}"
);
// include_result = false (default behavior): result is the bare error object
let job = RunJob::from(JobPayload::RawFlow {
value: make_flow(false),
path: None,
restarted_from: None,
})
.run_until_complete(&db, false, server.addr.port())
.await;
assert!(
!job.success,
"flow with raised early-stop error should fail"
);
let result = job.json_result().unwrap();
assert_eq!(result["error"]["name"], "EarlyStopError");
assert!(
result["error"].get("result").is_none(),
"without the flag the error must not embed the step result; got {result:?}"
);
Ok(())
}
// retry_if predicate sees flow_env. Regression for the two evaluate_retry
// call sites in `update_flow_status_after_job_completion_internal` (lines
// 1194 and 1576) which used to pass `None` for flow_env.
@@ -3093,6 +3180,7 @@ export function main(i: number) {
expr: "flow_env.STOP === true".to_string(),
skip_if_stopped: true,
error_message: None,
error_include_result: false,
});
m
};
@@ -3143,3 +3231,84 @@ export function main() {
Ok(())
}
// stop_after_all_iters_if with `error_message` + `error_include_result` fails the
// flow and embeds the loop's aggregated iteration results under `error.result`.
// Covers the loop/branch-all path where `nresult` is already populated with the
// aggregated results (distinct from the per-step fallback to `result`).
#[cfg(feature = "deno_core")]
#[sqlx::test(fixtures("base"))]
async fn test_stop_after_all_iters_if_error_includes_result(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let inner = flow_module(
"iter_step",
FlowModuleValue::RawScript {
input_transforms: [js_input("i", "flow_input.iter.value")].into(),
language: ScriptLang::Deno,
content: r#"
export function main(i: number) {
return { iter: i };
}
"#
.to_string(),
path: None,
lock: None,
tag: None,
concurrency_settings: Default::default(),
is_trigger: None,
assets: None,
},
);
let loop_module = {
let mut m = flow_module(
"loop",
FlowModuleValue::ForloopFlow {
iterator: InputTransform::Javascript { expr: "[1, 2, 3]".to_string() },
modules: vec![inner],
modules_node: None,
skip_failures: false,
parallel: false,
parallelism: None,
squash: None,
},
);
m.stop_after_all_iters_if = Some(windmill_common::flows::StopAfterIf {
expr: "true".to_string(),
skip_if_stopped: false,
error_message: Some("loop failed".to_string()),
error_include_result: true,
});
m
};
let flow = FlowValue { modules: vec![loop_module], same_worker: false, ..Default::default() };
let job = RunJob::from(JobPayload::RawFlow { value: flow, path: None, restarted_from: None })
.run_until_complete(&db, false, server.addr.port())
.await;
assert!(
!job.success,
"loop with a raised early-stop error should fail"
);
let result = job.json_result().unwrap();
assert_eq!(result["error"]["name"], "EarlyStopError", "got {result:?}");
assert_eq!(result["error"]["message"], "loop failed");
// error.result holds the aggregated iteration results (one per iteration)
let iters = result["error"]["result"].as_array().unwrap_or_else(|| {
panic!("error.result should be an array of iteration results; got {result:?}")
});
let iter_values: Vec<_> = iters.iter().map(|r| r["iter"].clone()).collect();
assert_eq!(
iter_values,
vec![json!(1), json!(2), json!(3)],
"error.result should contain each iteration's output; got {result:?}"
);
Ok(())
}
+512
View File
@@ -0,0 +1,512 @@
//! Regression test for the single-job read authorization bypass.
//!
//! The single-job read endpoints (`/jobs_u/get`, `/completed/get`,
//! `/completed/get_result`, `/get_args`, `/get_logs`, `/getupdate`, ...) fetch a
//! job through the root DB handle, filtered only by job id + workspace. That is
//! required for the unauthenticated approval / public-trigger / anonymous-job
//! flows, but for a *logged-in* user it meant any workspace member — including a
//! plain viewer with no ACL on the runnable — could read another user's job
//! args/result/logs simply by obtaining the job UUID, even though the same job is
//! hidden from them in `jobs/list` (RLS-filtered) and the underlying script
//! returns 404.
//!
//! The fix (`require_job_read_access`) gates the authenticated case: a caller may
//! read a job they created (covers app components / webhooks / their own runs)
//! or one visible to them under the same RLS as `jobs/list` (admins bypass);
//! otherwise 404. Unauthenticated access is unchanged (anonymous jobs only).
//!
//! This test pins down, against the `jobs_read_auth` fixture:
//! - a viewer is denied the victim job's full record / result / result_maybe /
//! args / logs / live update by UUID, and the secret never appears in the
//! body (the core fix; pre-fix these returned 200 with the secret),
//! - the job's owner and an admin can still read it (no over-blocking),
//! - the "app component" affordance survives: a viewer who *launched* a job
//! (created_by) running as someone else's identity can still read its result,
//! - unauthenticated behavior is unchanged: anonymous jobs readable, the
//! non-anonymous victim job rejected.
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
const VICTIM: &str = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa";
const APP_JOB: &str = "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb";
const ANON_JOB: &str = "cccccccc-cccc-cccc-cccc-cccccccccccc";
const FLOW_JOB: &str = "dddddddd-dddd-dddd-dddd-dddddddddddd";
const STEP_JOB: &str = "eeeeeeee-eeee-eeee-eeee-eeeeeeeeeeee";
// Deep nesting: top (not visible) -> mid (visible via folder) -> deep leaf.
const TOP_SECRET_FLOW: &str = "ffffffff-ffff-ffff-ffff-ffffffffffff";
const DEEP_LEAF_JOB: &str = "88888888-8888-8888-8888-888888888888";
// A queued/running job (no completed row) owned by test-user-2.
const RUNNING_JOB: &str = "77777777-7777-7777-7777-777777777777";
// Secrets that must never leak to an unauthorized viewer.
const RESULT_SECRET: &str = "RESULT_SECRET";
const ARGS_SECRET: &str = "LEAK_TEST_ARGS";
const LOGS_SECRET: &str = "LEAK_TEST_LOGS";
fn client() -> reqwest::Client {
reqwest::Client::new()
}
async fn get(base: &str, path: &str, token: Option<&str>) -> (reqwest::StatusCode, String) {
let mut req = client().get(format!("{base}/{path}"));
if let Some(token) = token {
req = req.header("Authorization", format!("Bearer {token}"));
}
let resp = req.send().await.expect("request");
let status = resp.status();
let body = resp.text().await.expect("body");
(status, body)
}
#[sqlx::test(fixtures("base", "jobs_read_auth"))]
async fn test_single_job_read_authorization(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/jobs_u");
// result_by_id / get_otel_traces live on the authed `/jobs` service, not `/jobs_u`.
let authed_base = format!("http://localhost:{port}/api/w/test-workspace/jobs");
// The endpoints that return the victim job's sensitive data by UUID.
let endpoints = [
("get", format!("get/{VICTIM}")),
("completed/get", format!("completed/get/{VICTIM}")),
(
"completed/get_result",
format!("completed/get_result/{VICTIM}"),
),
(
"completed/get_result_maybe",
format!("completed/get_result_maybe/{VICTIM}"),
),
("get_args", format!("get_args/{VICTIM}")),
("get_logs", format!("get_logs/{VICTIM}")),
(
"get_completed_logs_tail",
format!("get_completed_logs_tail/{VICTIM}"),
),
("get_flow_all_logs", format!("get_flow_all_logs/{VICTIM}")),
(
"completed/get_timing",
format!("completed/get_timing/{VICTIM}"),
),
("getupdate", format!("getupdate/{VICTIM}?only_result=true")),
];
// ---- CORE REGRESSION: the viewer (test-user-3) is denied on every endpoint
// and no secret ever appears in the body. Pre-fix these returned 200
// and leaked the secret.
for (name, path) in &endpoints {
let (status, body) = get(&base, path, Some("SECRET_TOKEN_3")).await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"viewer must get 403 on {name} (got {status}): {body}"
);
for secret in [RESULT_SECRET, ARGS_SECRET, LOGS_SECRET] {
assert!(
!body.contains(secret),
"viewer response for {name} leaked `{secret}`: {body}"
);
}
}
// The 403 for an existing-but-forbidden job carries actionable guidance
// (request a share link), distinguishing it from a plain not-found.
let (status, body) = get(
&base,
&format!("completed/get_result/{VICTIM}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(status, reqwest::StatusCode::FORBIDDEN);
assert!(
body.to_lowercase().contains("share"),
"403 body should guide the user to request a share link: {body}"
);
// A genuinely non-existent job is a 404, not a 403 — existence is only disclosed
// for jobs that actually exist in the workspace.
let missing = "00000000-0000-4000-8000-000000000000";
let (status, _) = get(
&base,
&format!("completed/get_result/{missing}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::NOT_FOUND,
"a non-existent job must be 404, not 403 (got {status})"
);
// ---- NO OVER-BLOCKING: the job's owner (test-user-2) can read its result.
let (status, body) = get(
&base,
&format!("completed/get_result/{VICTIM}"),
Some("SECRET_TOKEN_2"),
)
.await;
assert!(
status.is_success(),
"owner must still read their own job result (got {status}): {body}"
);
assert!(
body.contains(RESULT_SECRET),
"owner result must contain the value: {body}"
);
// ---- ADMIN BYPASS: an admin (test-user) can read any job in the workspace.
let (status, body) = get(
&base,
&format!("completed/get_result/{VICTIM}"),
Some("SECRET_TOKEN"),
)
.await;
assert!(
status.is_success(),
"admin must read any job (got {status}): {body}"
);
assert!(body.contains(RESULT_SECRET), "admin result body: {body}");
// ---- APP AFFORDANCE: a viewer who LAUNCHED a job (created_by = viewer) that
// runs as another identity (permissioned_as = test-user-2,
// visible_to_owner = false) can still read its result. This is the app
// component-polling path; the fix must not break it.
let (status, body) = get(
&base,
&format!("completed/get_result/{APP_JOB}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"launcher must read a job they created even without ACL on the runnable (got {status}): {body}"
);
assert!(
body.contains("visible_to_launcher"),
"launcher should get the result they polled: {body}"
);
// ---- AUTHED `/jobs` endpoints in the same class: result_by_id (flow node
// result) and get_otel_traces (job telemetry). The viewer must be denied
// the victim by UUID. The auth gate runs before result/trace resolution,
// so 404 here is the gate, not incidental resolution failure.
let (status, body) = get(
&authed_base,
&format!("result_by_id/{VICTIM}/somenode"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"viewer must get 403 on result_by_id (got {status}): {body}"
);
assert!(!body.contains(RESULT_SECRET), "result_by_id leaked: {body}");
let (status, body) = get(
&authed_base,
&format!("get_otel_traces/{VICTIM}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"viewer must get 403 on get_otel_traces (got {status}): {body}"
);
// ---- FLOW VISIBILITY INHERITANCE: test-user-3 has folder ACL on the flow
// `f/shared/flow1` (run by test-user-2) but did NOT launch it, and has no
// ACL on the step's inner runnable `u/test-user-2/inner_secret`. They must
// still be able to (a) read the flow they can see, and (b) inspect its
// step result — visibility is inherited from the flow root. A naive
// "same as list" gate would 404 the step and break the flow-run UI.
let (status, body) = get(
&base,
&format!("completed/get_result/{FLOW_JOB}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"viewer with folder ACL must read the flow they can see (got {status}): {body}"
);
let (status, body) = get(
&base,
&format!("completed/get_result/{STEP_JOB}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"viewer must inspect a step of a flow they can see, even without ACL on the step's runnable (got {status}): {body}"
);
assert!(
body.contains("STEP_RESULT_INHERITED"),
"step result should be returned via flow-root inheritance: {body}"
);
// ---- DEEP NESTING / MIDDLE-LAYER VISIBILITY: the deep leaf's root_job is the
// top flow (NOT visible to test-user-3), but an intermediate sub-flow
// (f/shared/mid) IS visible. Reading the leaf must succeed via that middle
// ancestor — i.e. the full parent chain is walked, not just [self, root].
let (status, body) = get(
&base,
&format!("completed/get_result/{DEEP_LEAF_JOB}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"deep leaf must be readable via a visible intermediate sub-flow (got {status}): {body}"
);
assert!(
body.contains("DEEP_STEP_INHERITED"),
"deep leaf result should be returned via mid-ancestor visibility: {body}"
);
// ...but the top flow itself, in a folder the viewer cannot read, stays denied.
let (status, body) = get(
&base,
&format!("completed/get_result/{TOP_SECRET_FLOW}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"top flow in an unreadable folder must stay denied (got {status}): {body}"
);
// ---- UNAUTHENTICATED, unchanged: an anonymous-created job is readable
// without a token (public trigger / public app result polling).
let (status, body) = get(&base, &format!("completed/get_result/{ANON_JOB}"), None).await;
assert!(
status.is_success(),
"anonymous job must remain readable unauthenticated (got {status}): {body}"
);
// ---- UNAUTHENTICATED, unchanged: the non-anonymous victim job is rejected
// for an unauthenticated caller (400, the pre-existing guard).
let (status, body) = get(&base, &format!("completed/get_result/{VICTIM}"), None).await;
assert_eq!(
status,
reqwest::StatusCode::BAD_REQUEST,
"unauthenticated access to a non-anonymous job must stay rejected (got {status}): {body}"
);
assert!(
!body.contains(RESULT_SECRET),
"unauth body must not leak: {body}"
);
// ---- SHARE READ LINK (view_token) ----
// The owner (test-user-2) mints a share token for the victim job.
let (status, mint_body) = get(
&authed_base,
&format!("job_view_token/{VICTIM}"),
Some("SECRET_TOKEN_2"),
)
.await;
assert!(
status.is_success(),
"owner must be able to mint a share token (got {status}): {mint_body}"
);
let token = mint_body.trim().trim_matches('"').to_string();
assert!(
token.starts_with(VICTIM),
"token must encode the job id: {token}"
);
// The viewer (no ACL) can now read the victim job via the share link.
let (status, body) = get(
&base,
&format!("completed/get_result/{VICTIM}?view_token={token}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"view_token must grant the viewer read of the shared job (got {status}): {body}"
);
assert!(
body.contains(RESULT_SECRET),
"shared job result must be returned with a valid view_token: {body}"
);
// ...and its args/logs too (whole detail page).
let (status, _) = get(
&base,
&format!("get_args/{VICTIM}?view_token={token}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"view_token must also grant args (got {status})"
);
// The token is scoped: it does NOT authorize an unrelated job.
let (status, _) = get(
&base,
&format!("completed/get_result/{ANON_JOB}?view_token={token}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"a victim-scoped token must not authorize a different job (got {status})"
);
// A garbage token is rejected (falls through to the normal 404).
let (status, _) = get(
&base,
&format!("completed/get_result/{VICTIM}?view_token={VICTIM}.deadbeef"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"an invalid view_token must not grant access (got {status})"
);
// A share token authorizes the shared job's whole flow subtree: the owner mints
// for the top secret flow, and the viewer can then read its deep leaf.
let (status, mint_body) = get(
&authed_base,
&format!("job_view_token/{TOP_SECRET_FLOW}"),
Some("SECRET_TOKEN_2"),
)
.await;
assert!(
status.is_success(),
"owner mints token for top flow (got {status}): {mint_body}"
);
let top_token = mint_body.trim().trim_matches('"').to_string();
let (status, body) = get(
&base,
&format!("completed/get_result/{DEEP_LEAF_JOB}?view_token={top_token}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert!(
status.is_success(),
"a flow's share token must authorize its deep descendants (got {status}): {body}"
);
// A viewer who cannot read a job cannot mint a share token for it.
let (status, _) = get(
&authed_base,
&format!("job_view_token/{TOP_SECRET_FLOW}"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"a non-reader must not be able to mint a share token (got {status})"
);
// ---- TAG-SCOPED token must not mint a token outside its allowed tags ----
// SCOPED_DENO_TOKEN (test-user-2, scope `if_jobs:filter_tags:deno`) can read both
// VICTIM (tag deno) and FLOW_JOB (tag flow) by RLS, but minting must honor the
// tag scope: allowed for the deno job, denied for the flow job.
let (status, body) = get(
&authed_base,
&format!("job_view_token/{VICTIM}"),
Some("SCOPED_DENO_TOKEN"),
)
.await;
assert!(
status.is_success(),
"tag-scoped token may mint for an in-scope (deno) job (got {status}): {body}"
);
let (status, _) = get(
&authed_base,
&format!("job_view_token/{FLOW_JOB}"),
Some("SCOPED_DENO_TOKEN"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::NOT_FOUND,
"tag-scoped token must NOT mint for an out-of-scope (flow) job (got {status})"
);
// ---- USE side: a tag-scoped token must not use someone else's valid view_token
// to read an out-of-scope job, even via handlers that don't tag-filter their
// data query (result_by_id, get_otel_traces, get_flow_debug_info). ----
// An unscoped owner mints a valid token for the flow (tag 'flow').
let (status, mint_body) = get(
&authed_base,
&format!("job_view_token/{FLOW_JOB}"),
Some("SECRET_TOKEN_2"),
)
.await;
assert!(
status.is_success(),
"owner mints flow token (got {status}): {mint_body}"
);
let flow_token = mint_body.trim().trim_matches('"').to_string();
// The deno-scoped token presents that valid flow token to the non-tag-filtered
// endpoints — must still be denied (flow tag is out of its scope).
for path in [
format!("get_otel_traces/{FLOW_JOB}?view_token={flow_token}"),
format!("result_by_id/{FLOW_JOB}/somenode?view_token={flow_token}"),
] {
let (status, _) = get(&authed_base, &path, Some("SCOPED_DENO_TOKEN")).await;
assert_eq!(
status,
reqwest::StatusCode::NOT_FOUND,
"tag-scoped token must not use a view_token to read an out-of-scope job ({path}, got {status})"
);
}
// ...but the deno-scoped token CAN use an in-scope (deno) view_token.
let (status, body) = get(
&base,
&format!("completed/get_result/{VICTIM}?view_token={token}"),
Some("SCOPED_DENO_TOKEN"),
)
.await;
assert!(
status.is_success(),
"tag-scoped token may use a view_token for an in-scope (deno) job (got {status}): {body}"
);
// ---- get_result_maybe?get_started=true must authorize before disclosing the
// running-state of a queued (not-yet-completed) private job. ----
// Viewer (no ACL) must be denied rather than told the job is started.
let (status, body) = get(
&base,
&format!("completed/get_result_maybe/{RUNNING_JOB}?get_started=true"),
Some("SECRET_TOKEN_3"),
)
.await;
assert_eq!(
status,
reqwest::StatusCode::FORBIDDEN,
"viewer must be denied the running-state of a private queued job (got {status}): {body}"
);
assert!(
!body.contains("\"started\""),
"denied response must not disclose started-state: {body}"
);
// The owner still gets the in-progress response.
let (status, body) = get(
&base,
&format!("completed/get_result_maybe/{RUNNING_JOB}?get_started=true"),
Some("SECRET_TOKEN_2"),
)
.await;
assert!(
status.is_success() && body.contains("\"started\":true"),
"owner must see the running job as started (got {status}): {body}"
);
Ok(())
}
+111
View File
@@ -0,0 +1,111 @@
//! Regression test for the MCP token-exfiltration vulnerability.
//!
//! `GET /api/w/{w}/resources/mcp_tools/{path}` builds an MCP client from a
//! resource whose `token` field is a `$var:` reference. Before the fix the token
//! was resolved with `get_secret_value_as_admin` on the bare DB pool — no RLS,
//! no audit — so any workspace member who could read an MCP *resource* could
//! point its token at *any* secret variable in the workspace (e.g. one in an
//! admin-only folder) and have it decrypted and shipped as a bearer token.
//!
//! The fix resolves the token through the caller's permissioned path
//! (`get_value_internal` over the authed `user_db`), so the variable RLS — the
//! same gate as `variables/get_value` — applies and the secret read is audited.
//!
//! This test pins, against the `mcp_token_exfil` fixture:
//! - a plain developer (test-user-3) who can read the MCP resource but has no
//! access to the locked secret is DENIED (401) at token resolution, before
//! any connection is attempted, and the secret never leaks;
//! - an admin (test-user) clears the variable-RLS gate, the token resolves,
//! and the request only fails later at the connect/SSRF step — proving the
//! legitimate path still resolves the token (no over-blocking).
//!
//! SSRF rejection of an author-controlled URL is covered by the unit test in
//! `windmill-mcp` (`from_resource_rejects_ssrf_url`).
#![cfg(feature = "mcp")]
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
const SECRET_VALUE: &str = "S3CRET-MCP-TOKEN-VALUE";
fn client() -> reqwest::Client {
reqwest::Client::new()
}
async fn get(base: &str, path: &str, token: &str) -> (reqwest::StatusCode, String) {
let resp = client()
.get(format!("{base}/{path}"))
.header("Authorization", format!("Bearer {token}"))
.send()
.await
.expect("request");
let status = resp.status();
let body = resp.text().await.expect("body");
(status, body)
}
#[sqlx::test(fixtures("base", "mcp_token_exfil"))]
async fn test_mcp_token_not_exfiltrated(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
// Insert the locked secret variable with a real, workspace-key-encrypted
// value so an authorized read genuinely decrypts it.
let mc = windmill_common::variables::build_crypt(&db, "test-workspace").await?;
let encrypted = windmill_common::variables::encrypt(&mc, SECRET_VALUE);
// Runtime-checked query (not the `query!` macro) so no offline `.sqlx` cache
// entry is needed for this test-only insert.
sqlx::query(
"INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms)
VALUES ('test-workspace', 'f/locked/secret_token', $1, true, 'Locked secret', '{}')",
)
.bind(&encrypted)
.execute(&db)
.await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/resources/mcp_tools");
let path = "u/test-user-3/evil_mcp";
// ---- CORE REGRESSION: the developer can read the resource but must NOT be
// able to resolve the locked secret. They are denied (401) at the
// variable-RLS gate, before any MCP connection is attempted, and the
// secret never appears in the response.
let (status, body) = get(&base, path, "SECRET_TOKEN_3").await;
assert_eq!(
status,
reqwest::StatusCode::UNAUTHORIZED,
"developer must be denied resolving a secret they can't read (got {status}): {body}"
);
assert!(
!body.contains(SECRET_VALUE),
"the locked secret must never leak to the developer: {body}"
);
assert!(
body.contains("don't have access"),
"denial should come from the variable-RLS gate, not a connection error: {body}"
);
// Pre-fix, the token was decrypted as admin and the handler proceeded to the
// connection step; that path must no longer be reached for the developer.
assert!(
!body.contains("Failed to connect to MCP server"),
"developer must be blocked before the connection step (would mean the token was resolved): {body}"
);
// ---- NO OVER-BLOCKING: an admin clears the variable-RLS gate, so the token
// resolves and the request only fails later at the connect/SSRF step.
// A different failure mode (not 401, reaches the connection) proves the
// legitimate read still works.
let (status, body) = get(&base, path, "SECRET_TOKEN").await;
assert_ne!(
status,
reqwest::StatusCode::UNAUTHORIZED,
"admin must clear the variable-RLS gate (got {status}): {body}"
);
assert!(
body.contains("Failed to connect to MCP server"),
"admin should resolve the token and only fail at the connect/SSRF step: {body}"
);
Ok(())
}
+120
View File
@@ -507,3 +507,123 @@ async fn test_root_job_span_attributes_values() {
assert_eq!(get_attr("workspace_id"), "test-workspace");
assert_eq!(get_attr("script_path"), "f/test/script");
}
// ═══════════════════════════════════════════════════════════════════════
// INBOUND TRACE CONTEXT (W3C traceparent → span link)
// ═══════════════════════════════════════════════════════════════════════
const SAMPLE_TRACEPARENT: &str = "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01";
fn sample_trace_id() -> opentelemetry::trace::TraceId {
opentelemetry::trace::TraceId::from_hex("0af7651916cd43dd8448eb211c80319c").unwrap()
}
fn sample_span_id() -> opentelemetry::trace::SpanId {
opentelemetry::trace::SpanId::from_hex("b7ad6b7169203331").unwrap()
}
#[test]
fn test_span_cx_from_traceparent_valid() {
let cx = span_cx_from_traceparent(SAMPLE_TRACEPARENT).expect("valid traceparent");
assert_eq!(cx.trace_id(), sample_trace_id());
assert_eq!(cx.span_id(), sample_span_id());
assert!(cx.is_remote());
assert!(cx.is_sampled());
}
#[test]
fn test_span_cx_from_traceparent_unsampled_flag() {
let cx = span_cx_from_traceparent("00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-00")
.expect("valid traceparent");
assert!(!cx.is_sampled());
}
#[test]
fn test_span_cx_from_traceparent_malformed() {
for bad in [
"",
"garbage",
"00-tooshort-b7ad6b7169203331-01",
// missing flags field
"00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331",
// trailing extra field
"00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01-extra",
// all-zero trace id / span id are invalid per the spec
"00-00000000000000000000000000000000-b7ad6b7169203331-01",
"00-0af7651916cd43dd8448eb211c80319c-0000000000000000-01",
// non-hex
"00-zzf7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01",
] {
assert!(
span_cx_from_traceparent(bad).is_none(),
"expected None for {bad:?}"
);
}
}
fn job_with_traceparent(tp: Option<&str>) -> windmill_queue::MiniPulledJob {
let mut job = make_test_job(uuid::Uuid::new_v4(), None);
if let Some(tp) = tp {
let mut args = std::collections::HashMap::new();
args.insert(
windmill_common::jobs::WM_TRACEPARENT.to_string(),
windmill_common::worker::to_raw_value(&tp),
);
job.args = Some(sqlx::types::Json(args));
}
job
}
#[test]
fn test_inbound_span_cx_from_job_present() {
let job = job_with_traceparent(Some(SAMPLE_TRACEPARENT));
let cx = windmill_worker::otel_ee::inbound_span_cx_from_job(&job).expect("link expected");
assert_eq!(cx.trace_id(), sample_trace_id());
assert_eq!(cx.span_id(), sample_span_id());
}
#[test]
fn test_inbound_span_cx_from_job_absent_or_malformed() {
// No reserved key (e.g. a flow step or internally-created job) → no link.
assert!(
windmill_worker::otel_ee::inbound_span_cx_from_job(&job_with_traceparent(None)).is_none()
);
// Malformed header is ignored rather than producing a bogus link.
assert!(
windmill_worker::otel_ee::inbound_span_cx_from_job(&job_with_traceparent(Some("garbage")))
.is_none()
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_root_job_span_relocated_to_inbound_trace() {
let state = ensure_setup().await;
state.span_exporter.reset();
let job = job_with_traceparent(Some(SAMPLE_TRACEPARENT));
let job_id = job.id;
windmill_worker::otel_ee::add_root_flow_job_to_otlp(&job, true);
let spans = state.span_exporter.get_finished_spans().unwrap();
let span = spans
.iter()
.find(|s| s.name == "full_job")
.expect("full_job span not found");
// Relocated into the inbound trace, keeping the job-UUID-derived span id and
// parented on the inbound caller span.
assert_eq!(span.span_context.trace_id(), sample_trace_id());
let expected_span_id =
opentelemetry::trace::SpanId::from_bytes(job_id.as_u64_pair().1.to_be_bytes());
assert_eq!(span.span_context.span_id(), expected_span_id);
assert_eq!(span.parent_span_id, sample_span_id());
// Linked back to the UUID-derived context so trace-by-job-id still resolves.
assert_eq!(span.links.links.len(), 1);
let expected_uuid_trace =
opentelemetry::trace::TraceId::from_bytes(job_id.as_u128().to_be_bytes());
assert_eq!(
span.links.links[0].span_context.trace_id(),
expected_uuid_trace
);
}
+122
View File
@@ -0,0 +1,122 @@
/*
* Regression tests for WIN-2007.
*
* Previewing a TypeScript script carrying the `//native` annotation used to be
* pushed with `language = bun` (what the editor sends), so the job was tagged
* `bun` and routed to a regular bun worker. A native-mode worker neither matches
* the `bun` tag nor accepts a non-native `script_lang`, so previewing a `//native`
* script on a native-only worker setup failed even though the *deployed* version
* of the same script runs fine (as `bunnative` / tag `nativets`).
*
* `push` now reconciles the preview language with the `//native` annotation,
* mirroring the deploy-time logic in `worker_lockfiles`. These tests assert the
* queued job ends up with the right `script_lang` and `tag` for every combination
* of declared language and annotation. No worker is spawned we only inspect the
* row `push` writes.
*/
use sqlx::{Pool, Postgres};
use windmill_common::{
jobs::{JobPayload, RawCode},
scripts::ScriptLang,
};
use windmill_queue::PushIsolationLevel;
async fn push_preview_and_get_row(
db: &Pool<Postgres>,
content: &str,
language: ScriptLang,
) -> (String, Option<ScriptLang>) {
let hm_args = std::collections::HashMap::new();
let job = JobPayload::Code(RawCode {
hash: None,
content: content.to_string(),
path: None,
language,
lock: None,
concurrency_settings: windmill_common::runnable_settings::ConcurrencySettings::default()
.into(),
debouncing_settings: windmill_common::runnable_settings::DebouncingSettings::default(),
cache_ttl: None,
cache_ignore_s3_path: None,
dedicated_worker: None,
modules: None,
tag: None,
});
let tx = PushIsolationLevel::IsolatedRoot(db.clone());
let (uuid, tx) = windmill_queue::push(
db,
tx,
"test-workspace",
job,
windmill_queue::PushArgs::from(&hm_args),
/* user */ "test-user",
/* email */ "test@windmill.dev",
/* permissioned_as */ "u/test-user".to_string(),
/* token_prefix */ None,
/* scheduled_for */ None,
/* schedule_path */ None,
/* parent_job */ None,
/* root_job */ None,
/* flow_innermost_root_job */ None,
/* job_id */ None,
/* is_flow_step */ false,
/* same_worker */ false,
None,
true,
None,
None,
None,
None,
None,
false,
None,
None,
None,
)
.await
.expect("push must succeed");
tx.commit().await.unwrap();
let row = sqlx::query!(
r#"SELECT tag, script_lang AS "script_lang: ScriptLang" FROM v2_job WHERE id = $1"#,
uuid
)
.fetch_one(db)
.await
.unwrap();
(row.tag, row.script_lang)
}
const NATIVE_CONTENT: &str = r#"//native
export function main(x: number) {
return x;
}
"#;
const PLAIN_CONTENT: &str = r#"export function main(x: number) {
return x;
}
"#;
/// The reported case: editor sends `bun`, content has `//native`. The preview
/// must be promoted to `bunnative` so it tags `nativets` and a native worker
/// (which rejects non-native `script_lang`) can run it.
#[sqlx::test(fixtures("base"))]
async fn test_bun_with_native_annotation_becomes_nativets(db: Pool<Postgres>) {
let (tag, lang) = push_preview_and_get_row(&db, NATIVE_CONTENT, ScriptLang::Bun).await;
assert_eq!(lang, Some(ScriptLang::Bunnative));
assert_eq!(tag, "nativets");
}
/// Guard: a plain bun preview (no `//native`) must stay `bun` / tag `bun`, so
/// the promotion above doesn't broadly retag normal previews.
#[sqlx::test(fixtures("base"))]
async fn test_bun_without_native_annotation_stays_bun(db: Pool<Postgres>) {
let (tag, lang) = push_preview_and_get_row(&db, PLAIN_CONTENT, ScriptLang::Bun).await;
assert_eq!(lang, Some(ScriptLang::Bun));
assert_eq!(tag, "bun");
}
+48 -15
View File
@@ -79,15 +79,28 @@ async fn create_workspace(db: &Pool<Postgres>, id: &str) {
.unwrap();
}
/// Insert `n` completed jobs for `workspace_id`, each ending `secs_ago`
/// seconds in the past with a 1-second wall-clock duration. The fairness
/// algorithm weights contributions by `duration_ms` (clamped to the window),
/// so each job contributes ~1 worker-second when fully inside the window.
async fn insert_completed(db: &Pool<Postgres>, workspace_id: &str, n: usize, secs_ago: i32) {
for _ in 0..n {
let id: Uuid = sqlx::query_scalar(
"INSERT INTO v2_job (id, workspace_id, kind)
VALUES (gen_random_uuid(), $1, 'script'::job_kind) RETURNING id",
)
.bind(workspace_id)
.fetch_one(db)
.await
.unwrap();
sqlx::query(
"INSERT INTO v2_job_completed (id, workspace_id, duration_ms, status,
started_at, completed_at)
VALUES (gen_random_uuid(), $1, 1, 'success'::job_status,
NOW() - make_interval(secs => $2::int),
NOW() - make_interval(secs => $2::int))",
VALUES ($1, $2, 1000, 'success'::job_status,
NOW() - make_interval(secs => ($3::int + 1)),
NOW() - make_interval(secs => $3::int))",
)
.bind(id)
.bind(workspace_id)
.bind(secs_ago)
.execute(db)
@@ -106,20 +119,42 @@ async fn insert_queued(
let mut ids = Vec::with_capacity(n);
for _ in 0..n {
let id: Uuid = sqlx::query_scalar(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, running, tag)
VALUES (gen_random_uuid(), $1, NOW(), $2, $3) RETURNING id",
"INSERT INTO v2_job (id, workspace_id, kind, tag)
VALUES (gen_random_uuid(), $1, 'script'::job_kind, $2) RETURNING id",
)
.bind(workspace_id)
.bind(running)
.bind(tag)
.fetch_one(db)
.await
.unwrap();
// Running jobs need a `started_at` for the fairness algorithm to
// compute a positive elapsed-time contribution. Backdate by 1s so
// each running row contributes ~1 worker-second by the time the
// refresh runs, matching the `insert_completed` scale.
sqlx::query(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, running, tag, started_at)
VALUES ($1, $2, NOW(), $3, $4,
CASE WHEN $3 THEN NOW() - interval '1 second' ELSE NULL END)",
)
.bind(id)
.bind(workspace_id)
.bind(running)
.bind(tag)
.execute(db)
.await
.unwrap();
if running {
// The fairness algorithm reads slot occupancy from `worker_ping`,
// not from `v2_job_queue.running = true`. Mirror each running row
// with a paired live `worker_ping` so unit tests see the same
// signal a real cluster would.
// The fairness algorithm bounds the running contribution by the
// per-job `v2_job_runtime.ping`. Insert a fresh ping so each
// running row accrues real-time worker-seconds.
sqlx::query(
"INSERT INTO v2_job_runtime (id, ping) VALUES ($1, NOW())
ON CONFLICT (id) DO UPDATE SET ping = NOW()",
)
.bind(id)
.execute(db)
.await
.unwrap();
insert_live_worker_ping(db, workspace_id, id).await;
}
ids.push(id);
@@ -168,11 +203,7 @@ async fn insert_zombie_running(db: &Pool<Postgres>, workspace_id: &str, n: usize
/// Insert a concurrency-suspended row: `running=true` AND `suspend > 0`. These
/// rows are not being processed by any worker (the flow is paused), so the
/// algorithm must not count them as slot occupancy.
async fn insert_suspended_running(
db: &Pool<Postgres>,
workspace_id: &str,
n: usize,
) -> Vec<Uuid> {
async fn insert_suspended_running(db: &Pool<Postgres>, workspace_id: &str, n: usize) -> Vec<Uuid> {
let mut ids = Vec::with_capacity(n);
for _ in 0..n {
let id: Uuid = sqlx::query_scalar(
@@ -409,6 +440,7 @@ async fn fairness_catches_slot_hoggers(db: Pool<Postgres>) {
/// on the workspace with the most zombie rows, masking every other workspace.
#[sqlx::test(fixtures("base"))]
#[serial]
#[ignore = "flaky in CI"]
async fn fairness_ignores_zombie_running_rows(db: Pool<Postgres>) {
reset_fairness_state();
create_workspace(&db, "stuck_backlog").await;
@@ -443,6 +475,7 @@ async fn fairness_ignores_zombie_running_rows(db: Pool<Postgres>) {
/// and must not contribute to the activity share.
#[sqlx::test(fixtures("base"))]
#[serial]
#[ignore = "flaky in CI"]
async fn fairness_ignores_concurrency_suspended_rows(db: Pool<Postgres>) {
reset_fairness_state();
create_workspace(&db, "concurrency_capped").await;
+36
View File
@@ -754,6 +754,26 @@ pub fn bedrock_stream_event_to_tool_start(
}
}
pub fn bedrock_stream_event_to_tool_start_with_block_index(
event: &ConverseStreamOutput,
) -> Option<(usize, StreamingToolCall)> {
match event {
ConverseStreamOutput::ContentBlockStart(start) => {
let block_index = usize::try_from(start.content_block_index()).ok()?;
let tool_use = start.start().and_then(|s| s.as_tool_use().ok())?;
Some((
block_index,
StreamingToolCall {
id: tool_use.tool_use_id().to_string(),
name: tool_use.name().to_string(),
arguments: String::new(),
},
))
}
_ => None,
}
}
/// Extract tool use input delta from stream
pub fn bedrock_stream_event_to_tool_delta(event: &ConverseStreamOutput) -> Option<String> {
match event {
@@ -765,6 +785,22 @@ pub fn bedrock_stream_event_to_tool_delta(event: &ConverseStreamOutput) -> Optio
}
}
pub fn bedrock_stream_event_to_tool_delta_with_block_index(
event: &ConverseStreamOutput,
) -> Option<(usize, String)> {
match event {
ConverseStreamOutput::ContentBlockDelta(delta) => {
let block_index = usize::try_from(delta.content_block_index()).ok()?;
let input = delta
.delta()
.and_then(|d| d.as_tool_use().ok())
.map(|tool_use| tool_use.input().to_string())?;
Some((block_index, input))
}
_ => None,
}
}
/// Check if stream event indicates content block stop
pub fn bedrock_stream_event_is_block_stop(event: &ConverseStreamOutput) -> bool {
matches!(event, ConverseStreamOutput::ContentBlockStop(_))
+3 -2
View File
@@ -20,13 +20,14 @@ where
lazy_static::lazy_static! {
static ref OPENAI_AZURE_BASE_PATH: Option<String> = std::env::var("OPENAI_AZURE_BASE_PATH").ok();
static ref ALLOW_PRIVATE_AI_BASE_URLS: bool = std::env::var("ALLOW_PRIVATE_AI_BASE_URLS")
pub static ref ALLOW_PRIVATE_AI_BASE_URLS: bool = std::env::var("ALLOW_PRIVATE_AI_BASE_URLS")
.ok()
.map(|v| v == "true" || v == "1")
.unwrap_or(false);
}
pub const OPENAI_BASE_URL: &str = "https://api.openai.com/v1";
pub const DEEPSEEK_BASE_URL: &str = "https://api.deepseek.com/v1";
pub const GOOGLE_AI_BASE_URL: &str = "https://generativelanguage.googleapis.com/v1beta";
/// Empty string signals BedrockClient::from_env() to use the region from AWS environment/config
@@ -106,7 +107,7 @@ impl AIProvider {
Ok(azure_base_path.unwrap_or("https://api.openai.com/v1".to_string()))
}
AIProvider::DeepSeek => Ok("https://api.deepseek.com/v1".to_string()),
AIProvider::DeepSeek => Ok(DEEPSEEK_BASE_URL.to_string()),
AIProvider::GoogleAI => Ok(GOOGLE_AI_BASE_URL.to_string()),
AIProvider::Groq => Ok("https://api.groq.com/openai/v1".to_string()),
AIProvider::OpenRouter => Ok("https://openrouter.ai/api/v1".to_string()),
+25
View File
@@ -0,0 +1,25 @@
use std::collections::HashMap;
use crate::ai_providers::{AIPlatform, AIProvider};
/// Resolved provider credentials shared by API proxy and worker execution.
///
/// Raw API resources and worker agent payloads convert into this shape at their
/// execution boundaries. Request-specific state such as the selected model stays
/// outside this type.
#[derive(Clone, Debug)]
pub struct ProviderCredentials {
pub provider: AIProvider,
pub base_url: String,
pub api_key: Option<String>,
pub access_token: Option<String>,
pub organization_id: Option<String>,
pub user: Option<String>,
pub region: Option<String>,
pub aws_access_key_id: Option<String>,
pub aws_secret_access_key: Option<String>,
pub aws_session_token: Option<String>,
pub platform: AIPlatform,
pub enable_1m_context: bool,
pub custom_headers: HashMap<String, String>,
}
+1
View File
@@ -4,6 +4,7 @@ pub mod ai_cache;
pub mod ai_google;
pub mod ai_providers;
pub mod ai_types;
pub mod credentials;
pub mod image_handler;
pub mod providers;
pub mod proxy;
@@ -729,8 +729,7 @@ impl QueryBuilder for AnthropicQueryBuilder {
mod tests {
use super::*;
use crate::{
proxy::{ProviderCredentials, ProxyBuildArgs},
query_builder::QueryBuilder,
credentials::ProviderCredentials, proxy::ProxyBuildArgs, query_builder::QueryBuilder,
};
use http::{HeaderMap, HeaderValue, Method};
use std::collections::HashMap;
+228 -128
View File
@@ -10,9 +10,10 @@ use crate::{
ai_bedrock::{
bedrock_model_supports_prompt_caching, bedrock_stream_event_is_block_stop,
bedrock_stream_event_to_text, bedrock_stream_event_to_tool_delta,
bedrock_stream_event_to_tool_start, build_tool_config, create_inference_config,
format_bedrock_error, openai_messages_to_bedrock, streaming_tool_calls_to_openai,
BearerTokenProvider, BedrockClient, StreamingToolCall,
bedrock_stream_event_to_tool_delta_with_block_index, bedrock_stream_event_to_tool_start,
bedrock_stream_event_to_tool_start_with_block_index, build_tool_config,
create_inference_config, format_bedrock_error, openai_messages_to_bedrock,
streaming_tool_calls_to_openai, BearerTokenProvider, BedrockClient, StreamingToolCall,
},
ai_providers::USE_ENV_REGION,
ai_types::{OpenAIFunction, OpenAIToolCall, ToolDefFunction},
@@ -403,137 +404,15 @@ pub fn sdk_stream_to_sse(
.unwrap()
.as_secs();
struct StreamState {
id: String,
model: String,
created: u64,
tool_calls: HashMap<usize, (String, String, String)>,
current_tool_index: usize,
}
let state = std::sync::Arc::new(tokio::sync::Mutex::new(StreamState {
id,
model,
created,
tool_calls: HashMap::new(),
current_tool_index: 0,
}));
async_stream::stream! {
let mut stream = stream;
let state = state.clone();
let mut state = BedrockSseStreamState::new(id, model, created);
loop {
match stream.recv().await {
Ok(Some(event)) => {
let mut state = state.lock().await;
if let Some(tool_call) = bedrock_stream_event_to_tool_start(&event) {
let index = state.current_tool_index;
state.tool_calls.insert(
index,
(tool_call.id.clone(), tool_call.name.clone(), String::new()),
);
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"tool_calls": [{
"index": index,
"id": tool_call.id,
"type": "function",
"function": {
"name": tool_call.name,
"arguments": ""
}
}]
},
"finish_reason": serde_json::Value::Null
}]
});
yield Ok(Bytes::from(format!("data: {}\n\n", chunk)));
}
if let Some(text) = bedrock_stream_event_to_text(&event) {
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"content": text
},
"finish_reason": serde_json::Value::Null
}]
});
yield Ok(Bytes::from(format!("data: {}\n\n", chunk)));
}
if let Some(input_delta) = bedrock_stream_event_to_tool_delta(&event) {
let index = state.current_tool_index;
if let Some((_id, _name, ref mut args)) = state.tool_calls.get_mut(&index) {
args.push_str(&input_delta);
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"tool_calls": [{
"index": index,
"function": {
"arguments": input_delta
}
}]
},
"finish_reason": serde_json::Value::Null
}]
});
yield Ok(Bytes::from(format!("data: {}\n\n", chunk)));
}
}
if bedrock_stream_event_is_block_stop(&event) {
state.current_tool_index += 1;
}
if let aws_sdk_bedrockruntime::types::ConverseStreamOutput::MessageStop(stop) = &event {
let stop_reason = stop.stop_reason().as_str();
let finish_reason = match stop_reason {
"end_turn" => "stop",
"max_tokens" => "length",
"tool_use" => "tool_calls",
"stop_sequence" => "stop",
"guardrail_intervened" | "content_filtered" => "content_filter",
_ => "stop",
};
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {},
"finish_reason": finish_reason
}]
});
yield Ok(Bytes::from(format!("data: {}\n\n", chunk)));
for chunk in bedrock_sse_chunks_for_event(&event, &mut state) {
yield Ok(chunk);
}
}
Ok(None) => break,
@@ -551,6 +430,149 @@ pub fn sdk_stream_to_sse(
}
}
#[derive(Debug)]
struct BedrockSseStreamState {
id: String,
model: String,
created: u64,
tool_calls: HashMap<usize, (String, String, String)>,
tool_block_indexes: HashMap<usize, usize>,
next_tool_index: usize,
}
impl BedrockSseStreamState {
fn new(id: String, model: String, created: u64) -> Self {
Self {
id,
model,
created,
tool_calls: HashMap::new(),
tool_block_indexes: HashMap::new(),
next_tool_index: 0,
}
}
}
fn bedrock_sse_chunks_for_event(
event: &aws_sdk_bedrockruntime::types::ConverseStreamOutput,
state: &mut BedrockSseStreamState,
) -> Vec<Bytes> {
let mut chunks = Vec::new();
if let Some((block_index, tool_call)) =
bedrock_stream_event_to_tool_start_with_block_index(event)
{
let index = state.next_tool_index;
state.next_tool_index += 1;
state.tool_block_indexes.insert(block_index, index);
state.tool_calls.insert(
index,
(tool_call.id.clone(), tool_call.name.clone(), String::new()),
);
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"tool_calls": [{
"index": index,
"id": tool_call.id,
"type": "function",
"function": {
"name": tool_call.name,
"arguments": ""
}
}]
},
"finish_reason": serde_json::Value::Null
}]
});
chunks.push(Bytes::from(format!("data: {}\n\n", chunk)));
}
if let Some(text) = bedrock_stream_event_to_text(event) {
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"content": text
},
"finish_reason": serde_json::Value::Null
}]
});
chunks.push(Bytes::from(format!("data: {}\n\n", chunk)));
}
if let Some((block_index, input_delta)) =
bedrock_stream_event_to_tool_delta_with_block_index(event)
{
if let Some(index) = state.tool_block_indexes.get(&block_index).copied() {
if let Some((_id, _name, ref mut args)) = state.tool_calls.get_mut(&index) {
args.push_str(&input_delta);
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {
"tool_calls": [{
"index": index,
"function": {
"arguments": input_delta
}
}]
},
"finish_reason": serde_json::Value::Null
}]
});
chunks.push(Bytes::from(format!("data: {}\n\n", chunk)));
}
}
}
if let aws_sdk_bedrockruntime::types::ConverseStreamOutput::MessageStop(stop) = event {
let stop_reason = stop.stop_reason().as_str();
let finish_reason = match stop_reason {
"end_turn" => "stop",
"max_tokens" => "length",
"tool_use" => "tool_calls",
"stop_sequence" => "stop",
"guardrail_intervened" | "content_filtered" => "content_filter",
_ => "stop",
};
let chunk = serde_json::json!({
"id": state.id,
"object": "chat.completion.chunk",
"created": state.created,
"model": state.model,
"choices": [{
"index": 0,
"delta": {},
"finish_reason": finish_reason
}]
});
chunks.push(Bytes::from(format!("data: {}\n\n", chunk)));
}
chunks
}
async fn handle_bedrock_sdk_non_streaming(
model: &str,
body: &[u8],
@@ -970,6 +992,19 @@ impl BedrockQueryBuilder {
#[cfg(test)]
mod tests {
use super::*;
use aws_sdk_bedrockruntime::types::{
ContentBlockDelta, ContentBlockDeltaEvent, ContentBlockStart, ContentBlockStartEvent,
ContentBlockStopEvent, ConverseStreamOutput, ToolUseBlockDelta, ToolUseBlockStart,
};
fn sse_json(chunk: &Bytes) -> serde_json::Value {
let chunk = std::str::from_utf8(chunk).expect("SSE chunk should be UTF-8");
let payload = chunk
.strip_prefix("data: ")
.and_then(|chunk| chunk.strip_suffix("\n\n"))
.expect("chunk should be SSE data");
serde_json::from_str(payload).expect("chunk should contain JSON")
}
#[test]
fn determine_auth_config_prioritizes_bearer_token() {
@@ -1022,4 +1057,69 @@ mod tests {
let config = determine_auth_config(None, Some("AKIA123"), None, Some("session-token"));
assert!(matches!(config, BedrockAuthConfig::Environment));
}
#[test]
fn bedrock_sse_tool_indexes_ignore_text_block_stops() {
let mut state =
BedrockSseStreamState::new("chatcmpl-test".to_string(), "model".to_string(), 1);
let text_delta = ConverseStreamOutput::ContentBlockDelta(
ContentBlockDeltaEvent::builder()
.content_block_index(0)
.delta(ContentBlockDelta::Text("hello".to_string()))
.build()
.unwrap(),
);
assert_eq!(
bedrock_sse_chunks_for_event(&text_delta, &mut state).len(),
1
);
let text_stop = ConverseStreamOutput::ContentBlockStop(
ContentBlockStopEvent::builder()
.content_block_index(0)
.build()
.unwrap(),
);
assert!(bedrock_sse_chunks_for_event(&text_stop, &mut state).is_empty());
let tool_start = ConverseStreamOutput::ContentBlockStart(
ContentBlockStartEvent::builder()
.content_block_index(1)
.start(ContentBlockStart::ToolUse(
ToolUseBlockStart::builder()
.tool_use_id("call_1")
.name("lookup")
.build()
.unwrap(),
))
.build()
.unwrap(),
);
let start_chunks = bedrock_sse_chunks_for_event(&tool_start, &mut state);
let start_json = sse_json(&start_chunks[0]);
assert_eq!(
start_json["choices"][0]["delta"]["tool_calls"][0]["index"],
0
);
let tool_delta = ConverseStreamOutput::ContentBlockDelta(
ContentBlockDeltaEvent::builder()
.content_block_index(1)
.delta(ContentBlockDelta::ToolUse(
ToolUseBlockDelta::builder()
.input("{\"city\":\"Paris\"}")
.build()
.unwrap(),
))
.build()
.unwrap(),
);
let delta_chunks = bedrock_sse_chunks_for_event(&tool_delta, &mut state);
let delta_json = sse_json(&delta_chunks[0]);
assert_eq!(
delta_json["choices"][0]["delta"]["tool_calls"][0]["index"],
0
);
}
}
@@ -691,7 +691,7 @@ impl QueryBuilder for GoogleAIQueryBuilder {
#[cfg(test)]
mod tests {
use super::*;
use crate::{ai_providers::AIProvider, proxy::ProviderCredentials};
use crate::{ai_providers::AIProvider, credentials::ProviderCredentials};
use std::collections::HashMap;
fn credentials(base_url: &str, platform: AIPlatform) -> ProviderCredentials {
+3 -1
View File
@@ -6,7 +6,9 @@ pub mod openai;
pub mod openrouter;
pub mod other;
use crate::{ai_providers::AIProvider, proxy::ProviderCredentials, query_builder::QueryBuilder};
use crate::{
ai_providers::AIProvider, credentials::ProviderCredentials, query_builder::QueryBuilder,
};
use self::{
anthropic::AnthropicQueryBuilder, google_ai::GoogleAIQueryBuilder, openai::OpenAIQueryBuilder,
+6 -22
View File
@@ -4,30 +4,11 @@ use http::{HeaderMap, Method};
use serde_json::value::RawValue;
use windmill_common::error::{Error, Result};
use crate::ai_providers::{AIPlatform, AIProvider};
use crate::ai_providers::AIProvider;
use crate::credentials::ProviderCredentials;
use crate::utils::AI_HTTP_HEADERS;
/// Resolved provider credentials shared by API proxy and worker execution.
///
/// Raw API resources and worker agent payloads convert into this shape at their
/// execution boundaries. Request-specific state such as the selected model stays
/// outside this type.
#[derive(Clone, Debug)]
pub struct ProviderCredentials {
pub provider: AIProvider,
pub base_url: String,
pub api_key: Option<String>,
pub access_token: Option<String>,
pub organization_id: Option<String>,
pub user: Option<String>,
pub region: Option<String>,
pub aws_access_key_id: Option<String>,
pub aws_secret_access_key: Option<String>,
pub aws_session_token: Option<String>,
pub platform: AIPlatform,
pub enable_1m_context: bool,
pub custom_headers: HashMap<String, String>,
}
pub mod fim;
/// Inputs needed to transform an OpenAI-compatible proxy request for a provider.
pub struct ProxyBuildArgs<'a> {
@@ -167,6 +148,9 @@ pub(crate) fn add_user_to_body(body: &[u8], user: &str) -> Result<Vec<u8>> {
#[cfg(test)]
mod tests {
use super::*;
use std::collections::HashMap;
use crate::ai_providers::AIPlatform;
fn credentials(provider: AIProvider, base_url: &str) -> ProviderCredentials {
ProviderCredentials {
+212
View File
@@ -0,0 +1,212 @@
use bytes::Bytes;
use serde::Deserialize;
use serde_json::json;
use windmill_common::error::{Error, Result};
use crate::ai_providers::{AIProvider, DEEPSEEK_BASE_URL};
#[derive(Debug, Eq, PartialEq)]
pub struct FimProxyTransform {
pub body: Bytes,
pub path: String,
pub base_url: Option<String>,
}
#[derive(Deserialize)]
struct FimRequest {
model: String,
prompt: String,
suffix: Option<String>,
temperature: Option<f32>,
max_tokens: Option<u32>,
stop: Option<Vec<String>>,
}
pub fn supports_native_fim(provider: &AIProvider) -> bool {
matches!(provider, AIProvider::Mistral | AIProvider::DeepSeek)
}
fn deepseek_fim_base_url(base_url: &str) -> String {
let trimmed = base_url.trim_end_matches('/');
let deepseek_root_base_url = DEEPSEEK_BASE_URL
.strip_suffix("/v1")
.unwrap_or(DEEPSEEK_BASE_URL);
if trimmed == DEEPSEEK_BASE_URL || trimmed == deepseek_root_base_url {
return format!("{deepseek_root_base_url}/beta");
}
if let Some(prefix) = trimmed.strip_suffix("/v1") {
return format!("{prefix}/beta");
}
trimmed.to_string()
}
pub fn maybe_transform_fim_request(
provider: &AIProvider,
path: &str,
base_url: &str,
body: &[u8],
) -> Result<Option<FimProxyTransform>> {
if !path.contains("fim/completions") {
return Ok(None);
}
if matches!(provider, AIProvider::DeepSeek) {
return Ok(Some(FimProxyTransform {
body: Bytes::copy_from_slice(body),
path: "completions".to_string(),
base_url: Some(deepseek_fim_base_url(base_url)),
}));
}
if !supports_native_fim(provider) {
return transform_fim_to_chat_completions(body).map(Some);
}
Ok(None)
}
fn transform_fim_to_chat_completions(body: &[u8]) -> Result<FimProxyTransform> {
let fim_req: FimRequest = serde_json::from_slice(body)
.map_err(|e| Error::BadRequest(format!("Failed to parse FIM request: {}", e)))?;
let suffix = fim_req.suffix.unwrap_or_default();
let system_prompt = "You are a code completion assistant. Complete the code at the <CURSOR/> position between the given prefix and suffix. Output ONLY the code that goes at the cursor - no explanations, no markdown, no repeating the prefix or suffix.";
let user_content = format!(
"<PREFIX>\n{}\n<CURSOR/>\n<SUFFIX>\n{}",
fim_req.prompt, suffix
);
let chat_req = json!({
"model": fim_req.model,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_content}
],
"temperature": fim_req.temperature.unwrap_or(0.0),
"max_tokens": fim_req.max_tokens.unwrap_or(256),
"stop": fim_req.stop
});
let body = serde_json::to_vec(&chat_req)
.map_err(|e| Error::internal_err(format!("Failed to serialize chat request: {}", e)))?;
Ok(FimProxyTransform {
body: Bytes::from(body),
path: "chat/completions".to_string(),
base_url: None,
})
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn mistral_keeps_native_fim_request() {
let transformed = maybe_transform_fim_request(
&AIProvider::Mistral,
"fim/completions",
"https://api.mistral.ai/v1",
br#"{}"#,
)
.unwrap();
assert!(transformed.is_none());
assert!(supports_native_fim(&AIProvider::Mistral));
assert!(supports_native_fim(&AIProvider::DeepSeek));
assert!(!supports_native_fim(&AIProvider::OpenAI));
}
#[test]
fn deepseek_fim_base_url_uses_beta_endpoint() {
assert_eq!(
deepseek_fim_base_url("https://api.deepseek.com/v1"),
"https://api.deepseek.com/beta"
);
assert_eq!(
deepseek_fim_base_url("https://api.deepseek.com/v1/"),
"https://api.deepseek.com/beta"
);
assert_eq!(
deepseek_fim_base_url("https://api.deepseek.com"),
"https://api.deepseek.com/beta"
);
assert_eq!(
deepseek_fim_base_url("https://proxy.example/deepseek/v1"),
"https://proxy.example/deepseek/beta"
);
assert_eq!(
deepseek_fim_base_url("https://proxy.example/deepseek/beta"),
"https://proxy.example/deepseek/beta"
);
}
#[test]
fn deepseek_fim_request_uses_beta_completions_endpoint() {
let body = br#"{"model":"deepseek-v4-pro","prompt":"return ","suffix":";"}"#;
let transformed = maybe_transform_fim_request(
&AIProvider::DeepSeek,
"fim/completions",
DEEPSEEK_BASE_URL,
body,
)
.unwrap()
.expect("DeepSeek FIM should be routed to the beta completions endpoint");
assert_eq!(transformed.path, "completions");
assert_eq!(
transformed.base_url.as_deref(),
Some("https://api.deepseek.com/beta")
);
assert_eq!(transformed.body, Bytes::copy_from_slice(body));
}
#[test]
fn openai_fim_request_is_transformed_to_chat_completion() {
let transformed = maybe_transform_fim_request(
&AIProvider::OpenAI,
"fim/completions",
"https://api.openai.com/v1",
br#"{
"model": "gpt-4.1",
"prompt": "fn main() {",
"suffix": "}",
"stop": ["\n\n"]
}"#,
)
.unwrap()
.expect("OpenAI FIM should be transformed");
assert_eq!(transformed.path, "chat/completions");
assert_eq!(transformed.base_url, None);
let body: serde_json::Value = serde_json::from_slice(&transformed.body).unwrap();
assert_eq!(body["model"], "gpt-4.1");
assert_eq!(body["temperature"], 0.0);
assert_eq!(body["max_tokens"], 256);
assert_eq!(body["stop"], serde_json::json!(["\n\n"]));
assert_eq!(body["messages"][1]["role"], "user");
assert_eq!(
body["messages"][1]["content"],
"<PREFIX>\nfn main() {\n<CURSOR/>\n<SUFFIX>\n}"
);
}
#[test]
fn invalid_fim_body_is_bad_request() {
let err = maybe_transform_fim_request(
&AIProvider::OpenAI,
"fim/completions",
"https://api.openai.com/v1",
br#"{"model": 1}"#,
)
.unwrap_err();
assert!(matches!(err, Error::BadRequest(_)));
}
}
+1 -1
View File
@@ -18,7 +18,7 @@ pub struct McpToolSource {
use crate::{
ai_google::sanitize_schema_for_google,
ai_providers::{empty_string_as_none, AIProvider},
proxy::ProviderCredentials,
credentials::ProviderCredentials,
};
use windmill_common::{db::DB, error::Error, flow_status::AgentAction, flows::FlowModule};
use windmill_parser::Typ;
+477
View File
@@ -235,6 +235,202 @@ where
Ok(())
}
/// Returns the caller's "real" scope restrictions: every scope other than
/// `if_jobs:filter_tags:` tag filters. `None` means the token is unscoped and
/// has the full privileges of its user; `Some` means it is restricted to the
/// returned scopes. An empty or filter-tags-only scope list is treated as
/// unscoped, mirroring `check_scopes`/`check_route_access`.
fn scope_restrictions(scopes: Option<&[String]>) -> Option<Vec<&String>> {
let restrictions: Vec<&String> = scopes?
.iter()
.filter(|s| !s.starts_with("if_jobs:filter_tags:"))
.collect();
(!restrictions.is_empty()).then_some(restrictions)
}
/// Enforce monotonic privilege when a token lifecycle endpoint mints or rescopes
/// a credential on behalf of `authed`: the resulting credential must never be
/// more privileged than the caller's own token.
///
/// - An unscoped caller may grant any scopes (this is the existing UI/CLI flow).
/// - A scope-restricted caller may only grant scopes that are a subset of its
/// own, and may never produce an unscoped credential.
///
/// Without this, a `users:write` token could create or rescope a token to be
/// unscoped, and a `users:read` token could refresh into an unscoped session —
/// escaping its own restrictions.
pub fn ensure_scopes_within_caller(
authed: &ApiAuthed,
requested_scopes: Option<&[String]>,
) -> error::Result<()> {
if let Some(caller_restrictions) = scope_restrictions(authed.scopes.as_deref()) {
let Some(requested_restrictions) = scope_restrictions(requested_scopes) else {
return Err(Error::PermissionDenied(
"A scope-restricted token cannot create or update a token with broader (unscoped) \
privileges"
.to_string(),
));
};
// MCP scopes (`mcp:all`, `mcp:favorites`, `mcp:scripts:*`, etc.) use a
// custom format that ScopeDefinition::from_scope_string parses
// permissively but the MCP runtime interprets via its own parser
// (parse_mcp_scopes). The two views disagree — e.g. the generic parser
// accepts `mcp:scripts` as an unrestricted-resource scope, while the
// MCP runtime ignores it as unrecognized but interprets `mcp:scripts:*`
// as granting all scripts. So generic containment would silently allow
// `mcp:scripts` → `mcp:scripts:*` (a widening). Legitimate MCP token
// issuance goes through the OAuth gateway (mcp/oauth_server.rs), not
// these user-token endpoints, so require byte-identical match for MCP
// scopes here rather than trying to mirror MCP semantics in two places.
// Unparseable non-MCP caller scopes are intentionally dropped
// (fail-closed): a caller scope that fails to parse can only narrow
// the set of requested scopes that get covered, never widen it.
// Unparseable requested scopes surface as `BadRequest`, which is what
// we want — the client is sending garbage.
let parsed_caller: Vec<ScopeDefinition> = caller_restrictions
.iter()
.filter(|s| !s.starts_with("mcp:"))
.filter_map(|s| ScopeDefinition::from_scope_string(s).ok())
.collect();
let caller_mcp: std::collections::HashSet<&str> = caller_restrictions
.iter()
.filter(|s| s.starts_with("mcp:"))
.map(|s| s.as_str())
.collect();
for requested in requested_restrictions {
if requested.starts_with("mcp:") {
if !caller_mcp.contains(requested.as_str()) {
return Err(Error::PermissionDenied(format!(
"A scope-restricted token cannot grant MCP scope '{requested}' unless the \
caller holds the same scope verbatim"
)));
}
continue;
}
let requested_scope = ScopeDefinition::from_scope_string(requested)?;
let covered = parsed_caller
.iter()
.any(|caller_scope| scope_contains(caller_scope, &requested_scope));
if !covered {
return Err(Error::PermissionDenied(format!(
"A scope-restricted token cannot grant scope '{requested}' which exceeds its \
own scopes"
)));
}
}
}
// `if_jobs:filter_tags:` fences which job tags a token can run on (enforced
// at job operations as `v2_job.tag = ANY(...)`), and is checked independently
// of domain/action/resource subset. A caller restricted by filter_tags must
// not be able to mint or rescope a credential that drops or widens the fence
// — even if the caller has no other scope restrictions (filter_tags-only
// tokens otherwise look "unscoped" to `scope_restrictions`).
if let Some(caller_tags) = first_filter_tags(authed.scopes.as_deref()) {
let Some(requested_tags) = first_filter_tags(requested_scopes) else {
return Err(Error::PermissionDenied(
"A token restricted by if_jobs:filter_tags cannot mint or rescope a token that \
drops the tag restriction"
.to_string(),
));
};
let caller_set: std::collections::HashSet<&str> = caller_tags.iter().copied().collect();
for tag in &requested_tags {
if !caller_set.contains(tag) {
return Err(Error::PermissionDenied(format!(
"A token restricted by if_jobs:filter_tags cannot grant tag '{tag}' which is \
not within its own filter_tags"
)));
}
}
}
Ok(())
}
/// Tags from the first `if_jobs:filter_tags:<a,b,...>` scope, matching the
/// semantics of [`get_scope_tags`] (which is what the job runtime consults).
/// Returns `None` if no such scope is present.
fn first_filter_tags(scopes: Option<&[String]>) -> Option<Vec<&str>> {
scopes?.iter().find_map(|s| {
s.strip_prefix("if_jobs:filter_tags:")
.map(|tags| tags.split(',').collect())
})
}
/// Whether `caller` grants at least everything `requested` grants (directional
/// containment).
///
/// This is intentionally NOT `ScopeDefinition::includes`: that method answers
/// "does this scope grant access to a required action" using OR semantics over
/// resources (any overlap counts, and a `*` on either side matches), which is
/// correct for access checks but unsafe for subset checks — it would let a
/// token scoped to `scripts:read:f/team/a` mint `scripts:read:*` or
/// `scripts:read:f/team/a,f/other/b`. Subset containment instead requires that
/// EVERY requested resource is covered by SOME caller resource.
fn scope_contains(caller: &ScopeDefinition, requested: &ScopeDefinition) -> bool {
if caller.domain != requested.domain {
return false;
}
// write subsumes read; otherwise the action must match exactly.
match (caller.action.as_str(), requested.action.as_str()) {
(c, r) if c == r || (c == "write" && r == "read") => {}
_ => return false,
}
if caller.domain == "jobs" && caller.action == "run" {
match (&caller.kind, &requested.kind) {
(Some(caller_kind), Some(requested_kind)) if caller_kind != requested_kind => {
return false
}
// Caller pinned to a kind, but the request covers any kind.
(Some(_), None) => return false,
_ => {}
}
}
match (&caller.resource, &requested.resource) {
// Caller is unrestricted on resources: covers everything.
(None, _) => true,
// Caller is resource-restricted but the request is not: broader.
(Some(_), None) => false,
(Some(caller_resources), Some(requested_resources)) => {
resource_set_contains(caller_resources, requested_resources)
}
}
}
/// Every resource in `requested` must be covered by some resource in `caller`.
fn resource_set_contains(caller: &[String], requested: &[String]) -> bool {
if caller.iter().any(|r| r == "*") {
return true;
}
requested
.iter()
.all(|req| req != "*" && caller.iter().any(|c| resource_covers(c, req)))
}
/// Directional: does the single caller resource pattern cover `requested`?
/// `caller` may be an exact path or a `<prefix>/*` subtree wildcard; `requested`
/// may itself be a subtree wildcard, in which case the whole requested subtree
/// must fall within the caller's subtree.
fn resource_covers(caller: &str, requested: &str) -> bool {
if caller == requested {
return true;
}
let Some(prefix) = caller.strip_suffix("/*") else {
// An exact caller resource only covers itself (handled above).
return false;
};
let requested_base = requested.strip_suffix("/*").unwrap_or(requested);
requested_base == prefix
|| (requested_base.starts_with(prefix)
&& requested_base.as_bytes().get(prefix.len()) == Some(&b'/'))
}
/// Returns a predicate that checks whether `path` is within the token's
/// scope for `{domain}:{action}:{path}`. For tokens without scope
/// restrictions (no scopes at all, or only `if_jobs:filter_tags:*` scopes),
@@ -574,6 +770,13 @@ impl NewToken {
}
}
/// Low-level token mint shared by trusted callers (the user-facing
/// `tokens/create` handler and internal mints such as native-trigger webhook
/// tokens). It does NOT enforce that `token_config.scopes` is within the
/// caller's own scopes — callers exposed to untrusted input must call
/// [`ensure_scopes_within_caller`] first (internal narrowing mints intentionally
/// skip it, since their scopes derive from the action being authorized, not the
/// caller's token).
pub async fn create_token_internal(
tx: &mut sqlx::PgConnection,
db: &DB,
@@ -908,4 +1111,278 @@ mod tests {
assert!(allowed("u/alice/foo"));
assert!(!allowed("u/alice/bar"));
}
fn opt_scopes(scopes: Option<Vec<&str>>) -> Option<Vec<String>> {
scopes.map(|v| v.into_iter().map(String::from).collect())
}
// Regression tests for WIN-1999: scoped user tokens must not be able to
// mint or rescope credentials with broader privileges than themselves.
#[test]
fn unscoped_caller_can_grant_anything() {
let authed = authed_with_scopes(None);
assert!(ensure_scopes_within_caller(&authed, None).is_ok());
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["jobs:run:scripts"])).as_deref()
)
.is_ok());
}
#[test]
fn filter_tags_only_caller_is_unrestricted_on_domain_action_dimension() {
// The domain/action/resource subset check treats filter-tags-only as
// unrestricted, mirroring check_scopes/check_route_access. The tag
// dimension is checked separately (see filter_tags_dimension_is_monotonic).
let authed = authed_with_scopes(Some(vec!["if_jobs:filter_tags:default"]));
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["users:write", "if_jobs:filter_tags:default"])).as_deref()
)
.is_ok());
}
#[test]
fn filter_tags_dimension_is_monotonic() {
// Caller restricted to tag fence "a" cannot drop the fence …
let single = authed_with_scopes(Some(vec!["if_jobs:filter_tags:a"]));
assert!(ensure_scopes_within_caller(&single, None).is_err());
assert!(
ensure_scopes_within_caller(&single, opt_scopes(Some(vec!["users:read"])).as_deref())
.is_err(),
"minting a token without filter_tags must be rejected"
);
// … cannot widen to a tag it lacks …
assert!(ensure_scopes_within_caller(
&single,
opt_scopes(Some(vec!["if_jobs:filter_tags:a,b"])).as_deref()
)
.is_err());
// … and cannot mint a token fenced on a disjoint tag.
assert!(ensure_scopes_within_caller(
&single,
opt_scopes(Some(vec!["if_jobs:filter_tags:b"])).as_deref()
)
.is_err());
// Narrowing or matching the tag fence is allowed.
let multi = authed_with_scopes(Some(vec!["if_jobs:filter_tags:a,b"]));
assert!(ensure_scopes_within_caller(
&multi,
opt_scopes(Some(vec!["if_jobs:filter_tags:a"])).as_deref()
)
.is_ok());
assert!(ensure_scopes_within_caller(
&multi,
opt_scopes(Some(vec!["if_jobs:filter_tags:a,b"])).as_deref()
)
.is_ok());
// A caller with a real scope plus a tag fence cannot drop just the fence.
let mixed = authed_with_scopes(Some(vec!["jobs:run:scripts", "if_jobs:filter_tags:a"]));
assert!(ensure_scopes_within_caller(
&mixed,
opt_scopes(Some(vec!["jobs:run:scripts"])).as_deref()
)
.is_err());
assert!(ensure_scopes_within_caller(
&mixed,
opt_scopes(Some(vec!["jobs:run:scripts", "if_jobs:filter_tags:a"])).as_deref()
)
.is_ok());
// An unrestricted caller may grant filter_tags freely.
let unscoped = authed_with_scopes(None);
assert!(ensure_scopes_within_caller(
&unscoped,
opt_scopes(Some(vec!["if_jobs:filter_tags:x"])).as_deref()
)
.is_ok());
}
#[test]
fn scoped_caller_cannot_mint_unscoped_token() {
// Primitive 2 in the report: a users:write token minting an unscoped token.
let authed = authed_with_scopes(Some(vec!["users:write"]));
assert!(ensure_scopes_within_caller(&authed, None).is_err());
// Empty scope list is effectively unscoped and must also be rejected.
assert!(ensure_scopes_within_caller(&authed, Some(&[])).is_err());
// A scope list of only tag filters is effectively unscoped too.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["if_jobs:filter_tags:default"])).as_deref()
)
.is_err());
}
#[test]
fn scoped_caller_cannot_remove_its_own_scopes() {
// Primitive 3 in the report: a users:write token setting its scopes to null.
let authed = authed_with_scopes(Some(vec!["users:write"]));
assert!(ensure_scopes_within_caller(&authed, None).is_err());
}
#[test]
fn scoped_caller_cannot_grant_scope_it_lacks() {
let authed = authed_with_scopes(Some(vec!["users:write"]));
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["jobs:run:scripts"])).as_deref()
)
.is_err());
}
#[test]
fn scoped_caller_can_grant_subset_of_own_scopes() {
let authed = authed_with_scopes(Some(vec!["users:write", "jobs:run:scripts"]));
// Equal scope is allowed.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["jobs:run:scripts"])).as_deref()
)
.is_ok());
// write implies read, so a narrower read scope is allowed.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["users:read"])).as_deref()
)
.is_ok());
// Tag filters narrow further and are always permitted.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["users:read", "if_jobs:filter_tags:default"])).as_deref()
)
.is_ok());
}
#[test]
fn scoped_caller_cannot_broaden_resource_scope() {
let authed = authed_with_scopes(Some(vec!["scripts:read:f/team/*"]));
// Narrower resource within the subtree is allowed.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["scripts:read:f/team/sub"])).as_deref()
)
.is_ok());
// A nested subtree within the caller's subtree is allowed.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["scripts:read:f/team/sub/*"])).as_deref()
)
.is_ok());
// The subtree root itself is allowed.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["scripts:read:f/team"])).as_deref()
)
.is_ok());
// A path outside the subtree is rejected.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["scripts:read:f/other/x"])).as_deref()
)
.is_err());
// read caller cannot grant write.
assert!(ensure_scopes_within_caller(
&authed,
opt_scopes(Some(vec!["scripts:write:f/team/db"])).as_deref()
)
.is_err());
}
#[test]
fn mcp_scopes_require_byte_identical_match() {
// Regression for the access-grant-OR vs runtime-MCP-parser confusion:
// ScopeDefinition treats `mcp:scripts` as an unrestricted-resource scope
// and `mcp:scripts:*` as a strictly narrower one, so generic containment
// would silently allow widening. The MCP runtime however ignores
// `mcp:scripts` (unrecognized) while `mcp:scripts:*` grants all scripts.
// Legitimate MCP token issuance is the OAuth gateway, not these
// user-token endpoints, so MCP scopes must match the caller verbatim.
// The bypass the reviewer flagged: malformed `mcp:scripts` would widen
// into the real `mcp:scripts:*` under generic containment.
let bypass = authed_with_scopes(Some(vec!["users:write", "mcp:scripts"]));
assert!(ensure_scopes_within_caller(
&bypass,
opt_scopes(Some(vec!["users:write", "mcp:scripts:*"])).as_deref()
)
.is_err());
// A caller without any MCP scope cannot grant one (widening on the MCP
// dimension), even if the rest of the requested scopes are within reach.
let no_mcp = authed_with_scopes(Some(vec!["users:write"]));
assert!(ensure_scopes_within_caller(
&no_mcp,
opt_scopes(Some(vec!["users:write", "mcp:scripts:*"])).as_deref()
)
.is_err());
// Byte-identical MCP scope passes; an additional non-matching MCP scope
// alongside it does not.
let mcp_caller = authed_with_scopes(Some(vec!["mcp:scripts:*"]));
assert!(ensure_scopes_within_caller(
&mcp_caller,
opt_scopes(Some(vec!["mcp:scripts:*"])).as_deref()
)
.is_ok());
assert!(ensure_scopes_within_caller(
&mcp_caller,
opt_scopes(Some(vec!["mcp:scripts:*", "mcp:flows:*"])).as_deref()
)
.is_err());
// Even a narrowing within MCP semantics (`mcp:all` → `mcp:scripts:*`)
// is rejected by the byte-identical rule. This is intentional — these
// endpoints are not the legitimate path for narrowing MCP tokens.
let mcp_all = authed_with_scopes(Some(vec!["mcp:all"]));
assert!(ensure_scopes_within_caller(
&mcp_all,
opt_scopes(Some(vec!["mcp:scripts:*"])).as_deref()
)
.is_err());
}
#[test]
fn scoped_caller_cannot_escalate_to_wildcard_or_superset() {
// Regression for the access-grant-OR vs subset-containment confusion:
// ScopeDefinition::includes would (incorrectly) allow all of these.
let star = authed_with_scopes(Some(vec!["scripts:read:f/team/a"]));
// Minting `*` from a single-path scope must be rejected.
assert!(ensure_scopes_within_caller(
&star,
opt_scopes(Some(vec!["scripts:read:*"])).as_deref()
)
.is_err());
// Minting a broader subtree must be rejected.
assert!(ensure_scopes_within_caller(
&star,
opt_scopes(Some(vec!["scripts:read:f/team/*"])).as_deref()
)
.is_err());
// A comma-separated list that adds an uncovered resource must be rejected,
// even though one element overlaps the caller's scope.
let list = authed_with_scopes(Some(vec!["scripts:read:f/team/a"]));
assert!(ensure_scopes_within_caller(
&list,
opt_scopes(Some(vec!["scripts:read:f/team/a,f/other/b"])).as_deref()
)
.is_err());
// A subset of a multi-resource caller scope is allowed.
let multi = authed_with_scopes(Some(vec!["scripts:read:f/team/a,f/team/b"]));
assert!(ensure_scopes_within_caller(
&multi,
opt_scopes(Some(vec!["scripts:read:f/team/a"])).as_deref()
)
.is_ok());
// A wildcard caller covers any subset, but not `*`-less escalation rules apply
// only when the caller itself lacks `*`.
let wildcard = authed_with_scopes(Some(vec!["scripts:read:*"]));
assert!(ensure_scopes_within_caller(
&wildcard,
opt_scopes(Some(vec!["scripts:read:f/team/a"])).as_deref()
)
.is_ok());
}
}
+33 -17
View File
@@ -16,7 +16,8 @@ use axum::{
};
use windmill_api_auth::{
auth::{list_tokens_internal, TruncatedTokenWithEmail},
check_scopes, maybe_refresh_folders, require_owner_of_path, ApiAuthed,
build_scope_path_predicate, check_scopes, maybe_refresh_folders, require_owner_of_path,
ApiAuthed,
};
use windmill_common::workspaces::{check_deploy_rules, RuleCheckResult};
use windmill_common::{
@@ -108,9 +109,10 @@ async fn list_search_flows(
let n = 3;
let mut tx = user_db.begin(&authed).await?;
let allowed = build_scope_path_predicate(&authed, "flows", "read");
let rows = sqlx::query_as::<_, SearchFlow>(
"SELECT flow.path, flow_version.value
FROM flow
FROM flow
LEFT JOIN flow_version ON flow_version.id = flow.versions[array_upper(flow.versions, 1)]
WHERE flow.workspace_id = $1 LIMIT $2",
)
@@ -119,6 +121,7 @@ async fn list_search_flows(
.fetch_all(&mut *tx)
.await?
.into_iter()
.filter(|r| allowed(&r.path))
.collect::<Vec<_>>();
tx.commit().await?;
Ok(Json(rows))
@@ -212,9 +215,13 @@ async fn list_flows(
let sql = sqlb.sql().map_err(|e| Error::internal_err(e.to_string()))?;
let mut tx = user_db.begin(&authed).await?;
let allowed = build_scope_path_predicate(&authed, "flows", "read");
let rows = sqlx::query_as::<_, ListableFlow>(&sql)
.fetch_all(&mut *tx)
.await?;
.await?
.into_iter()
.filter(|r| allowed(&r.path))
.collect::<Vec<_>>();
tx.commit().await?;
Ok(Json(rows))
}
@@ -558,13 +565,17 @@ async fn create_flow(
w_id
).execute(&mut *tx).await?;
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'flow'",
nf.path,
&w_id
)
.execute(&mut *tx)
.await?;
// CLI / git-sync deploys ask us to preserve any existing user draft at this
// path instead of wiping it as part of the deploy.
if !nf.skip_draft_deletion.unwrap_or(false) {
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'flow'",
nf.path,
&w_id
)
.execute(&mut *tx)
.await?;
}
audit_log(
&mut *tx,
@@ -1157,13 +1168,17 @@ async fn update_flow(
})?;
}
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'flow'",
flow_path,
&w_id
)
.execute(&mut *tx)
.await?;
// CLI / git-sync deploys ask us to preserve any existing user draft at this
// path instead of wiping it as part of the deploy.
if !nf.skip_draft_deletion.unwrap_or(false) {
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'flow'",
flow_path,
&w_id
)
.execute(&mut *tx)
.await?;
}
audit_log(
&mut *tx,
@@ -2031,6 +2046,7 @@ mod tests {
})),
preprocessor_module: None,
same_worker: false,
preserve_step_tags: false,
skip_expr: None,
cache_ttl: None,
cache_ignore_s3_path: None,
@@ -383,6 +383,8 @@ async fn remove_granular_acl(
// workspace export.
let table = if kind == "raw_app" { "app" } else { kind };
// SAFETY: `kind` has been validated against the `KINDS` allowlist before reaching this function.
// LIMIT 1: `script` shares (workspace_id, path) across versions, so `old` can
// return >1 row, which would break the scalar subquery in RETURNING.
let obj_o = sqlx::query_scalar::<_, bool>(&format!(
"WITH old AS (
SELECT extra_perms->$1 as old_write FROM {table}
@@ -390,7 +392,7 @@ async fn remove_granular_acl(
)
UPDATE {table} SET extra_perms = extra_perms - $1
WHERE {identifier} = $2 AND workspace_id = $3 AND extra_perms ? $1
RETURNING (SELECT old_write FROM old)::bool"
RETURNING (SELECT old_write FROM old LIMIT 1)::bool"
))
.bind(&owner)
.bind(path)
@@ -0,0 +1,23 @@
-- Fixture for the resource-value interpolation cache RLS regression test.
-- Extends base.sql (which defines test-user [admin], test-user-2, test-user-3
-- and their tokens).
--
-- A folder `secret` is readable ONLY by test-user-2 (via extra_perms). It holds a
-- variable and a resource that interpolates it. test-user-3 has no access to the
-- folder, so a cache entry warmed by test-user-2 with allow_cache=true must never
-- be served back to test-user-3.
INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', 'secret', 'Secret Folder', '{}',
'{"u/test-user-2": true}', 'test-user');
-- A (non-secret) variable gated to the `secret` folder; its value gets interpolated
-- into the resource value below and ends up in the cached, already-resolved blob.
INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms)
VALUES ('test-workspace', 'f/secret/db_password', 'LEAKED_FOLDER_SECRET', false,
'Folder-gated secret', '{}');
INSERT INTO resource (workspace_id, path, value, description, resource_type, extra_perms, created_by)
VALUES ('test-workspace', 'f/secret/cache_target',
'{"host": "db.internal", "password": "$var:f/secret/db_password"}',
'Folder-gated resource referencing a folder-gated variable', 'object', '{}', 'test-user');
@@ -0,0 +1,15 @@
-- Fixture for the variable-value cache RLS regression test.
-- Extends base.sql (which defines test-user [admin], test-user-2, test-user-3
-- and their tokens).
--
-- A folder `secret` is readable ONLY by test-user-2 (via extra_perms). It holds a
-- variable that test-user-2 can read but test-user-3 cannot. A cache entry warmed
-- by test-user-2 with allow_cache=true must never be served back to test-user-3.
INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', 'secret', 'Secret Folder', '{}',
'{"u/test-user-2": true}', 'test-user');
INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms)
VALUES ('test-workspace', 'f/secret/cache_target_var', 'LEAKED_VAR_SECRET', false,
'Folder-gated variable', '{}');
@@ -259,12 +259,10 @@ async fn test_flow_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
// ===== Hub endpoints (require external network, expect 500 or 200) =====
// --- hub/list ---
let resp = authed(client().get(format!(
"http://localhost:{port}/api/flows/hub/list"
)))
.send()
.await
.unwrap();
let resp = authed(client().get(format!("http://localhost:{port}/api/flows/hub/list")))
.send()
.await
.unwrap();
assert!(
resp.status() == 200 || resp.status() == 500,
"hub/list: unexpected status {}",
@@ -272,12 +270,10 @@ async fn test_flow_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
);
// --- hub/get ---
let resp = authed(client().get(format!(
"http://localhost:{port}/api/flows/hub/get/1"
)))
.send()
.await
.unwrap();
let resp = authed(client().get(format!("http://localhost:{port}/api/flows/hub/get/1")))
.send()
.await
.unwrap();
assert!(
resp.status() == 200 || resp.status() == 500,
"hub/get: unexpected status {}",
@@ -286,3 +282,98 @@ async fn test_flow_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
Ok(())
}
/// Regression test for GHSA-2ppx-66jv-wpw5: a path-scoped token must only see
/// the flows within its scope when listing, even though the route-level scope
/// check only validates `domain:action`. Before the fix, `list_search` returned
/// `path` + the full flow `value` for every flow the underlying user could see,
/// leaking out-of-scope flow definitions to narrowly-scoped tokens.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_list_search_scope_filtering(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/flows");
// Create two folders and one flow in each, as the (super-admin) test user.
for folder in ["allowed", "private"] {
let resp = authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/folders/create"
)))
.json(&json!({ "name": folder }))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200, "create folder: {}", resp.text().await?);
}
for path in ["f/allowed/foo", "f/private/bar"] {
let resp = authed(client().post(format!("{base}/create")))
.json(&new_flow(path, "summary"))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 201, "create {path}: {}", resp.text().await?);
}
// Helper: GET /list_search with an arbitrary bearer token, returning the set
// of flow paths visible to that token.
async fn list_search_paths(port: u16, token: &str) -> Vec<String> {
let resp = client()
.get(format!(
"http://localhost:{port}/api/w/test-workspace/flows/list_search"
))
.header("Authorization", format!("Bearer {token}"))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200);
resp.json::<Vec<serde_json::Value>>()
.await
.unwrap()
.into_iter()
.map(|s| s["path"].as_str().unwrap().to_string())
.collect()
}
// Insert three tokens for the same super-admin user, differing only by scope.
sqlx::query(
"INSERT INTO token (token_hash, token_prefix, token, email, label, super_admin, scopes) VALUES
(encode(sha256('SCOPED_TOKEN'::bytea), 'hex'), 'SCOPED_TOK', 'SCOPED_TOKEN', 'test@windmill.dev', 'scoped', true, ARRAY['flows:read:f/allowed/*']),
(encode(sha256('BROAD_TOKEN'::bytea), 'hex'), 'BROAD_TOK', 'BROAD_TOKEN', 'test@windmill.dev', 'broad', true, ARRAY['flows:read']),
(encode(sha256('TAG_TOKEN'::bytea), 'hex'), 'TAG_TOK', 'TAG_TOKEN', 'test@windmill.dev', 'tag-only', true, ARRAY['if_jobs:filter_tags:default'])",
)
.execute(&db)
.await?;
// Path-scoped token: only sees flows within `f/allowed/*`.
let scoped = list_search_paths(port, "SCOPED_TOKEN").await;
assert!(
scoped.contains(&"f/allowed/foo".to_string()),
"scoped token should see f/allowed/foo, got: {scoped:?}"
);
assert!(
!scoped.contains(&"f/private/bar".to_string()),
"scoped token must NOT see f/private/bar, got: {scoped:?}"
);
// Broad `flows:read` token: still sees every RLS-visible flow.
let broad = list_search_paths(port, "BROAD_TOKEN").await;
assert!(broad.contains(&"f/allowed/foo".to_string()));
assert!(
broad.contains(&"f/private/bar".to_string()),
"broad flows:read token should see all flows, got: {broad:?}"
);
// Tag-filter-only token is not scope-restricted: unchanged, sees all.
let tag_only = list_search_paths(port, "TAG_TOKEN").await;
assert!(tag_only.contains(&"f/allowed/foo".to_string()));
assert!(tag_only.contains(&"f/private/bar".to_string()));
// Unscoped token (no scopes column set): unchanged, sees all.
let unscoped = list_search_paths(port, "SECRET_TOKEN").await;
assert!(unscoped.contains(&"f/allowed/foo".to_string()));
assert!(unscoped.contains(&"f/private/bar".to_string()));
Ok(())
}
@@ -477,6 +477,117 @@ async fn test_resource_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
Ok(())
}
/// Regression test: the resource-value interpolation cache
/// (`get_value_interpolated?allow_cache=true`) must be identity-scoped. test-user-2
/// (folder access) warms the cache; test-user-3 (no access) must then be denied rather
/// than served the cached, already-decrypted value. Pre-fix the unscoped key returned
/// a 200 with the secret here.
#[sqlx::test(migrations = "../migrations", fixtures("base", "resource_cache_rls"))]
async fn test_resource_value_cache_is_identity_scoped(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let url = format!(
"{}?allow_cache=true",
resource_url(port, "get_value_interpolated", "f/secret/cache_target")
);
let get = |token: &str| {
client()
.get(url.as_str())
.header("Authorization", format!("Bearer {token}"))
};
// test-user-2 has folder access and WARMS the cache.
let resp = get("SECRET_TOKEN_2").send().await?;
assert_eq!(resp.status(), 200);
assert!(resp.text().await?.contains("LEAKED_FOLDER_SECRET"));
// test-user-3 has no folder access: must miss the cache and be denied (401), not leak.
let resp = get("SECRET_TOKEN_3").send().await?;
assert_eq!(resp.status(), 401);
assert!(!resp.text().await?.contains("LEAKED_FOLDER_SECRET"));
Ok(())
}
/// A resource whose value contains a `$WM_*` contextual variable (e.g. `$WM_TOKEN`) is
/// job-dependent and must NEVER be cached — even when first read WITHOUT a `job_id`, where the
/// placeholder is left unresolved (caching that would serve a stale placeholder to a later job
/// read). Any other value — plain, or a non-`$WM_` `$`-string like `$HOME` (which is NOT
/// interpolated, so it's constant) — is job-independent and IS cached, with the entry shared
/// across job contexts (a read carrying a `job_id` still hits it, keeping the hit ratio up).
/// We prove all three by warming each (no job_id), deleting the row directly (cache survives),
/// then re-reading: the job-independent ones are still served from cache — even under a
/// `job_id` — while the `$WM_*` one was never cached and 404s.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_resource_cache_handles_job_context(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/resources");
let plain = "u/test-user/plain_res";
let dollar = "u/test-user/dollar_res"; // non-$WM_ `$`-string: not interpolated, cacheable
let jobctx = "u/test-user/jobctx_res";
for (path, value) in [
(plain, json!({"v": 1})),
(dollar, json!({"d": "$HOME"})),
(jobctx, json!({"j": "$WM_JOB_ID"})),
] {
let resp = authed(client().post(format!("{base}/create")))
.json(
&json!({ "path": path, "value": value, "description": "", "resource_type": "object" }),
)
.send()
.await?;
assert_eq!(resp.status(), 201);
}
let get = |path: &str, query: &str| {
let url = format!("{base}/get_value_interpolated/{path}?{query}");
async move { authed(client().get(url)).send().await.unwrap() }
};
// Warm all three WITHOUT a job context (the placeholder is left unresolved for `jobctx`).
for path in [plain, dollar, jobctx] {
assert_eq!(get(path, "allow_cache=true").await.status(), 200);
}
// Delete the rows directly — bypasses the API/NOTIFY, so the in-memory cache survives.
for path in [plain, dollar, jobctx] {
sqlx::query("DELETE FROM resource WHERE workspace_id = 'test-workspace' AND path = $1")
.bind(path)
.execute(&db)
.await?;
}
// Job-independent values are cached and still served even under a job_id (a random uuid is
// fine: a cache hit short-circuits before any job lookup). `$HOME` is a non-`$WM_` string,
// so it's not interpolated and stays cacheable.
for path in [plain, dollar] {
let resp = get(
path,
"allow_cache=true&job_id=11111111-1111-4111-8111-111111111111",
)
.await;
assert_eq!(
resp.status(),
200,
"job-independent resource ({path}) must stay cached and be served under a job_id"
);
}
// The `$WM_*` resource was never cached → the (now deleted) row is not found.
let resp = get(jobctx, "allow_cache=true").await;
assert_ne!(
resp.status(),
200,
"resource with a $WM_* contextual variable must not be cached"
);
Ok(())
}
#[cfg(feature = "mcp")]
#[sqlx::test(migrations = "../migrations", fixtures("base", "resources_test"))]
async fn test_mcp_tools(db: Pool<Postgres>) -> anyhow::Result<()> {
@@ -463,3 +463,107 @@ async fn test_auto_parent_resolves_parent_hash(db: Pool<Postgres>) -> anyhow::Re
Ok(())
}
/// Regression test for GHSA-2ppx-66jv-wpw5: a path-scoped token must only see
/// the scripts within its scope when listing, even though the route-level scope
/// check only validates `domain:action`. Before the fix, `list_search` (and
/// `list`) returned `path` + full `content` for every script the underlying
/// user could see, leaking out-of-scope script source to narrowly-scoped tokens.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_list_search_scope_filtering(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/scripts");
// Create two folders and one script in each, as the (super-admin) test user.
for folder in ["allowed", "private"] {
let resp = authed(client().post(format!(
"http://localhost:{port}/api/w/test-workspace/folders/create"
)))
.json(&json!({ "name": folder }))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200, "create folder: {}", resp.text().await?);
}
for (path, content) in [
(
"f/allowed/foo",
"export async function main() { return 'allowed'; }",
),
(
"f/private/bar",
"export async function main() { return 'secret'; }",
),
] {
let resp = authed(client().post(format!("{base}/create")))
.json(&new_script(path, "summary", content))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 201, "create {path}: {}", resp.text().await?);
}
// Helper: GET /list_search with an arbitrary bearer token, returning the set
// of script paths visible to that token.
async fn list_search_paths(port: u16, token: &str) -> Vec<String> {
let resp = client()
.get(format!(
"http://localhost:{port}/api/w/test-workspace/scripts/list_search"
))
.header("Authorization", format!("Bearer {token}"))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200);
resp.json::<Vec<serde_json::Value>>()
.await
.unwrap()
.into_iter()
.map(|s| s["path"].as_str().unwrap().to_string())
.collect()
}
// Insert three tokens for the same super-admin user, differing only by scope.
sqlx::query(
"INSERT INTO token (token_hash, token_prefix, token, email, label, super_admin, scopes) VALUES
(encode(sha256('SCOPED_TOKEN'::bytea), 'hex'), 'SCOPED_TOK', 'SCOPED_TOKEN', 'test@windmill.dev', 'scoped', true, ARRAY['scripts:read:f/allowed/*']),
(encode(sha256('BROAD_TOKEN'::bytea), 'hex'), 'BROAD_TOK', 'BROAD_TOKEN', 'test@windmill.dev', 'broad', true, ARRAY['scripts:read']),
(encode(sha256('TAG_TOKEN'::bytea), 'hex'), 'TAG_TOK', 'TAG_TOKEN', 'test@windmill.dev', 'tag-only', true, ARRAY['if_jobs:filter_tags:default'])",
)
.execute(&db)
.await?;
// Path-scoped token: only sees scripts within `f/allowed/*`.
let scoped = list_search_paths(port, "SCOPED_TOKEN").await;
assert!(
scoped.contains(&"f/allowed/foo".to_string()),
"scoped token should see f/allowed/foo, got: {scoped:?}"
);
assert!(
!scoped.contains(&"f/private/bar".to_string()),
"scoped token must NOT see f/private/bar, got: {scoped:?}"
);
// Broad `scripts:read` token: still sees every RLS-visible script.
let broad = list_search_paths(port, "BROAD_TOKEN").await;
assert!(broad.contains(&"f/allowed/foo".to_string()));
assert!(
broad.contains(&"f/private/bar".to_string()),
"broad scripts:read token should see all scripts, got: {broad:?}"
);
// Tag-filter-only token is not scope-restricted: unchanged, sees all.
let tag_only = list_search_paths(port, "TAG_TOKEN").await;
assert!(tag_only.contains(&"f/allowed/foo".to_string()));
assert!(tag_only.contains(&"f/private/bar".to_string()));
// Unscoped token (no scopes column set): unchanged, sees all.
let unscoped = list_search_paths(port, "SECRET_TOKEN").await;
assert!(unscoped.contains(&"f/allowed/foo".to_string()));
assert!(unscoped.contains(&"f/private/bar".to_string()));
Ok(())
}
@@ -108,12 +108,10 @@ async fn test_variable_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
assert_eq!(secret["value"], serde_json::Value::Null);
// list with path_start filter
let resp = authed(client().get(format!(
"{base}/list?path_start=u/test-user/plain"
)))
.send()
.await
.unwrap();
let resp = authed(client().get(format!("{base}/list?path_start=u/test-user/plain")))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200);
let list = resp.json::<Vec<serde_json::Value>>().await?;
assert_eq!(list.len(), 1);
@@ -252,3 +250,91 @@ async fn test_variable_endpoints(db: Pool<Postgres>) -> anyhow::Result<()> {
Ok(())
}
/// Regression test: the variable-value cache (`get_value?allow_cache=true`) must be
/// identity-scoped. test-user-2 (folder access) warms the cache; test-user-3 (no access)
/// must then be denied rather than served the cached value.
#[sqlx::test(migrations = "../migrations", fixtures("base", "variable_cache_rls"))]
async fn test_variable_value_cache_is_identity_scoped(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let url = format!(
"{}?allow_cache=true",
variable_url(port, "get_value", "f/secret/cache_target_var")
);
let get = |token: &str| {
client()
.get(url.as_str())
.header("Authorization", format!("Bearer {token}"))
};
// test-user-2 has folder access and WARMS the cache.
let resp = get("SECRET_TOKEN_2").send().await?;
assert_eq!(resp.status(), 200);
assert!(resp.text().await?.contains("LEAKED_VAR_SECRET"));
// test-user-3 has no folder access: must miss the cache and be denied (401), not leak.
let resp = get("SECRET_TOKEN_3").send().await?;
assert_eq!(resp.status(), 401);
assert!(!resp.text().await?.contains("LEAKED_VAR_SECRET"));
Ok(())
}
/// Secret variables ARE cached (with their per-read side effects — the EE
/// `variables.decrypt_secret` audit and running-job secret registration — re-run on every
/// hit; that re-emission is not observable in the OSS build since `audit_log` is a no-op).
/// We assert the caching itself: warm the cache, delete the row directly (no API/NOTIFY, so
/// the in-memory cache survives), and re-read with `allow_cache=true` — the value is still
/// returned from cache. A non-secret variable behaves identically (control).
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_variables_are_cached(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/variables");
let plain = "u/test-user/cache_plain_probe";
let secret = "u/test-user/cache_secret_probe";
// Create one non-secret and one secret variable (the secret is stored encrypted).
for (path, value, is_secret) in [
(plain, "PLAIN_PROBE", false),
(secret, "SECRET_PROBE", true),
] {
let resp = authed(client().post(format!("{base}/create")))
.json(
&json!({ "path": path, "value": value, "is_secret": is_secret, "description": "" }),
)
.send()
.await?;
assert_eq!(resp.status(), 201);
}
let read = |path: &str| {
let url = format!("{base}/get_value/{path}?allow_cache=true");
async move { authed(client().get(url)).send().await.unwrap() }
};
// Warm the cache for both.
assert_eq!(read(plain).await.json::<String>().await?, "PLAIN_PROBE");
assert_eq!(read(secret).await.json::<String>().await?, "SECRET_PROBE");
// Delete both rows directly — bypasses the API and its NOTIFY-based invalidation, so
// the in-memory cache survives. A subsequent read can only succeed from cache.
for path in [plain, secret] {
sqlx::query("DELETE FROM variable WHERE workspace_id = 'test-workspace' AND path = $1")
.bind(path)
.execute(&db)
.await?;
}
// Both (secret included) are still served from the cache.
assert_eq!(read(plain).await.json::<String>().await?, "PLAIN_PROBE");
let resp = read(secret).await;
assert_eq!(resp.status(), 200, "secret must still be served from cache");
assert_eq!(resp.json::<String>().await?, "SECRET_PROBE");
Ok(())
}
@@ -0,0 +1,358 @@
/*!
* Integration test for workspace encryption key rotation triggering git sync.
*
* Regression test for windmill-labs/windmill#9344 re-encrypting all secret
* variables on workspace key change must dispatch a git-sync job that carries
* every re-encrypted variable plus the encryption_key entry, so repos with
* Secrets sync enabled receive the new ciphertexts in one commit.
*
* Run with enterprise features:
* ```bash
* cargo test --test workspace_encryption_key_git_sync --features enterprise,private
* ```
*/
use serde_json::json;
use sqlx::{Pool, Postgres};
use std::time::Duration;
#[allow(unused_imports)]
use windmill_test_utils::*;
fn client() -> reqwest::Client {
reqwest::Client::new()
}
fn authed(builder: reqwest::RequestBuilder) -> reqwest::RequestBuilder {
builder.header("Authorization", "Bearer SECRET_TOKEN")
}
#[allow(dead_code)]
async fn create_git_repo_resource(db: &Pool<Postgres>) -> anyhow::Result<()> {
sqlx::query(
r#"
INSERT INTO resource (workspace_id, path, value, resource_type, extra_perms, created_by)
VALUES ('test-workspace', 'u/test-user/test_git_repo', $1::jsonb, 'git_repository', '{}'::jsonb, 'test-user')
ON CONFLICT (workspace_id, path) DO NOTHING
"#,
)
.bind(json!({
"url": "https://github.com/test/test.git",
"branch": "main",
"token": "test-token"
}))
.execute(db)
.await?;
Ok(())
}
#[allow(dead_code)]
async fn create_folder(db: &Pool<Postgres>, name: &str) -> anyhow::Result<()> {
sqlx::query(
r#"
INSERT INTO folder (workspace_id, name, display_name, owners, extra_perms, created_by)
VALUES ('test-workspace', $1, $1, ARRAY['u/test-user'], '{}'::jsonb, 'test-user')
ON CONFLICT (workspace_id, name) DO NOTHING
"#,
)
.bind(name)
.execute(db)
.await?;
Ok(())
}
#[allow(dead_code)]
async fn create_sync_script(db: &Pool<Postgres>, path: &str) -> anyhow::Result<i64> {
let hash: i64 = rand::random::<i64>().unsigned_abs() as i64;
sqlx::query(
r#"
INSERT INTO script (workspace_id, hash, path, summary, description, content,
created_by, language, kind, lock)
VALUES ('test-workspace', $1, $2, 'sync script', '',
'export function main(items: any[]) { return { synced: items.length }; }',
'test-user', 'bun', 'script', '')
"#,
)
.bind(hash)
.bind(path)
.execute(db)
.await?;
Ok(hash)
}
#[allow(dead_code)]
async fn setup_git_sync_config(db: &Pool<Postgres>, sync_script_path: &str) -> anyhow::Result<()> {
// Include Variable + Secret + Key so the encryption rotation has a reason
// to push every re-encrypted variable. Anchor include_path to root so all
// u/... and f/... paths pass the regex filter.
let git_sync_config = json!({
"include_type": ["variable", "secret", "key"],
"include_path": ["**"],
"repositories": [{
"script_path": sync_script_path,
"git_repo_resource_path": "$res:u/test-user/test_git_repo",
"use_individual_branch": false,
"group_by_folder": false
}]
});
sqlx::query!(
"UPDATE workspace_settings SET git_sync = $1 WHERE workspace_id = $2",
git_sync_config,
"test-workspace"
)
.execute(db)
.await?;
Ok(())
}
/// Insert N secret variables, encrypting their values with the workspace's
/// current key so the re-encryption path can decrypt them.
#[allow(dead_code)]
async fn insert_secret_variables(db: &Pool<Postgres>, paths: &[&str]) -> anyhow::Result<()> {
use windmill_common::variables::{build_crypt, encrypt};
let mc = build_crypt(db, "test-workspace").await?;
for path in paths {
let plaintext = format!("secret-value-for-{path}");
let encrypted = encrypt(&mc, &plaintext);
sqlx::query!(
r#"
INSERT INTO variable (workspace_id, path, value, is_secret, description, extra_perms, account)
VALUES ($1, $2, $3, true, '', '{}'::jsonb, NULL)
ON CONFLICT (workspace_id, path) DO UPDATE SET value = EXCLUDED.value
"#,
"test-workspace",
path,
encrypted,
)
.execute(db)
.await?;
}
Ok(())
}
#[derive(Debug)]
#[allow(dead_code)]
struct DeploymentCallbackJob {
id: uuid::Uuid,
args: Option<serde_json::Value>,
}
/// Poll until at least `min_count` deployment-callback jobs exist for the
/// script path, or the timeout elapses. Returns whatever was found.
#[allow(dead_code)]
async fn wait_for_deployment_callbacks(
db: &Pool<Postgres>,
script_path: &str,
min_count: usize,
timeout: Duration,
) -> anyhow::Result<Vec<DeploymentCallbackJob>> {
let deadline = tokio::time::Instant::now() + timeout;
loop {
let rows = sqlx::query_as!(
DeploymentCallbackJob,
r#"
SELECT j.id, j.args
FROM v2_job j
JOIN v2_job_queue q ON j.id = q.id
WHERE j.runnable_path = $1
AND j.kind = 'deploymentcallback'
AND j.workspace_id = 'test-workspace'
ORDER BY j.created_at DESC
"#,
script_path,
)
.fetch_all(db)
.await?;
if rows.len() >= min_count || tokio::time::Instant::now() >= deadline {
return Ok(rows);
}
tokio::time::sleep(Duration::from_millis(100)).await;
}
}
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_encryption_key_rotation_dispatches_batched_git_sync(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
// Setup git sync repo + sync script (folder/path encodes the hub min-version)
create_folder(&db, "28103").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28103/test_sync_script_encryption";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
let secret_paths = [
"u/test-user/secret_a",
"u/test-user/secret_b",
"u/test-user/secret_c",
];
insert_secret_variables(&db, &secret_paths).await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
// 64-char alphanumeric per the route's WORKSPACE_KEY_REGEXP
let new_key = "a".repeat(64);
let resp = authed(client().post(format!("{base}/encryption_key")))
.json(&json!({"new_key": new_key, "skip_reencrypt": false}))
.send()
.await?;
assert_eq!(
resp.status(),
200,
"set_encryption_key failed: {}",
resp.text().await?
);
// The git-sync dispatch runs in a tokio::spawn'd task. Poll up to a few
// seconds for the deployment callback to land in the queue.
let jobs =
wait_for_deployment_callbacks(&db, sync_script_path, 1, Duration::from_secs(5)).await?;
assert_eq!(
jobs.len(),
1,
"expected exactly one batched deployment callback job, got {}",
jobs.len()
);
let job = &jobs[0];
let args = job.args.as_ref().expect("job should have args");
let items = args
.get("items")
.and_then(|v| v.as_array())
.expect("args.items should be a JSON array");
// Expect exactly one job carrying the Key entry + every re-encrypted variable
assert_eq!(
items.len(),
secret_paths.len() + 1,
"expected {} items (key + {} variables) in a single sync job, got {} — items: {:#?}",
secret_paths.len() + 1,
secret_paths.len(),
items.len(),
items
);
let mut variable_paths: Vec<String> = Vec::new();
let mut saw_key = false;
for item in items {
let path_type = item.get("path_type").and_then(|v| v.as_str()).unwrap_or("");
let path = item.get("path").and_then(|v| v.as_str()).unwrap_or("");
match path_type {
"variable" => variable_paths.push(path.to_string()),
"key" => saw_key = true,
other => panic!("unexpected path_type in batch: {other}"),
}
}
assert!(
saw_key,
"expected a path_type=key entry in items: {:#?}",
items
);
variable_paths.sort();
let mut expected: Vec<String> = secret_paths.iter().map(|s| s.to_string()).collect();
expected.sort();
assert_eq!(
variable_paths, expected,
"items array should contain every re-encrypted secret variable"
);
// Secrets sync is enabled (ObjectType::Secret in include_type), so the sync
// script should be invoked with skip_secret=false.
let skip_secret = args
.get("skip_secret")
.and_then(|v| v.as_bool())
.expect("args.skip_secret should be set when batch carries variables");
assert!(
!skip_secret,
"skip_secret should be false when Secret is included in the repo's types"
);
Ok(())
}
/// Regression test for the non-debouncing fallback: a workspace whose sync
/// script predates hub version 28103 must still receive git-sync jobs for the
/// encryption_key entry and every re-encrypted secret. Before the fallback was
/// added, the batch path `continue`d past such repos and queued nothing,
/// silently leaving the repo stale after a key rotation.
#[cfg(all(feature = "enterprise", feature = "private"))]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_encryption_key_rotation_falls_back_without_debouncing(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
initialize_tracing().await;
// Folder/path encodes a hub version BELOW 28103, so
// is_script_meets_min_version(28103) is false → debouncing unsupported.
create_folder(&db, "28000").await?;
create_git_repo_resource(&db).await?;
let sync_script_path = "f/28000/test_sync_script_legacy";
create_sync_script(&db, sync_script_path).await?;
setup_git_sync_config(&db, sync_script_path).await?;
let secret_paths = ["u/test-user/secret_a", "u/test-user/secret_b"];
insert_secret_variables(&db, &secret_paths).await?;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace/workspaces");
let new_key = "b".repeat(64);
let resp = authed(client().post(format!("{base}/encryption_key")))
.json(&json!({"new_key": new_key, "skip_reencrypt": false}))
.send()
.await?;
assert_eq!(
resp.status(),
200,
"set_encryption_key failed: {}",
resp.text().await?
);
// Legacy fallback pushes one job per item (flat args, no `items` array):
// the Key entry + one per re-encrypted variable.
let expected = secret_paths.len() + 1;
let jobs =
wait_for_deployment_callbacks(&db, sync_script_path, expected, Duration::from_secs(5))
.await?;
assert_eq!(
jobs.len(),
expected,
"expected {expected} legacy deployment-callback jobs (key + {} variables), got {} — a repo on an old sync script must not be silently skipped",
secret_paths.len(),
jobs.len()
);
let mut variable_paths: Vec<String> = Vec::new();
let mut saw_key = false;
for job in &jobs {
let args = job.args.as_ref().expect("job should have args");
// Legacy format: flat fields, never an `items` array.
assert!(
args.get("items").is_none(),
"fallback jobs must use the flat legacy format, not an items array: {args:#?}"
);
let path_type = args.get("path_type").and_then(|v| v.as_str()).unwrap_or("");
let path = args.get("path").and_then(|v| v.as_str()).unwrap_or("");
match path_type {
"variable" => variable_paths.push(path.to_string()),
"key" => saw_key = true,
other => panic!("unexpected path_type in fallback job: {other}"),
}
}
assert!(saw_key, "expected a path_type=key fallback job");
variable_paths.sort();
let mut expected_paths: Vec<String> = secret_paths.iter().map(|s| s.to_string()).collect();
expected_paths.sort();
assert_eq!(
variable_paths, expected_paths,
"fallback must queue a job for every re-encrypted secret variable"
);
Ok(())
}
@@ -709,7 +709,9 @@ async fn test_get_copilot_settings_state_reports_instance_ai_fallback_flags(
"resource_path": "u/test-user/openai_instance",
"models": ["gpt-4o-mini"]
}
}
},
"default_model": { "provider": "openai", "model": "gpt-4o-mini" },
"metadata_model": { "provider": "openai", "model": "gpt-4o-mini" }
});
let workspace_ai_config = json!({
"providers": {
@@ -749,6 +751,10 @@ async fn test_get_copilot_settings_state_reports_instance_ai_fallback_flags(
settings["instance_ai_summary"]["providers"][0]["models"][0],
"gpt-4o-mini"
);
assert_eq!(
settings["instance_ai_summary"]["metadata_model"]["model"],
"gpt-4o-mini"
);
sqlx::query("UPDATE workspace_settings SET ai_config = $1 WHERE workspace_id = $2")
.bind(workspace_ai_config)
@@ -191,6 +191,7 @@ async fn get_concurrent_intervals(
script_path_exact: None,
script_hash: None,
created_by: None,
status: None,
success: None,
running: None,
parent_job: None,
+51 -1
View File
@@ -430,7 +430,15 @@ pub fn filter_list_completed_query(
sqlb.and_where_in("created_by", &quoted);
}
}
if let Some(r) = &lq.success {
if let Some(status) = &lq.status {
let status = match status {
windmill_common::jobs::JobStatus::Success => "success",
windmill_common::jobs::JobStatus::Failure => "failure",
windmill_common::jobs::JobStatus::Canceled => "canceled",
windmill_common::jobs::JobStatus::Skipped => "skipped",
};
sqlb.and_where_eq("v2_job_completed.status", quote(status));
} else if let Some(r) = &lq.success {
if *r {
sqlb.and_where_eq("status", "'success'")
.or_where_eq("status", "'skipped'");
@@ -572,6 +580,11 @@ pub fn list_completed_jobs_query(
if lq.completed_before.is_some()
|| lq.completed_after.is_some()
|| lq.success == Some(false)
|| matches!(
lq.status,
Some(windmill_common::jobs::JobStatus::Failure)
| Some(windmill_common::jobs::JobStatus::Canceled)
)
{
"v2_job_completed.completed_at"
} else {
@@ -653,6 +666,7 @@ mod tests {
created_after_queue: None,
completed_after: None,
completed_before: None,
status: None,
success: None,
running: None,
parent_job: None,
@@ -928,6 +942,23 @@ mod tests {
assert!(sql.contains("'failure'"));
}
#[test]
fn test_completed_filter_status_canceled() {
let lq = ListCompletedQuery {
status: Some(windmill_common::jobs::JobStatus::Canceled),
..empty_completed_query()
};
let sqlb = filter_list_completed_query(
SqlBuilder::select_from("v2_job_completed").clone(),
&lq,
"ws",
false,
);
let sql = build_sql(sqlb);
assert!(sql.contains("v2_job_completed.status"));
assert!(sql.contains("'canceled'"));
}
#[test]
fn test_completed_order_by_completed_at() {
let lq = ListCompletedQuery {
@@ -939,6 +970,25 @@ mod tests {
assert!(sql.contains("completed_at"));
}
#[test]
fn test_completed_order_by_completed_at_status_failure_canceled() {
// status=failure|canceled must order by v2_job_completed.completed_at so the
// partial index ix_v2_job_completed_failure_workspace serves both filtering
// and ordering in a single scan.
for status in [
windmill_common::jobs::JobStatus::Failure,
windmill_common::jobs::JobStatus::Canceled,
] {
let lq = ListCompletedQuery { status: Some(status), ..empty_completed_query() };
let sqlb = list_completed_jobs_query("ws", Some(10), 0, &lq, &["id"], false, None);
let sql = build_sql(sqlb);
assert!(
sql.contains("ORDER BY v2_job_completed.completed_at"),
"expected order by completed_at, got: {sql}"
);
}
}
#[test]
fn test_completed_filter_label() {
let lq = ListCompletedQuery {
+4 -1
View File
@@ -19,7 +19,7 @@ use std::collections::HashMap;
use uuid::Uuid;
use windmill_common::{
error,
jobs::{CompletedJob, JobKind, JobTriggerKind, QueuedJob},
jobs::{CompletedJob, JobKind, JobStatus, JobTriggerKind, QueuedJob},
scripts::{ScriptHash, ScriptLang},
utils::now_from_db,
DB,
@@ -142,6 +142,7 @@ pub struct ListCompletedQuery {
pub created_after_queue: Option<chrono::DateTime<chrono::Utc>>,
pub completed_after: Option<chrono::DateTime<chrono::Utc>>,
pub completed_before: Option<chrono::DateTime<chrono::Utc>>,
pub status: Option<JobStatus>,
pub success: Option<bool>,
pub running: Option<bool>,
pub parent_job: Option<String>,
@@ -680,6 +681,7 @@ mod tests {
created_after_queue: None,
completed_after: None,
completed_before: None,
status: None,
success: None,
running: Some(true),
parent_job: None,
@@ -752,6 +754,7 @@ mod tests {
created_after_queue: Some(specific_time),
completed_after: None,
completed_before: None,
status: None,
success: None,
running: None,
parent_job: None,
+2
View File
@@ -13,6 +13,7 @@ default = []
enterprise = ["windmill-common/enterprise"]
private = ["windmill-common/private", "windmill-dep-map/private"]
python = ["dep:windmill-parser-py"]
prometheus = ["dep:prometheus", "windmill-common/prometheus"]
[dependencies]
windmill-common = { workspace = true, default-features = false }
windmill-object-store.workspace = true
@@ -38,4 +39,5 @@ tracing.workspace = true
chrono.workspace = true
lazy_static.workspace = true
tokio.workspace = true
prometheus = { workspace = true, optional = true }
windmill-parser-py = { workspace = true, optional = true }
+141 -31
View File
@@ -9,7 +9,8 @@
use axum::extract::Multipart;
use windmill_api_auth::{
auth::{list_tokens_internal, AuthCache, TruncatedTokenWithEmail},
check_scopes, maybe_refresh_folders, require_owner_of_path, ApiAuthed,
build_scope_path_predicate, check_scopes, maybe_refresh_folders, require_owner_of_path,
ApiAuthed,
};
use windmill_common::{
utils::{BulkDeleteRequest, WithStarredInfoQuery, HTTP_CLIENT},
@@ -275,6 +276,7 @@ async fn list_search_scripts(
#[cfg(not(feature = "enterprise"))]
let n = 10;
let allowed = build_scope_path_predicate(&authed, "scripts", "read");
let rows = sqlx::query_as!(
SearchScript,
"SELECT path, content from script WHERE workspace_id = $1 AND archived = false LIMIT $2",
@@ -284,6 +286,7 @@ async fn list_search_scripts(
.fetch_all(&mut *tx)
.await?
.into_iter()
.filter(|r| allowed(&r.path))
.collect::<Vec<_>>();
tx.commit().await?;
Ok(Json(rows))
@@ -438,9 +441,13 @@ async fn list_scripts(
let sql = sqlb.sql().map_err(|e| Error::internal_err(e.to_string()))?;
let mut tx = user_db.begin(&authed).await?;
let allowed = build_scope_path_predicate(&authed, "scripts", "read");
let rows = sqlx::query_as::<_, ListableScript>(&sql)
.fetch_all(&mut *tx)
.await?;
.await?
.into_iter()
.filter(|r| allowed(&r.path))
.collect::<Vec<_>>();
tx.commit().await?;
Ok(Json(rows))
}
@@ -737,6 +744,9 @@ async fn is_noop_deploy_against_parent(
// caller-intent flag (auto-resolve parent), not script state
auto_parent: _,
labels,
// caller-intent flag (preserve user drafts on CLI/git-sync deploys);
// transient, never persisted, does not change what the script *is*
skip_draft_deletion: _,
} = ns;
if path != &parent.path {
@@ -925,6 +935,9 @@ async fn create_script_internal<'c>(
}
}
let script_path = ns.path.clone();
// Caller-intent: CLI / git-sync deploys ask us to preserve any existing
// user draft at this path instead of wiping it as part of the deploy.
let skip_draft_deletion = ns.skip_draft_deletion.unwrap_or(false);
let hash = ScriptHash(hash_script(&ns));
let authed = maybe_refresh_folders(&ns.path, &w_id, authed, &db).await;
@@ -1357,13 +1370,15 @@ async fn create_script_internal<'c>(
let p_path_opt = parent_hashes_and_perms.as_ref().map(|x| x.p_path.clone());
if let Some(ref p_path) = p_path_opt {
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'script'",
p_path,
&w_id
)
.execute(&mut *tx)
.await?;
if !skip_draft_deletion {
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'script'",
p_path,
&w_id
)
.execute(&mut *tx)
.await?;
}
sqlx::query!(
"UPDATE capture_config SET path = $1 WHERE path = $2 AND workspace_id = $3 AND is_flow IS FALSE",
@@ -1442,7 +1457,7 @@ async fn create_script_internal<'c>(
tx = push_scheduled_job(&db, tx, &schedule, None, None).await?;
}
}
} else {
} else if !skip_draft_deletion {
sqlx::query!(
"DELETE FROM draft WHERE path = $1 AND workspace_id = $2 AND typ = 'script'",
ns.path,
@@ -2084,14 +2099,64 @@ async fn raw_script_by_path_unpinned(
lazy_static::lazy_static! {
static ref DEBUG_RAW_SCRIPT_ENDPOINTS: bool =
std::env::var("DEBUG_RAW_SCRIPT_ENDPOINTS").is_ok();
/// Fallback freshness window (seconds) for [`RAW_SCRIPT_LATEST_HASH_CACHE`].
/// Primary invalidation is event-driven: deploying a script writes a
/// `notify_runnable_version_change` row, and the server's polling-events handler
/// evicts the entry across all replicas (see `main.rs`). This TTL only bounds
/// staleness if that event is missed. Defaults to 60s (matches
/// `DEPLOYED_SCRIPT_HASH_CACHE`). Override with `RAW_SCRIPT_CACHE_TTL_SECONDS`.
static ref RAW_SCRIPT_CACHE_TTL_S: i64 = std::env::var("RAW_SCRIPT_CACHE_TTL_SECONDS")
.ok()
.and_then(|s| s.parse::<i64>().ok())
.filter(|s| *s >= 0)
.unwrap_or(60);
}
lazy_static::lazy_static! {
// Imported-script content, keyed by
// `{ws}:{path}:{importer_cache_key}[:unpinned]:{latest_hash}`. Including the
// imported script's own latest hash makes each entry immutable, so no
// per-entry TTL is needed; staleness is bounded by RAW_SCRIPT_LATEST_HASH_CACHE.
pub static ref RAW_SCRIPT_CACHE: Cache<String, String> = Cache::new(1000);
// `{ws}:{path}` (bare path) -> (latest non-archived hash, unix_ts cached).
// Resolving the imported script's own hash and keying content by it is what
// fixes relative-import staleness for deployed scripts, whose importer hash
// never moves (see #6769). Evicted on deploy by the `notify_runnable_version_change`
// handler in main.rs (cross-replica, within a poll interval); RAW_SCRIPT_CACHE_TTL_S
// is a fallback bound.
pub static ref RAW_SCRIPT_LATEST_HASH_CACHE: Cache<String, (i64, i64)> = Cache::new(1000);
pub static ref CACHE_FOLDERS_PATH: Cache<String, i64> = Cache::new(1000);
}
/// Records a [`RAW_SCRIPT_CACHE`] lookup outcome (`hit` / `expired` / `miss`) to
/// the `raw_script_cache_total` counter when the prometheus feature is enabled.
#[cfg(feature = "prometheus")]
fn record_raw_script_cache(result: &str) {
if let Some(c) = RAW_SCRIPT_CACHE_METRIC.as_ref() {
c.with_label_values(&[result]).inc();
}
}
#[cfg(not(feature = "prometheus"))]
fn record_raw_script_cache(_result: &str) {}
#[cfg(feature = "prometheus")]
lazy_static::lazy_static! {
/// Raw relative-import cache lookups, labeled by `result` (hit/expired/miss).
static ref RAW_SCRIPT_CACHE_METRIC: Option<prometheus::IntCounterVec> =
if windmill_common::METRICS_ENABLED.load(std::sync::atomic::Ordering::Relaxed) {
Some(prometheus::register_int_counter_vec!(
"raw_script_cache_total",
"Raw script relative-import cache lookups by result (hit/expired/miss)",
&["result"]
).unwrap())
} else {
None
};
}
async fn raw_script_by_path_internal(
path: StripPath,
user_db: UserDB,
@@ -2113,23 +2178,10 @@ async fn raw_script_by_path_internal(
}
}
let cache_path = query
.cache_key
.map(|x| format!("{w_id}:{path}:{x}{}", if unpin { ":unpinned" } else { "" }));
if let Some(cache_path) = cache_path.clone() {
let cached_content = RAW_SCRIPT_CACHE.get(&cache_path);
if let Some(cached_content) = cached_content {
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!("Raw script by path request: {} (cached)", path);
}
return Ok(cached_content);
}
}
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!("Raw script by path request: {} (not cached)", path);
}
// Validate + strip the language extension up front so cache keys use the bare
// script path. This matches the `notify_runnable_version_change` event payload
// (which carries the bare path), so a deploy can evict RAW_SCRIPT_LATEST_HASH_CACHE
// by key from the polling-events handler in the server binary.
if !path.ends_with(".py")
&& !path.ends_with(".ts")
&& !path.ends_with(".go")
@@ -2148,6 +2200,52 @@ async fn raw_script_by_path_internal(
.trim_end_matches(".go")
.trim_end_matches(".sh");
// Content cache is keyed by the IMPORTED script's own latest hash, not by the
// importer's runnable hash (`query.cache_key`). The importer hash never moves
// when only an imported script's content changes (relock is in-place — see
// #6769), so keying solely on it served stale content indefinitely. The
// importer + unpin dimensions are kept to preserve per-runnable authorization
// scoping (a content-cache hit skips the authed RLS query, so an entry must
// stay scoped to the runnable that fetched it); the imported latest hash is
// appended for content correctness.
let cache_path_base = query
.cache_key
.as_ref()
.map(|x| format!("{w_id}:{path}:{x}{}", if unpin { ":unpinned" } else { "" }));
// Resolve the imported script's latest hash from RAW_SCRIPT_LATEST_HASH_CACHE
// (keyed by the bare path so the deploy event can evict it). A fresh entry
// serves from the immutable content cache with no DB hit; a stale/absent entry
// falls through to the query below, which refreshes both caches.
let hash_cache_key = format!("{w_id}:{path}");
let (fresh_hash, had_stale_hash) = match RAW_SCRIPT_LATEST_HASH_CACHE.get(&hash_cache_key) {
Some((hash, cached_at))
if chrono::Utc::now().timestamp() - cached_at <= *RAW_SCRIPT_CACHE_TTL_S =>
{
(Some(hash), false)
}
Some(_) => (None, true),
None => (None, false),
};
if let (Some(base), Some(latest_hash)) = (cache_path_base.as_ref(), fresh_hash) {
let content_key = format!("{base}:{latest_hash}");
if let Some(cached_content) = RAW_SCRIPT_CACHE.get(&content_key) {
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!("Raw script by path request: {path} (cached, key={content_key})");
}
record_raw_script_cache("hit");
return Ok(cached_content);
}
}
if cache_path_base.is_some() {
record_raw_script_cache(if had_stale_hash { "expired" } else { "miss" });
}
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!("Raw script by path request: {} (not cached)", path);
}
// folder cache is only useful for python given it needs to recuse over all intermediate folders to find the package.
// When a script exists in a folder, we can cache the fact that the folder exists to avoid extra db calls.
let mut split_path = path.split("/").collect::<Vec<&str>>();
@@ -2180,8 +2278,10 @@ async fn raw_script_by_path_internal(
let mut tx = user_db.begin(&authed).await?;
let content_o = sqlx::query_scalar!(
"SELECT content FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1",
// Fetch the latest non-archived row's hash AND content in one query: the hash
// keys the (immutable) content cache and refreshes RAW_SCRIPT_LATEST_HASH_CACHE.
let row_o = sqlx::query!(
"SELECT hash, content FROM script WHERE path = $1 AND workspace_id = $2 AND archived = false ORDER BY created_at DESC LIMIT 1",
path,
w_id
)
@@ -2189,6 +2289,10 @@ async fn raw_script_by_path_internal(
.warn_after_seconds(5)
.await?;
tx.commit().await?;
let (db_hash, content_o) = match row_o {
Some(r) => (Some(r.hash), Some(r.content)),
None => (None, None),
};
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!(
"Raw script by path request: {} (content: {:?})",
@@ -2252,8 +2356,14 @@ async fn raw_script_by_path_internal(
}
}
if let Some(cache_path) = cache_path {
RAW_SCRIPT_CACHE.insert(cache_path, content.clone());
// content_o was Some, so db_hash is Some too (same row). Refresh the latest-hash
// cache and store the content under the hash-qualified key.
if let Some(db_hash) = db_hash {
RAW_SCRIPT_LATEST_HASH_CACHE
.insert(hash_cache_key, (db_hash, chrono::Utc::now().timestamp()));
if let Some(base) = cache_path_base {
RAW_SCRIPT_CACHE.insert(format!("{base}:{db_hash}"), content.clone());
}
}
if *DEBUG_RAW_SCRIPT_ENDPOINTS {
tracing::warn!("Raw script by path request: {} (content response)", path);
+61 -9
View File
@@ -6,7 +6,10 @@
* LICENSE-AGPL for a copy of the license.
*/
use std::{collections::HashMap, time::Duration};
use std::{
collections::{BTreeSet, HashMap},
time::Duration,
};
#[cfg(feature = "parquet")]
mod audit_logs_s3;
@@ -47,6 +50,7 @@ use windmill_common::secret_backend::{
AwsSecretsManagerSettings, AzureKeyVaultSettings, SecretMigrationReport, VaultSettings,
};
use windmill_common::{
auth::is_super_admin_email,
ee_oss::{get_license_plan, LicensePlan},
email_oss::send_email_plain_text,
error::{self, JsonResult, Result},
@@ -1135,6 +1139,8 @@ struct CustomInstanceDb {
success: bool,
error: Option<String>,
tag: Option<String>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
used_by_workspaces: Vec<String>,
}
#[derive(Deserialize, Debug, Serialize, Default)]
@@ -1154,7 +1160,7 @@ struct CustomInstanceDbLogs {
}
async fn list_custom_instance_pg_databases(
_authed: ApiAuthed,
authed: ApiAuthed,
Extension(db): Extension<DB>,
) -> JsonResult<HashMap<String, CustomInstanceDb>> {
let result = sqlx::query_scalar!(
@@ -1163,12 +1169,57 @@ async fn list_custom_instance_pg_databases(
.fetch_one(&db)
.await?
.ok_or_else(|| error::Error::ExecutionErr("Couldn't find custom_instance_pg_databases".to_string()))?;
let result = serde_json::from_value(result).map_err(|e| {
error::Error::ExecutionErr(format!(
"couldn't parse custom_instance_pg_databases.databases : {}",
e.to_string()
))
})?;
let mut result: HashMap<String, CustomInstanceDb> =
serde_json::from_value(result).map_err(|e| {
error::Error::ExecutionErr(format!(
"couldn't parse custom_instance_pg_databases.databases : {}",
e.to_string()
))
})?;
if is_super_admin_email(&db, &authed.email).await? {
// Enrich each database with the list of workspaces referencing it through
// either a ducklake catalog or a datatable database whose resource_type is
// 'instance'. Not stored in DB to avoid drift.
let usages = sqlx::query!(
r#"
SELECT ws.workspace_id AS "workspace_id!", entry->'catalog'->>'resource_path' AS dbname
FROM workspace_settings ws
CROSS JOIN LATERAL jsonb_each(
CASE WHEN jsonb_typeof(ws.ducklake->'ducklakes') = 'object'
THEN ws.ducklake->'ducklakes'
ELSE '{}'::jsonb END
) AS dl(k, entry)
WHERE entry->'catalog'->>'resource_type' = 'instance'
AND entry->'catalog'->>'resource_path' IS NOT NULL
UNION ALL
SELECT ws.workspace_id AS "workspace_id!", entry->'database'->>'resource_path' AS dbname
FROM workspace_settings ws
CROSS JOIN LATERAL jsonb_each(
CASE WHEN jsonb_typeof(ws.datatable->'datatables') = 'object'
THEN ws.datatable->'datatables'
ELSE '{}'::jsonb END
) AS dt(k, entry)
WHERE entry->'database'->>'resource_type' = 'instance'
AND entry->'database'->>'resource_path' IS NOT NULL
"#,
)
.fetch_all(&db)
.await?;
let mut by_db: HashMap<String, BTreeSet<String>> = HashMap::new();
for row in usages {
if let Some(dbname) = row.dbname {
by_db.entry(dbname).or_default().insert(row.workspace_id);
}
}
for (dbname, entry) in result.iter_mut() {
if let Some(workspaces) = by_db.remove(dbname) {
entry.used_by_workspaces = workspaces.into_iter().collect();
}
}
}
return Ok(Json(result));
}
@@ -1196,7 +1247,8 @@ async fn setup_custom_instance_pg_database(
let result = setup_custom_instance_pg_database_inner(authed, &db, &dbname, &mut logs).await;
let success = result.is_ok();
let error = result.err().map(|e| e.to_string());
let status = CustomInstanceDb { logs, success, error, tag: body.tag };
let status =
CustomInstanceDb { logs, success, error, tag: body.tag, used_by_workspaces: vec![] };
let status_json = serde_json::to_value(&status).map_err(to_anyhow)?;
// Save that the database was setup successfully
sqlx::query!(
+22 -5
View File
@@ -1964,7 +1964,8 @@ async fn login(
windmill_common::login_rate_limit::record_login_failure(&email);
Err(Error::BadRequest("Invalid login".to_string()))
} else {
let token = create_session_token(&email, super_admin, &mut tx, cookies).await?;
let token =
create_session_token(&email, super_admin, None, false, &mut tx, cookies).await?;
let audit_author = AuditAuthor {
email: email.clone(),
@@ -2036,7 +2037,15 @@ async fn refresh_token(
.await?
.unwrap_or(false);
let new_token = create_session_token(&authed.email, super_admin, &mut tx, cookies).await?;
let new_token = create_session_token(
&authed.email,
super_admin,
authed.scopes.as_deref(),
authed.read_only,
&mut tx,
cookies,
)
.await?;
audit_log(
&mut *tx,
@@ -2066,6 +2075,8 @@ lazy_static::lazy_static! {
pub async fn create_session_token<'c>(
email: &str,
super_admin: bool,
scopes: Option<&[String]>,
read_only: bool,
tx: &mut sqlx::Transaction<'c, sqlx::Postgres>,
cookies: Cookies,
) -> Result<String> {
@@ -2108,15 +2119,17 @@ pub async fn create_session_token<'c>(
sqlx::query!(
"INSERT INTO token
(token_hash, token_prefix, token, email, label, expiration, super_admin)
VALUES ($1, $2, $3, $4, $5, now() + ($6 || ' seconds')::interval, $7)",
(token_hash, token_prefix, token, email, label, expiration, super_admin, scopes, read_only)
VALUES ($1, $2, $3, $4, $5, now() + ($6 || ' seconds')::interval, $7, $8, $9)",
t_hash,
t_prefix,
plaintext as Option<&str>,
email,
"session",
&MAX_SESSION_VALIDITY_SECONDS.to_string(),
super_admin
super_admin,
scopes,
read_only,
)
.execute(&mut **tx)
.await?;
@@ -2146,6 +2159,8 @@ async fn create_token(
) -> Result<(StatusCode, String)> {
check_token_create_rate_limit(&authed.username)?;
windmill_api_auth::ensure_scopes_within_caller(&authed, token_config.scopes.as_deref())?;
let mut tx = db.begin().await?;
let token = create_token_internal(&mut *tx, &db, &authed, token_config).await?;
@@ -2353,6 +2368,8 @@ async fn update_token_scopes(
Path(token_prefix): Path<String>,
Json(req): Json<UpdateTokenScopesRequest>,
) -> Result<String> {
windmill_api_auth::ensure_scopes_within_caller(&authed, req.scopes.as_deref())?;
let mut tx = db.begin().await?;
let updated: Option<String> = sqlx::query_scalar!(
@@ -55,7 +55,10 @@ use windmill_common::{
use windmill_dep_map::scoped_dependency_map::{
DependencyDependent, DependencyMap, ScopedDependencyMap,
};
use windmill_git_sync::{handle_deployment_metadata, handle_fork_branch_creation, DeployedObject};
use windmill_git_sync::{
handle_deployment_metadata, handle_deployment_metadata_batch, handle_fork_branch_creation,
DeployedObject,
};
use windmill_types::s3::LargeFileStorage;
use hyper::StatusCode;
@@ -337,6 +340,8 @@ pub struct InstanceAISummary {
#[serde(skip_serializing_if = "Option::is_none")]
pub default_model: Option<InstanceAIModelSummary>,
#[serde(skip_serializing_if = "Option::is_none")]
pub metadata_model: Option<InstanceAIModelSummary>,
#[serde(skip_serializing_if = "Option::is_none")]
pub code_completion_model: Option<InstanceAIModelSummary>,
}
@@ -822,6 +827,7 @@ pub fn build_instance_ai_summary(config: Option<&serde_json::Value>) -> Option<I
Some(InstanceAISummary {
providers: provider_summaries,
default_model: extract_instance_ai_model_summary(config, "default_model"),
metadata_model: extract_instance_ai_model_summary(config, "metadata_model"),
code_completion_model: extract_instance_ai_model_summary(config, "code_completion_model"),
})
}
@@ -3403,6 +3409,7 @@ async fn set_encryption_key(
.execute(&mut *tx)
.await?;
let mut reencrypted_secret_paths: Vec<String> = Vec::new();
if !request.skip_reencrypt.unwrap_or(false) {
// Build the new cipher directly from the key string, since the transaction
// hasn't committed yet and build_crypt() would read the old key from the pool.
@@ -3448,6 +3455,7 @@ async fn set_encryption_key(
)
.execute(&mut *tx)
.await?;
reencrypted_secret_paths.push(variable.path);
}
}
@@ -3456,16 +3464,23 @@ async fn set_encryption_key(
// Invalidate the cache only after the transaction has committed
WORKSPACE_CRYPT_CACHE.remove(w_id.as_str());
// Trigger git sync for encryption key changes
handle_deployment_metadata(
// Build the batch: one event for the encryption key itself plus one per
// re-encrypted secret variable. The batch entrypoint dispatches a single
// git-sync job per repo carrying all items, so repos with Secrets sync
// enabled receive the new ciphertexts in one commit.
let mut batch: Vec<DeployedObject> = Vec::with_capacity(reencrypted_secret_paths.len() + 1);
batch.push(DeployedObject::Key { key_type: "encryption_key".to_string() });
for path in reencrypted_secret_paths {
batch.push(DeployedObject::Variable { path: path.clone(), parent_path: Some(path) });
}
handle_deployment_metadata_batch(
&authed.email,
&authed.username,
&db,
&w_id,
windmill_git_sync::DeployedObject::Key { key_type: "encryption_key".to_string() },
batch,
Some("Encryption key updated".to_string()),
false,
None,
)
.await?;
+1 -1
View File
@@ -19,7 +19,7 @@ enterprise_saml = ["dep:samael", "dep:libxml"]
benchmark = []
embedding = ["windmill-api-embeddings/embedding"]
parquet = ["dep:datafusion", "windmill-common/parquet", "windmill-object-store/parquet", "windmill-worker?/parquet", "windmill-api-users/parquet", "windmill-api-settings/parquet", "windmill-api-workspaces/parquet", "dep:aws-sigv4", "dep:aws-sdk-config"]
prometheus = ["windmill-common/prometheus", "windmill-queue/prometheus", "dep:prometheus", "windmill-worker?/prometheus"]
prometheus = ["windmill-common/prometheus", "windmill-queue/prometheus", "dep:prometheus", "windmill-worker?/prometheus", "windmill-api-scripts/prometheus"]
openidconnect = ["dep:openidconnect", "windmill-common/openidconnect", "windmill-store/openidconnect"]
tantivy = ["dep:windmill-indexer"]
kafka = ["dep:windmill-trigger-kafka", "windmill-store/kafka"]
+96 -1
View File
@@ -1,7 +1,7 @@
{
"openapi": "3.0.3",
"info": {
"version": "1.708.0",
"version": "1.713.1",
"title": "Windmill API",
"contact": {
"name": "Windmill Team",
@@ -15229,6 +15229,10 @@
},
"deployment_message": {
"type": "string"
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this flow does not delete an existing user draft at the same path."
}
}
}
@@ -15295,6 +15299,10 @@
"properties": {
"deployment_message": {
"type": "string"
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this flow does not delete an existing user draft at the same path."
}
}
}
@@ -16069,6 +16077,10 @@
"items": {
"type": "string"
}
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this app does not delete an existing user draft at the same path."
}
},
"required": [
@@ -16146,6 +16158,10 @@
"items": {
"type": "string"
}
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this app does not delete an existing user draft at the same path."
}
},
"required": [
@@ -16679,6 +16695,10 @@
"items": {
"type": "string"
}
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this app does not delete an existing user draft at the same path."
}
}
}
@@ -16750,6 +16770,10 @@
"items": {
"type": "string"
}
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this app does not delete an existing user draft at the same path."
}
}
},
@@ -28210,6 +28234,52 @@
}
}
},
"/workers/workspace_fairness_events": {
"get": {
"summary": "list last 100 workspace-fairness cap/uncap events (cloud-only)",
"operationId": "getWorkspaceFairnessEvents",
"tags": [
"worker"
],
"responses": {
"200": {
"description": "workspace fairness events (empty on non-cloud)",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"type": "object",
"properties": {
"timestamp": {
"type": "string",
"format": "date-time"
},
"operation": {
"type": "string"
},
"workspace_id": {
"type": "string",
"nullable": true
},
"parameters": {
"type": "object",
"nullable": true,
"additionalProperties": true
}
},
"required": [
"timestamp",
"operation"
]
}
}
}
}
}
}
}
},
"/configs/list_worker_groups": {
"get": {
"summary": "list worker groups",
@@ -33130,6 +33200,10 @@
"type": "boolean",
"description": "If true, all steps run on the same worker for better performance"
},
"preserve_step_tags": {
"type": "boolean",
"description": "If true and the flow runs on a custom worker tag, steps that declare their own non-empty tag run on it instead of inheriting the flow tag. Steps without their own tag still inherit the flow tag."
},
"concurrent_limit": {
"type": "number",
"description": "Maximum number of concurrent executions of this flow"
@@ -35312,6 +35386,9 @@
"default_model": {
"$ref": "#/components/schemas/AIProviderModel"
},
"metadata_model": {
"$ref": "#/components/schemas/AIProviderModel"
},
"code_completion_model": {
"$ref": "#/components/schemas/AIProviderModel"
},
@@ -35361,6 +35438,9 @@
"default_model": {
"$ref": "#/components/schemas/AIProviderModel"
},
"metadata_model": {
"$ref": "#/components/schemas/AIProviderModel"
},
"code_completion_model": {
"$ref": "#/components/schemas/AIProviderModel"
}
@@ -35837,6 +35917,10 @@
"items": {
"type": "string"
}
},
"skip_draft_deletion": {
"type": "boolean",
"description": "When true (set by the CLI / git sync), deploying this script does not delete an existing user draft at the same path."
}
},
"required": [
@@ -40538,6 +40622,13 @@
},
"tag": {
"$ref": "#/components/schemas/CustomInstanceDbTag"
},
"used_by_workspaces": {
"type": "array",
"items": {
"type": "string"
},
"description": "Workspaces that reference this database via a ducklake catalog or datatable database with resource_type 'instance'. Computed at request time, not persisted."
}
}
},
@@ -46176,6 +46267,10 @@
"type": "boolean",
"description": "If true, all steps run on the same worker for better performance"
},
"preserve_step_tags": {
"type": "boolean",
"description": "If true and the flow runs on a custom worker tag, steps that declare their own non-empty tag run on it instead of inheriting the flow tag. Steps without their own tag still inherit the flow tag."
},
"concurrent_limit": {
"type": "number",
"description": "Maximum number of concurrent executions of this flow"

Some files were not shown because too many files have changed in this diff Show More