* oauth: add ServiceNow provider; make per-instance OAuth registry-driven
ServiceNow's OAuth endpoints are per-instance
(https://<instance>.service-now.com/oauth_auth.do + /oauth_token.do), like
Snowflake's. Rather than add another bespoke special-case, generalize:
a registry entry may carry a `connect_config_template` (label/placeholder/
help_url + {instance}-templated auth_url/token_url + req_body_auth +
optional extra_params_key/strip_suffix). The instance-settings UI renders
one generic instance-name input for any such provider and substitutes
{instance} to build the per-client connect_config — a new per-instance
provider needs only a JSON entry, no frontend code.
- oauth_connect.json: servicenow + snowflake_oauth now carry a
connect_config_template (snowflake keeps its account_identifier
extra_params key for backward compatibility).
- windmill-oauth: add the ConnectConfigTemplate struct (frontend-only
metadata; the backend's existing connect_config override resolves the
concrete URLs generically — no other backend change).
- AuthSettings/InstanceSettings: replace the Snowflake + ServiceNow
special-cases with one registry-driven path (instanceInputs map,
setupTemplatedOauthUrls, loadInstanceInputs); per-instance providers are
derived from the registry for the builtins list + dropdown.
Pairs with windmill-integrations#139 (ServiceNow hub integration).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: point ee-repo-ref at servicenow-oauth EE branch (revert at merge)
Temporary CI pointer so check_ee_full / cargo_test build against the EE
slack-literal fix (windmill-ee-private#602). Revert to a pinned SHA once
that EE PR is merged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): early stop can include the stopping step's result in the raised error
When a step uses Early Stop with "Raise an error message if stopped", the
flow result was entirely replaced with a static error object
({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the
stopping step's own output. This made it impossible to stop+fail a flow
while preserving the data the step produced (e.g. an API that returns
HTTP 200 with a userErrors payload).
Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on
the raise-error path, the raised payload becomes
{"error": {...}, "result": <step result>} instead of dropping the result.
Default is false, so existing behavior is unchanged. The option is threaded
through the worker's stop-after-if handling (including stop_after_all_iters_if
for loops/branchall) and exposed in the flow editor's Early Stop panel.
Fixes WIN-2012
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover early-stop error_include_result payload shaping
Add a regression test asserting that a step using Early Stop with a raised
error message and error_include_result=true fails the flow while preserving
the step output as {"error": {..}, "result": <step result>}, and that with
the flag off the result is the bare {"error": {..}} object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(flows): nest early-stop step result inside the error object
Embed the stopping step's result under `error.result` rather than as a
top-level sibling of `error`. This keeps the flow result shape as
`{ "error": { .. } }` — identical to a normal error — so consumers that
key off the top-level shape (single `error` key) keep working, while the
data is still preserved for those that look inside the error object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): always include the stopping step's result in early-stop errors
Drop the opt-in `error_include_result` gate. Since the step result is nested
inside the error object (`error.result`), the top-level result shape stays
`{ "error": .. }` — identical to a normal error — so consumers that detect or
parse failures by the top-level shape are unaffected. Gating it added schema
surface, plumbing, and a UI toggle for no real compatibility benefit.
Now, whenever a step early-stops with a raised error message, the flow fails
and the raised error embeds the stopping step's own result under
`error.result` (aggregated iteration results for loops/branchall). This
reverts the `StopAfterIf.error_include_result` field, its threading, the
OpenAPI/generated-client surface, and the editor toggle; the "Raise an error
message" tooltip now notes that the step result is included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): gate early-stop result inclusion behind opt-in flag
Re-introduce the per-step `error_include_result` flag (default off) instead
of always embedding the step result. Although nesting the result under
`error.result` keeps the result *shape* backward-compatible, it does not
address data exposure: a failed flow's result is propagated to synchronous
webhook callers, the flow's failure module, and the workspace/global error
handler (commonly a Slack/email/outbound-webhook notifier). Always including
the step output would surface previously-redacted intermediate data to all of
those sinks for every existing error-stop flow.
Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and
only embeds `error.result` when the flow author explicitly opts in, matching
the original issue's intent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(flows): omit error_include_result when false; refresh generated prompts
- Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result`
so serialized flows are byte-identical when the flag is off. Fixes the
`flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing
flows.
- Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts`
for the new OpenFlow `error_include_result` property. Fixes check-freshness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover error_include_result for the loop "stop after all iters" path
Add a regression test for the stop_after_all_iters_if branch, where `nresult`
already holds the aggregated iteration results — confirming `error.result`
carries each iteration's output (distinct from the per-step fallback path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: resolve workspace imports via /f/,/u/ not $f/,$u/ aliases
Keep the CLI managed tsconfig.wmill.json / `refresh tsconfig` / Deno
import-map QoL from #9378, but re-key it on the existing /f/,/u/ workspace
paths instead of the new $f/,$u/ specifiers. Verified /f/,/u/ resolves in
tsc, Bun, Deno, the in-app ATA editor, and the worker, so the $-prefixed
alias added no value. Drop the $f/,$u/ handling from the parser, dep-map,
deno_executor, bun loaders, ATA, relative_imports and monaco paths; revert
the windmill-parser-wasm-ts bump (1.714.0 -> 1.695.0). Also fold in the
cli/package-lock.json sync for the already-committed pg-gateway dependency.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: drop duplicate relative-path check and restore rustfmt formatting
Follow-up cleanups to the previous commit's full-file reverts, which
restored pre-#9378 state that main had since improved:
- relative_imports.ts: remove the redundant duplicate d.startsWith('/')
(pre-#9378 had it; #9378 had repurposed that line, so main has no dup).
- windmill-parser-ts/src/lib.rs: restore the multi-line new_source_file(...)
formatting required by backend/rustfmt.toml (the single-line revert would
fail `cargo fmt --check`). Now differs from main only by the $f//$u/ removal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When a dependency job for an app is triggered by a relative/workspace
import (e.g. an imported script was updated), handle_app_dependency_job
re-appended the version captured at job-creation time to the versions
array. On a git-sync/CLI push that deploys both the imported script and
the importing app in the same batch, the script's dependency job
snapshots the app's old version; the app push then creates a newer
version (uploading its bundle against that new version); finally the
relock runs and re-appends the old version, making it latest again.
For raw apps this is fatal: bundle_secret is computed from the latest
version, so the served HTML requests /apps_u/get_data/v/<secret>.{js,css}
for a version that has no stored bundle -> 404 and a white screen.
Manually redeploying fixes it until the next merge re-triggers the revert.
Two changes:
- Re-query the current latest version to relock (mirrors the flow
dependency handler, #8673), so we don't lock a stale snapshot.
- Guard the re-publish append with `versions[array_upper(...)] = $1` so
it is a single atomic, never-demoting statement: it can only re-append
the version that is already latest, never revert to an older one. A
relock never creates a new app_version, so there is never a version to
legitimately promote here.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: support $f/ and $u/ import path aliases for scripts
$f/ and $u/ are local-friendly aliases for the absolute workspace
import paths /f/ and /u/. Unlike the /-prefixed form (which local tools
treat as a filesystem-root path), the $-prefixed form is a bare specifier
that can be remapped via tsconfig paths / Deno import maps, so the same
import resolves on the Windmill worker and in a local editor.
- worker: recognize $f//$u/ in the Deno import map and both Bun loaders
- dep-map/parser: normalize $f/->f/, $u/->u/ for lockgen + dep tracking
- cli: emit $f/$u path aliases in generated tsconfig.json / deno.json
- frontend: ATA + Monaco paths resolve $f//$u/ type hints in the editor
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): split generated tsconfig into managed + user file with refresh command
Mirror the AGENTS.cli.md/AGENTS.md prompts model for the IDE tsconfig so the
recommended settings can evolve without ever clobbering user customizations:
- tsconfig.wmill.json: wmill-managed, always refreshed, holds recommended
compilerOptions incl. the $f/$u path aliases (Deno: import_map.wmill.json)
- tsconfig.json: user-owned, created once, just extends the managed file;
warn (never auto-edit) when an existing one doesn't reference it
- add 'wmill refresh tsconfig'; init generates it unconditionally (no longer
gated behind resource-type namespace / a bound workspace)
- regenerate CLI guidance docs for the new subcommand
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): address PR review on $f/ tsconfig generation
- handle existing deno.jsonc so we don't shadow it with a new deno.json
(P1 identified by cubic)
- fix the bun-types hint that pointed users at the managed do-not-edit
tsconfig.wmill.json; tell them to install + re-run 'wmill refresh tsconfig'
- document the .ts-extension-only local-resolution limitation (cross-flavor
.bun.ts/.deno.ts/.fetch.ts scripts won't resolve in a local editor)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): warn when a project's tsconfig isn't wired to tsconfig.wmill.json
Mirror the prompts freshness check for the managed tsconfig so users with an
existing setup actually discover they're missing $f//$u/ resolution:
- embed a version hash in tsconfig.wmill.json (excludes the env-dependent
bun-types 'types' entry so it doesn't false-positive)
- add warnIfTsconfigStale to the main.ts freshness hook, gated identically to
the prompts check (skips init/refresh/help/version). When a tsconfig.json
exists it warns one line (stderr) if the managed file is missing, not
referenced via extends, or out of date; silent for non-TS projects.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cli): make tsconfig setup equivalent to prompts (auto-wire + stale-only)
Unify the two managed-file systems so they behave identically:
- auto-wire an existing unlinked tsconfig.json/deno.json on init/refresh
(add extends / importMap; merge into an array extends), instead of only
warning. Parses JSON and falls back to a warning when it can't round-trip
(JSONC comments, or a conflicting deno imports/importMap) — never corrupts.
- narrow warnIfTsconfigStale to stale-only, gated on the managed file
existing, exactly like warnIfPromptsStale: it no longer nags about a
missing or unlinked tsconfig.json, so a deliberately-custom/unlinked setup
stays silent and a not-yet-initialized project isn't bothered.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): place tsconfig.wmill.json first in extends to preserve user base config
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cli): migrate legacy tsconfig and require consent for custom configs
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cli): align prompts wiring to the same consent model
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): bump windmill-parser-wasm-ts to 1.714.0 for $f/ $u/ aliases
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(worker): resolve $f/ and $u/ in deno lock generation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: narrow relative-imports lock-gen guard to deno import-map failure
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(cli): sync bun.lock with windmill-parser-wasm-ts 1.714.0
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): warn when a custom tsconfig's paths would shadow $f/ $u/ aliases
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The variable and resource value caches (backing
`GET /api/w/{w}/variables/get_value/{path}?allow_cache=true` and
`.../resources/get_value_interpolated/{path}?allow_cache=true`) are consulted
before the per-folder RLS query and store the already-decrypted value. The
resource cache was keyed only by `workspace:path` with no caller identity, so a
cache entry warmed by a privileged peer using `allow_cache=true` could be
returned to a caller with no access to the resource's folder on a cache hit
within the 30s TTL — leaking another folder's decrypted secrets.
Scope both caches to the caller's full authorization identity. The key is now
`auth_identity(authed):workspace:path`, where `auth_identity` is a SHA-256 of the
caller's effective authorization context (email, username, is_admin, is_operator,
sorted groups, sorted folders, sorted scopes) — mirroring
`job_read_access_cache_key`. Email alone is insufficient: the same email can
resolve to different effective permissions via job/owner-scoped tokens, so a
lower-privilege context must not reuse a higher-privilege context's entry.
Job-context resource interpolation is handled correctly: only `$WM_*` contextual
variables are resolved (and only when a `job_id` is present). The interpolation
reports whether the value contains a `$WM_*` placeholder
(`transform_json_value_tracked` + an `AtomicBool`). A value containing one is
job-dependent — even on a no-job read where it's left unresolved — and is never
cached (so a later job read never gets a stale placeholder or another job's
context). Any value without a `$WM_*` placeholder is job-independent and cached
under the identity key, shared across job contexts, so reads carrying a `job_id`
still hit the cache.
BEHAVIOR CHANGE: custom workspace environment variables are no longer interpolated
into resource values via `$NAME` (this was undocumented and prevented caching of
any `$`-prefixed value). Custom envs remain available to scripts/workers as before.
Built-in `$WM_*` contextual variables in resource values are unchanged.
The variable cache previously wrote with an identity-scoped key but read with the
unscoped key, so it never hit (a latent functional bug that happened to be safe).
Aligning the read path enables the cache and makes it identity-scoped by
construction. Secret variables are cached too, but the entry carries the
`is_secret` flag so a cache hit re-runs the per-read side effects a secret read
performs — the EE `variables.decrypt_secret` audit and running-job secret
registration (factored into `audit_decrypt_secret`, shared by both paths).
The unused `invalidate_{variable,resource}_cache` helpers can no longer target
identity-scoped entries; documented the constraint and refreshed the stale
key-format docs on the cache statics.
Tests:
- integration regression for both caches: a folder-scoped user warms the cache via
allow_cache=true, then a user without folder access is denied (401) and never
receives the cached value.
- integration regression that variables (secret included) are served from cache.
- integration regression for job context: plain and non-`$WM_` `$`-string resources
stay cached and are served under a job_id, while a `$WM_*` resource (warmed without
a job_id) is not cached.
- unit tests for `auth_identity`.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): clone user_db for oauth2 refresh and drop advisory ids from comments
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(mcp): disable redirects on MCP client to prevent SSRF bypass
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A token scoped to a single script or flow path (e.g.
`scripts:read:f/allowed/*`) could call `GET .../scripts/list_search` (or
`/list`) and receive `path` + full `content` for every script the
underlying user could see — likewise `flows/list_search` leaked the full
flow `value`. Route-level scope checks only validate `domain:action`, and
the listing handlers did no per-row scope filtering, leaking out-of-scope
source/definitions to narrowly-scoped tokens.
Apply `build_scope_path_predicate` (added in #9302 for resources/variables)
to `list_search_scripts`, `list_scripts`, `list_search_flows`, and
`list_flows`, mirroring the resources/variables fix exactly. Unscoped
tokens and tokens whose only scopes are `if_jobs:filter_tags:*` are
unaffected.
Adds integration regression tests (scripts + flows) covering: path-scoped
token sees only in-scope paths, broad `*:read` token still sees all
RLS-visible items, tag-filter-only and unscoped tokens unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): authorize single-job read endpoints by job/flow visibility
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(jobs): share read links + cached access checks for run visibility
- Cache the job read-access RLS probe (size-bounded LRU keyed by the caller's
authz-relevant identity + job id; no TTL since job-side inputs are immutable).
- Inherit visibility along the full parent_job chain so any flow you can see lets
you read its (deeply nested) steps.
- Share read links: GET /jobs/job_view_token/{id} mints a stateless
HMAC(workspace_key, job_id) token (only if the caller can read the job); the
token grants an authenticated member read of that job and its flow subtree via a
?view_token query param or X-View-Token header. Run page gains a Share button and
honors a ?view_token link.
- Denied-but-existing reads now return 403 with guidance to request a share link
(vs 404 for non-existent), and the run page renders that case with instructions.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(jobs): address PR review — scope-tag check on mint, constant-time view-token verify
- P1 (Codex): get_job_view_token now enforces the caller's if_jobs:filter_tags
scope before minting, so a tag-scoped token can't mint a transferable link for a
job outside its tags. Adds a scoped-token regression test (allowed + denied).
- Constant-time view-token verification (HmacSha256::verify_slice) instead of
comparing hex strings (Claude/Pi nit).
- get_completed_job_result: an authed reader passing an invalid suspended-secret
triple now falls through to the normal visibility gate instead of erroring out
(Claude nit); unauthenticated callers still rejected.
- Length-prefix the read-access cache key fields so no input values can collide.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(api): add job_view_token to openapi spec; use generated client in run page
Addresses Codex review nit: the new GET /jobs/job_view_token/{id} endpoint was
missing from openapi.yaml (the source the frontend client is generated from). Adds
the path + operationId getJobViewToken, and switches the run page's Share button
from a raw fetch to JobService.getJobViewToken.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): carry view_token on share-link downloads
Addresses Codex review: download actions bypass the request interceptor that adds
X-View-Token (downloadViaClient uses raw fetch; cookie-mode downloads use plain
hrefs), so a share-link viewer got 403 downloading logs/results/args. Append the
view_token query param to the job download paths (result/logs/args/flow-all-logs)
via a new appendViewToken() helper, covering both client-fetch and href modes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(jobs): enforce tag scope in require_job_read_access (view-token use side)
Addresses Codex P1: the view_token use-side bypassed if_jobs:filter_tags on
handlers that don't tag-filter their data query (result_by_id,
get_flow_job_debug_info, get_otel_traces) — a tag-scoped token could use someone
else's valid share token to read out-of-scope job data. Move the tag-scope check
into require_job_read_access (runs before any created_by/view_token/RLS grant), so
it applies uniformly to every gated handler; removes the now-redundant explicit
check in get_job_view_token. Adds a use-side regression test (scoped token + valid
out-of-scope view_token denied on otel/result_by_id; in-scope still allowed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(frontend): include workspace in share read link
Addresses Codex P1: the copied share URL omitted the workspace. The token is
signed with the run's workspace key and the logged layout only switches
$workspaceStore when the URL carries workspace=, so a recipient whose persisted
active workspace differs would open the link against the wrong workspace and the
token would fail validation. Pin workspace= alongside view_token in the link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(jobs): authorize get_result_maybe get_started branch for queued jobs
Addresses Codex P1: get_completed_job_result_maybe only gated when a completed row
existed; with ?get_started=true a non-reader reached the fallback branch and got
started:true for a running private job. Now fetches created_by and authorizes
(created_by/view_token/RLS, or anonymous for unauth) before disclosing
running-state; a non-existent job still returns started:false (leaks nothing).
Adds a regression test with a queued (no completed row) private job.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Zoom URL-validation challenge handler in `handle_challenge_request`
would HMAC-sign any arbitrary `plainToken` and return the result. Since
Zoom webhook verification checks `HMAC-SHA256(secret, "v0:{ts}:{body}")`,
an attacker could craft a `plainToken` in that format to obtain a valid
signature for a forged body, bypassing authentication on a later request.
Unlike the Twitch handler, the Zoom handler verifies no signature on the
challenge request (Zoom's protocol does not include one). Reject any
`plainToken` containing `:` or longer than 128 chars: legitimate Zoom
validation tokens are short random hex strings that never contain colons,
while the exploit requires the colon-bearing `v0:{ts}:{body}` format.
Fixes WIN-2008
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#9393 added `--compile-bytecode` to the uv pip install run inside the
python download nsjail. uv spawns a Python interpreter that compiles .py
files with parallelism scaling to the host CPU count, opening many file
descriptors at once. The download nsjail capped `rlimit_nofile` at 64,
which is exhausted on high-core machines, failing every install with
"Failed to bytecode-compile ... Too many open files (os error 24)".
Low-core VMs never hit the cap, so this surfaced only as a regression on
larger workers after upgrading.
Raise `rlimit_nofile` to 10000, matching the runtime configs
(run.python3 / run.ansible) that already use that value.
Fixes WIN-2009
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(backend): route //native TypeScript previews to native workers
Previewing a TypeScript script carrying the `//native` annotation was
pushed with `language = bun` (what the editor sends), so the job was
tagged `bun` and routed to a regular bun worker. A native-mode worker
neither matches the `bun` tag nor accepts a non-native `script_lang`
(worker.rs rejects with "cannot execute non-native job with language
'bun'"), so previewing a `//native` script on a native-only worker setup
failed — even though the deployed version of the same script runs fine
as `bunnative` / tag `nativets`.
`push` now reconciles the preview language with the `//native`
annotation for `JobPayload::Code`, mirroring the deploy-time logic in
`worker_lockfiles`: `bun` + `//native` is promoted to `bunnative` (tag
`nativets`), and `bunnative` without `//native` is demoted back to
`bun`. This makes a preview run exactly like the deployed script would,
and covers every preview entry point (run_preview_script, inline
preview, codebase preview) since they all go through `JobPayload::Code`.
Adds regression tests asserting the queued job's `script_lang`/`tag` for
all four (declared language × annotation) combinations.
Fixes WIN-2007
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore(backend): add sqlx cache for preview_native_tag test query
The regression test's `sqlx::query!` for `v2_job` (tag, script_lang) needs
a cached entry so `SQLX_OFFLINE=true` CI compiles it. Adds exactly one new
cache file; no existing (OSS or EE) caches removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(backend): trim preview native-tag tests to the essentials
Keep the core regression (bun + //native → bunnative/nativets) and the
guard that plain bun previews are unaffected. Drop the two bunnative-
declared cases, which only re-verified the mirrored demote logic and
weren't the reported issue. The shared query is unchanged, so the sqlx
cache stays valid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): stop git-sync promotion deploys from dropping triggers/schedules
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: bump git-sync hub script to hub/28261 (windmill-cli 1.713.2)
Points LATEST_GIT_SYNC_SCRIPT_PATH at the republished sync-script-to-git-repo
that pins windmill-cli@1.713.2, which carries the promotion include-derivation
fix in this PR.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
#9400 (WIN-2003) added the ctrl_break() handler as a `#[cfg(windows)]`
branch inside the two Windows-path `tokio::select!` blocks in
shutdown_signal. tokio's `select!` macro does not accept `#[cfg(...)]`
attributes on individual branches, so windmill-common fails to compile
on Windows ("no rules expected this token in macro call").
This slipped through CI because the only job that builds the backend on
Windows is cli-tests.yml's `test-windows`, which triggers only on
`cli/**` changes — #9400 was backend-only.
Fix: define `ctrl_break()` for the whole `not(any(linux, macos))` scope
instead of just `windows`. On Windows it awaits the real CTRL_BREAK
signal; on other non-unix targets it is a never-resolving future, so the
branch is inert there. The select! branches become plain (no per-branch
`#[cfg]`), which the macro accepts.
Verified: the `#[cfg]`-on-branch form reproduces the exact macro error
against tokio 1.46.1, and the fixed form compiles clean.
Fixes WIN-2003 (Windows build regression)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The instance username-conflict resolver rewrote
workspace_runnable_dependencies.app_path to the new user path before the
app row itself was renamed, violating fk_workspace_runnable_dependencies_app_path.
That FK is ON UPDATE CASCADE, so renaming the app already propagates the new
path; the manual rewrite was redundant and mis-ordered. Any user owning an app
under u/<username>/ with a tracked runnable dependency hit HTTP 500 and could
not have their username conflict resolved.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
On Windows, shutdown_signal only registered ctrl_c() (CTRL_C_EVENT).
CTRL_BREAK_EVENT — the default kill signal sent by Nomad's raw_exec
driver on Windows — had no handler, so the worker terminated
immediately without graceful shutdown, interrupting running jobs.
Add a ctrl_break() helper (mirroring the Unix terminate() helper) and
register it as an additional branch in both Windows tokio::select!
blocks in shutdown_signal.
Fixes WIN-2003
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* oauth: add salesforce provider
Register Salesforce OAuth (Authorization Code) for Windmill resource connect.
Production uses login.salesforce.com; the sandbox block points at
test.salesforce.com (URL overrides only; scopes inherited) per #9358, so a single
canonical `salesforce` resource type covers both with separate `salesforce_sandbox`
instance credentials.
Paired with the hub integration: windmill-labs/windmill-integrations#131.
The Salesforce icon already exists in the frontend (SalesforceIcon.svelte).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix JSON syntax error in oauth_connect.json
* fix: add salesforce production tile to OAuth settings dropdown
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Python jobs under nsjail experience slow imports (~465ms for `import
requests`) because dependency directories are mounted read-only. Without
pre-compiled `.pyc` files, Python recompiles `.py` source to bytecode
in-memory on every import in every fresh nsjail process, paying the cost
repeatedly.
Add `--compile-bytecode` to both `uv pip install` invocations (the
Rust-driven install in python_executor.rs and the nsjail
download_deps.py.sh script) so `.pyc` files are generated at install
time and available at runtime even through read-only mounts. The files
are included in both the local cache and S3 piptar uploads.
The flag is supported by the uv version (0.9.24+) shipped in the
Dockerfile. The existing `__pycache__` skip only filters top-level dirs
for dedup logic, not subdirs within packages, so there is no conflict.
Fixes WIN-2001
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(api): handle multi-version scripts when removing granular ACL
remove_granular_acl built a CTE whose `old` subquery selects one row per
matching table row, then read it back via the scalar subquery
`(SELECT old_write FROM old)` in the RETURNING clause. For the `script`
table the PK is (workspace_id, hash), so a path with multiple deployed
versions yields several rows sharing the same (workspace_id, path). When
two or more versions carried the ACL key, `old` returned multiple rows and
PostgreSQL rejected the scalar subquery with "more than one row returned by
a subquery used as an expression", making it impossible to remove an ACL
entry from a script's permissions panel.
All versions share the same extra_perms value (the UPDATE applies to every
matching row), so any single row's old_write is representative. Add
`LIMIT 1` to the scalar subquery. Other kinds are unaffected because they
have a unique constraint on (workspace_id, path/name).
Introduced by b3603d872 (#7365). Add a regression test reproducing the
multi-version case at the SQL level.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: remove granular ACL regression test
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: trim ACL fix comment
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(cache): allow overriding hub base url via env in `cache` mode
The `windmill cache hubPaths.json` prebuild step (run in the Dockerfile) never
connects to the DB, so HUB_BASE_URL stays at its compiled default
(https://hub.windmill.dev) — unlike server/worker modes which load it from the
DB global setting. This made it impossible to point the prebuild cache step at
a private or staging hub.
Read HUB_BASE_URL from the environment at the start of cache_hub_scripts and
store it into the existing HUB_BASE_URL ArcSwap (the same static the hub fetch
functions read). No effect unless the env var is set and non-empty; server and
worker modes are unchanged (they still use the DB setting).
This also enables validating hub-script dependency changes end-to-end against a
local fake-hub before pushing to the real hub.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(security): re-pin cached hub scripts to patched versions
windmill-integrations#133 was merged and pushed to the hub, minting new
versions with regenerated (CVE-free) lockfiles. Bump the hubPaths.json pins so
the prebuild cache step (`windmill cache`) fetches the patched lockfiles instead
of the old vulnerable ones (the hub serves each version_id immutably, so the old
pins keep returning the vulnerable deps until bumped).
- slackErrorHandler 19741 -> 28241
- slackRecoveryHandler 9080 -> 28239
- slackSuccessHandler 28220 -> 28240
- smtpReport 9086 -> 28242
- appReport 28076 -> 28243 (puppeteer screenshot script)
- gitInitRepo 28219 -> 28229 (already-fixed hub version; pin was stale)
Validated end-to-end against the real hub: `windmill cache` with these pins
produces a clean cache_nomount/bun (axios 1.16.1, form-data 4.0.5,
follow-redirects 1.16.0, nodemailer 8.0.10, ws 8.21.0, svelte 5.55.8,
devalue 5.8.1; basic-ftp and ip-address no longer pulled). No vulnerable
versions remain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): preserve step/subflow worker tags under a custom-tagged flow
A flow running on a custom worker tag force-propagates that tag to every
descendant step, script and nested sub-flow, overriding their own declared
tags. This made it impossible to route a specific step or sub-flow to a
different worker group. The new opt-in FlowValue.preserve_step_tags lets a
step that declares its own non-empty tag run on it; untagged steps still
inherit the flow tag. Defaults off to preserve existing behavior.
* chore: regenerate system prompts for preserve_step_tags
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(flows): nest preserve_step_tags toggle under flow worker tag setting
The toggle only affects routing when the flow has a custom worker tag, so
show it as a sub-setting of the Worker Group tag picker, visible only once a
tag is set, instead of as a standalone option.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): allow step worker tag picker when preserve_step_tags is enabled
When a flow defines a worker tag, the per-step tag picker was replaced by a
read-only "Flow's WG" label. With preserve_step_tags enabled the step's own
tag is honored, so the picker must remain editable in that case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): propagate preserve_step_tags to branch and loop bodies
payload_from_modules built the synthetic RawFlow for branch/loop bodies with
a default FlowValue, dropping preserve_step_tags. Tagged steps inside a
branch or loop therefore still inherited the parent flow tag even with the
flag enabled. Thread the flag through to the synthetic FlowValue so the
behavior is consistent for nested containers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): clear preserve_step_tags when flow worker tag is removed
Avoids the flag lingering as invisible state after the flow tag (and its
toggle) are removed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): repair preserve_step_tags propagation to branch/loop bodies
The previous commit added flow.preserve_step_tags at the payload_from_modules
call sites but the parameter and FlowValue field were not actually threaded
through (a failed edit left the function unchanged), so the crate did not
compile. This completes the change: payload_from_modules takes preserve_step_tags
and sets it on the synthetic FlowValue for branch/loop bodies.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): complete preserve_step_tags propagation to branch/loop bodies
Previous two commits left windmill-worker uncompilable: payload_from_modules
received flow.preserve_step_tags at its call sites but the parameter and the
synthetic FlowValue field were not actually added. This adds the parameter,
sets preserve_step_tags on the synthetic FlowValue, and threads
flow.preserve_step_tags through all five call sites.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(flows): clear preserve_step_tags whenever the flow worker tag is removed
The flag was only reset when the Worker Group toggle was switched off, not
when the tag was cleared directly in the picker (or via the YAML editor),
leaving preserve_step_tags=true as invisible state with the advanced badge
still reporting it active. Move the cleanup into the reactive block that
already tracks the flow tag so every clear path is covered.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(apps): make public apps opt into cross-origin isolation via wm_coep
Public app pages served at /public/* and custom paths /a/* were not
getting the COEP/COOP/CORP headers, so they were blocked when embedded
as an iframe inside a cross-origin-isolated page (e.g. another raw app,
which sets Cross-Origin-Embedder-Policy: require-corp). A nested
document loaded into a require-corp context must itself set COEP for
the iframe to load.
Rather than applying the isolation headers to all public pages (which
would also force COEP on classic apps and break subresources without
CORP, e.g. external image URLs or embeds), public apps now opt in via
a `wm_coep` query param on the embed URL:
<iframe src="https://<domain>/public/<ws>/<secret>?wm_coep=on">
The app publish drawer gains a URL/Embed toggle: "URL" shows the plain
shareable link (param-free), "Embed" shows a ready-to-copy iframe
snippet with wm_coep baked in, so the flag is discoverable exactly when
embedding and absent otherwise.
`wm_coep` is consumed internally and stripped from the app `query`
context so it doesn't collide with app-defined params. Only params we
own are stripped (an explicit set), not the whole `wm_` prefix.
Fixes GIT-884
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* nit
* nit
* fix(apps): only bake wm_coep into embed snippet for raw apps
AppEditorHeaderDeploy is shared by the classic (AppEditorHeader) and raw
(RawAppEditorHeader) deploy drawers. The embed snippet unconditionally
appended ?wm_coep=on, which for a classic/low-code app forces COEP
require-corp on the document and breaks no-CORP cross-origin subresources
(external <img> in AppImage/AppStatCard/AppNavbar, {@html} embeds in
AppHtml, CDN import() in AppCustomComponent) — the exact regression the
opt-in design avoids.
Add a `rawApp` prop (default false); the raw header passes rawApp. The
flag is appended only for raw apps; classic apps get a plain iframe
snippet, and the wm_coep helper text is shown only for raw apps.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
CLI deploys (sync push, set-permissioned-as) went through the same
create/update endpoints as a UI "deploy from draft", which delete the
draft at that path. That silently wiped teammates' in-progress drafts on
every push. Add a transient skip_draft_deletion deploy flag (mirroring
deployment_message) that the CLI sets; the backend then skips the
DELETE FROM draft for scripts, flows, and apps. UI deploys are unchanged.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
get_token_using_oauth resolved the AI OAuth resource's token_url and
POSTed to it without any SSRF validation, while base_url is validated in
get_base_url. A workspace member with resources:write could point
token_url at an internal/metadata address (e.g. 169.254.169.254),
turning the server into an authenticated blind SSRF probe.
Validate the resolved token_url with validate_url_for_ssrf before the
request, gated behind the same ALLOW_PRIVATE_AI_BASE_URLS opt-in as
base_url so private AI deployments keep working consistently for both
URL fields. ALLOW_PRIVATE_AI_BASE_URLS is now pub so windmill-api can
reuse it instead of re-parsing the env var.
* fix: trigger git sync for re-encrypted secrets on encryption key change
When changing a workspace encryption key, the secret variables get
re-encrypted with the new key, but the git sync was only dispatched for
the encryption_key.yaml metadata file. Repos with Secrets sync enabled
were left with stale ciphertexts until the next per-variable deployment.
Now, after the transaction commits, we also dispatch a Variable git sync
event for each re-encrypted secret so the new encrypted values are
pushed to the configured repos. Errors are logged but don't roll back
the key rotation.
Fixes WIN-1994
Fixes#9344
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: batch encryption-key rotation into one git-sync job
Workspace encryption key rotation now re-encrypts every secret variable
and then dispatches a single batched git-sync job carrying the Key event
plus one Variable item per re-encrypted secret. Repos with Secrets sync
enabled receive every new ciphertext in one commit instead of nothing
(previously only `encryption_key.yaml` was pushed) — and instead of N
separate jobs the debouncer might or might not merge.
Wires through the new `handle_deployment_metadata_batch` entry point
added in the companion EE PR; OSS has a no-op shim so the build stays
green.
Adds an integration test (`workspace_encryption_key_git_sync`) asserting
that rotating the key with 3 secret variables in scope produces exactly
one deployment-callback job whose `items` array contains the Key event
+ all 3 variable entries and `skip_secret=false`.
Fixes WIN-1994
Fixes#9344
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump ee-repo-ref for git-sync helper simplification
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: cover non-debouncing git-sync fallback on key rotation
Adds a regression test exercising a workspace whose sync script predates
hub version 28103: the rotation must still queue a legacy-format
deployment-callback job per item (encryption_key + each re-encrypted
secret) instead of silently skipping the repo. Bumps ee-repo-ref to the
EE fallback fix.
Addresses the P1 raised in the PR review (Codex/Pi/Claude): batch path
dropped git sync entirely for repos without sync-job debouncing support.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: add sqlx offline cache for encryption-key git-sync test queries
The cargo_test CI job builds with SQLX_OFFLINE=true; the two new
sqlx::query!/query_as! calls in
windmill-api-integration-tests/tests/workspace_encryption_key_git_sync.rs
had no cached entries, failing the build with E0282. Regenerated and
added only the two new query caches (no EE/feature cache loss).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump ee-repo-ref to updated EE companion PR (08e3b9b)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ai): disable redirect following on AI proxy client to close SSRF
The AI proxy validates the configured base_url against SSRF rules but the
shared HTTP client followed up to 10 redirects without revalidating the
hops, so a public base_url could 3xx the server into a private/internal
address (e.g. the Docker socket or cloud metadata). Disable redirect
following so the validated host is the only one the server connects to.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ai): remove heavy redirect SSRF integration test
Drop the integration-test-level regression for redirect following; it
spins up a full API server + DB for a one-line client-config change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(api): don't follow symlinks when reading service log files
Defense in depth on top of the existing `..` path-traversal check in
the get_log_file handler: reject the request if the final path
component is a symlink, so a planted symlink in the logs directory
cannot be used to read arbitrary files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(api): authorize and harden the jobs_u get_log_file endpoint
The unauthenticated jobs_u get_log_file endpoint served any job's log
file to anyone who knew the job UUID, with no authorization. Gate it the
same way as get_job_logs: look up the job (the log directory name is the
job id) filtered by workspace and the caller's scope tags, and only allow
non-logged-in callers to read logs of jobs created by the anonymous user.
Also add defense in depth: refuse to read through a symlink so a planted
symlink in the logs directory cannot be used to exfiltrate arbitrary files.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The fork-branch deployment callback runs the hub script
sync-script-to-git-repo-windmill, which imports `windmill-cli` as a pinned
npm dependency and runs it in-process (it does NOT shell out to a PATH wmill).
hub/28236 pinned windmill-cli@1.706.1, whose `git-deploy --only-create-branch`
path returns early without pushing — so the fork branch was checked out
locally but never published to the remote. #9366 fixed the CLI and shipped it
as windmill-cli@1.712.0, but without a hub-script bump the running callback
still used 1.706.1.
Bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28238, which is identical to 28236
except it pins windmill-cli@1.712.0 (content + lockfile). This fixes
test_workspace_fork_creates_branch and production fork-branch creation.
Also add backend/windmill-common/src/workspaces.rs to the git-sync-test
path-gate so future script-path bumps trigger the e2e (the bump alone is not
otherwise covered by the gate).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(oauth): support per-provider sandbox URLs in registry + instance settings
* fix(oauth): polish sandbox review nits (cc lookup, header label, ee ref)
* refactor(oauth): drop dead build_oauth_clients duplicate in windmill-oauth
* refactor(oauth): derive sandbox-capable provider list from registry
* chore(docker): copy oauth_connect.json into frontend build stage
* test(oauth): cover sandbox helpers (as_sandbox, canonical_name, resolve)
* chore: update ee-repo-ref to 9297d8f790346e6a6ad540c7bca1a67f91ec11a2
This commit updates the EE repository reference after PR #595 was merged in windmill-ee-private.
Previous ee-repo-ref: 3ab3eca9ac15ebab6db991e7964bc5e48ce21f42
New ee-repo-ref: 9297d8f790346e6a6ad540c7bca1a67f91ec11a2
Automated by sync-ee-ref workflow.
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* [ee] fix(git-sync): publish fork branch on only_create_branch from the CLI
Fixes WIN-1997. Forking a git-sync-configured workspace must push a
`wm-fork/<branch>/<id>` branch to the repo, but the integration test
`test_workspace_fork_creates_branch` failed: the fork callback job
succeeded yet no branch appeared.
Root cause: the fork-branch callback runs the sync script with
`only_create_branch: true` and no items. The hub sync script delegates
branch checkout to `wmill sync git-deploy --only-create-branch` and runs
its own in-process commit+push ONLY for the `!only_create_branch` path
(`if (!only_create_branch) git_push(...)`). #9284 had moved commit+push
out of the CLI to the caller for the GPG-cache-warmth invariant
(WIN-1974) — but it also dropped the CLI's push for the branch-only
case. A branch-only publish has no commit, so no signing is involved and
the GPG concern does not apply; with neither the CLI nor the hub script
pushing, the empty fork branch was never published.
Restore the CLI push for the `only_create_branch` path (a bare
`git push --porcelain` of the checked-out branch ref). Adds a
deterministic CLI regression test that runs `git-deploy
--only-create-branch` for a fork workspace and asserts the branch
reaches the remote with no caller-side push.
EE companion: format the fork-branch commit message with Display instead
of Debug (no more `Some("...")` leak).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd
This commit updates the EE repository reference after PR #597 was merged in windmill-ee-private.
Previous ee-repo-ref: 8b02336fcebdfae4b9d2795cbb74fa7046530bcb
New ee-repo-ref: a30079e75dc5b7d7413aa8ee20e40e80bfea9cbd
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* [ee] feat(queue): duration-weighted fairness admission atomic
Add the `WORKSPACE_FAIRNESS_ADMISSION_PPM` atomic that the EE
`workspace_fairness_ee::refresh_overloaded` writes on each refresh
(see companion EE PR). The atomic is read on every pull by
`should_admit_capped` to decide whether the dispatch goes down the
standard or fairness path. Defaults to 10_000 (= admit all) so the
pre-fairness behaviour is preserved until the first refresh fires.
OSS stub in `workspace_fairness.rs` continues to return `true`
unconditionally, so non-EE builds are bit-identical.
* docs(queue): consolidate full fairness algorithm into workspace_fairness.rs
Move the algorithm doc — what "overloaded" means in worker-seconds, the
duration-weighted admission derivation, coordinated refresh structure,
audit emission, the SQL perf constraints (no params CTE, drive running
side from v2_job_runtime), and EE gating — into the OSS surface module
where it is readable without EE access. The EE file becomes implementation
only.
Also bump ee-repo-ref to the EE commit that strips the duplicate doc.
* docs(queue): clarify ADMISSION_PPM default is "admit all", not count-based
Addresses CI review (claude[bot]): the `10_000` initial value is the
"admit all" no-op default that applies before the first refresh
classifies an overloaded set — not the count-based value (which would
be `target * 10_000`). The count-based form is the empty-bucket fallback
inside `compute_admission_ppm`, a different thing.
* chore(queue): point ee-repo-ref at EE main (fairness admission merged via #593)
* fix(queue): duration-weighted admission uses unclamped service-time window
Bumps ee-repo-ref to the EE fix (windmill-ee-private#596) that sources
D_c/D_u for the admission probability from a separate 60s service-time
window of true `duration_ms`, instead of the occupancy aggregation whose
per-job contributions are clamped to the 10s occupancy window. The clamp
truncated D_c for capped jobs longer than the window, under-admitting the
duration skew (true 34s jobs → ~86% effective share instead of the target
65%). Occupancy worker-seconds still drive overload classification.
Updates the algorithm doc in workspace_fairness.rs accordingly.
Note: ee-repo-ref points at the EE feature branch; re-point to EE main
once #596 merges.
`announce_server_started` writes a `server_heartbeat:{INSTANCE_NAME}` row
on each startup. INSTANCE_NAME is a fresh random string per process, so
the row is never updated again and a new row is inserted on every
restart, growing background_task_state unboundedly.
Add an hourly monitor task that deletes server_heartbeat:* rows older
than 7 days. Older rows cannot influence check_any_server_started (which
only considers heartbeats refreshed after the restart was initiated), so
they are safe to prune.
Fixes WIN-1990.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(queue): bump EE ref to include worker_ping fairness signal
The current ee-repo-ref.txt pointed to 31cda7c (an unrelated merge
commit on the asset-graph-view-ee branch) instead of ddc9e80, which
contains the workspace-fairness fix that switches the active-share
signal from v2_job_queue.running=true to worker_ping. As a result
cloud was still computing overload off the legacy signal, so a
workspace with many in-flight/suspended flows (lancom01-prod, with
799 suspended flows × 3 v2_job_queue bookkeeping rows each = 2397
running-true rows) was flagged as 95% of cluster activity despite
consuming zero worker slots.
Bumping to ddc9e80 picks up the worker_ping-based signal, which
naturally excludes (a) suspended jobs (no worker pinging them),
(b) zombie running-rows from dead workers, and (c) flow/flownode
orchestration rows that never run on a worker in the first place.
* test(queue): seed v2_job rows + realistic durations for fairness helpers
The new duration-weighted fairness algorithm joins v2_job_queue and
v2_job_completed to v2_job for the `kind` filter (excluding flow
bookkeeping) and reads `duration_ms` for the completed contribution.
Update the test helpers to mirror that schema:
* `insert_completed` now inserts a matching v2_job row (kind=script)
and writes `duration_ms = 1000` with a 1-second [started_at,
completed_at] interval, so each completed row contributes ~1
worker-second when fully inside the refresh window.
* `insert_queued` likewise pre-inserts v2_job, sets `started_at`
to NOW() - 1s when running=true (so running rows contribute ~1
worker-second by the time the refresh runs), and seeds
v2_job_runtime.ping so the running side accrues real-time worker
seconds (the algorithm bounds end-of-interval by ping).
The zombie/suspended insert helpers are intentionally left without
v2_job rows — the new algorithm's INNER JOIN excludes them, so they
still correctly contribute zero worker-seconds.
* chore(queue): bump EE ref to duration-weighted fairness algorithm
Companion to windmill-ee-private#<TBD>: switch the EE workspace
fairness aggregation from a count-based UNION (worker_ping snapshot
+ v2_job_completed count) to a worker-seconds aggregation sourced
directly from v2_job_queue and v2_job_completed, with kind/suspend
filters mirroring handle_zombie_jobs and per-row defenses against
zombie inflation on both halves.
* chore(queue): bump EE ref for fairness perf fix (inline window_start)
* chore(queue): bump EE ref for fairness perf rewrite (driver-side flip)
* update ee ref
hugocasa
Ruben Fiszel
centdix
Diego Imbert