mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-20 08:01:35 +00:00
workflow-execution-time-display
346 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 hugocasa
|
fad1a549d9 |
feat(otel): connect jobs to the inbound distributed trace (#9456)
* feat(otel): propagate inbound W3C traceparent to job spans Capture the inbound traceparent header at the run endpoints (WebhookArgs::to_args_from_format) into a reserved _wm_traceparent arg key (gated on OTEL_TRACING_ENABLED), riding the args jsonb like _ENTRYPOINT_OVERRIDE. At pickup, create_span_with_name attaches a span link from the job's worker span to the originating distributed trace, so a job triggered by an instrumented service is connected to the caller's trace while keeping its UUID-derived trace id (trace-by-job-id unaffected). The link/parse logic lives in the EE otel modules; this OSS side only captures the header and calls the (no-op outside EE) hook. Companion EE PR required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to inbound-trace-propagation EE branch Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(agents): don't attribute work to specific customers in repo content Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(otel): relocate job + script spans into the inbound trace Builds on the captured _wm_traceparent: the worker job span is re-parented on the inbound caller context, the script subprocess's TRACEPARENT env is the inbound context (so its spans join the caller's trace), and the context is propagated to flow steps so the whole flow relocates. Carried to the worker via a new LogContext.inbound_traceparent field. Non-inbound jobs are unchanged. Adds a relocation integration test. Companion EE PR required. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to inbound-trace-propagation relocate commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(otel): harden inbound traceparent capture Address review feedback: - strip any caller-supplied _wm_traceparent from args/extra before stashing the header-captured value, so the reserved key is Windmill-controlled only - valid_w3c_traceparent: reject version ff and require lowercase hex, so we don't forward an inbound header that downstream OTel parsers would reject - clarify that the capture helper does not validate the W3C format (done at use) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a This commit updates the EE repository reference after PR #604 was merged in windmill-ee-private. Previous ee-repo-ref: 8fc04fb105dc49769205f7174d551a0d134d1bec New ee-repo-ref: 2c7964460327fab5e3a27c0f74b8d6f26ab7f79a Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
f2f0812a04 |
feat(flows): opt-in to include the stopping step's result in early-stop errors (#9446)
* feat(flows): early stop can include the stopping step's result in the raised error
When a step uses Early Stop with "Raise an error message if stopped", the
flow result was entirely replaced with a static error object
({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the
stopping step's own output. This made it impossible to stop+fail a flow
while preserving the data the step produced (e.g. an API that returns
HTTP 200 with a userErrors payload).
Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on
the raise-error path, the raised payload becomes
{"error": {...}, "result": <step result>} instead of dropping the result.
Default is false, so existing behavior is unchanged. The option is threaded
through the worker's stop-after-if handling (including stop_after_all_iters_if
for loops/branchall) and exposed in the flow editor's Early Stop panel.
Fixes WIN-2012
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover early-stop error_include_result payload shaping
Add a regression test asserting that a step using Early Stop with a raised
error message and error_include_result=true fails the flow while preserving
the step output as {"error": {..}, "result": <step result>}, and that with
the flag off the result is the bare {"error": {..}} object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(flows): nest early-stop step result inside the error object
Embed the stopping step's result under `error.result` rather than as a
top-level sibling of `error`. This keeps the flow result shape as
`{ "error": { .. } }` — identical to a normal error — so consumers that
key off the top-level shape (single `error` key) keep working, while the
data is still preserved for those that look inside the error object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): always include the stopping step's result in early-stop errors
Drop the opt-in `error_include_result` gate. Since the step result is nested
inside the error object (`error.result`), the top-level result shape stays
`{ "error": .. }` — identical to a normal error — so consumers that detect or
parse failures by the top-level shape are unaffected. Gating it added schema
surface, plumbing, and a UI toggle for no real compatibility benefit.
Now, whenever a step early-stops with a raised error message, the flow fails
and the raised error embeds the stopping step's own result under
`error.result` (aggregated iteration results for loops/branchall). This
reverts the `StopAfterIf.error_include_result` field, its threading, the
OpenAPI/generated-client surface, and the editor toggle; the "Raise an error
message" tooltip now notes that the step result is included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): gate early-stop result inclusion behind opt-in flag
Re-introduce the per-step `error_include_result` flag (default off) instead
of always embedding the step result. Although nesting the result under
`error.result` keeps the result *shape* backward-compatible, it does not
address data exposure: a failed flow's result is propagated to synchronous
webhook callers, the flow's failure module, and the workspace/global error
handler (commonly a Slack/email/outbound-webhook notifier). Always including
the step output would surface previously-redacted intermediate data to all of
those sinks for every existing error-stop flow.
Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and
only embeds `error.result` when the flow author explicitly opts in, matching
the original issue's intent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(flows): omit error_include_result when false; refresh generated prompts
- Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result`
so serialized flows are byte-identical when the flag is off. Fixes the
`flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing
flows.
- Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts`
for the new OpenFlow `error_include_result` property. Fixes check-freshness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover error_include_result for the loop "stop after all iters" path
Add a regression test for the stop_after_all_iters_if branch, where `nresult`
already holds the aggregated iteration results — confirming `error.result`
carries each iteration's output (distinct from the per-step fallback path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
hugocasa
|
468aa230e5 |
refactor: resolve workspace imports via /f/,/u/ not $f/,$u/ aliases (#9438)
* refactor: resolve workspace imports via /f/,/u/ not $f/,$u/ aliases Keep the CLI managed tsconfig.wmill.json / `refresh tsconfig` / Deno import-map QoL from #9378, but re-key it on the existing /f/,/u/ workspace paths instead of the new $f/,$u/ specifiers. Verified /f/,/u/ resolves in tsc, Bun, Deno, the in-app ATA editor, and the worker, so the $-prefixed alias added no value. Drop the $f/,$u/ handling from the parser, dep-map, deno_executor, bun loaders, ATA, relative_imports and monaco paths; revert the windmill-parser-wasm-ts bump (1.714.0 -> 1.695.0). Also fold in the cli/package-lock.json sync for the already-committed pg-gateway dependency. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: drop duplicate relative-path check and restore rustfmt formatting Follow-up cleanups to the previous commit's full-file reverts, which restored pre-#9378 state that main had since improved: - relative_imports.ts: remove the redundant duplicate d.startsWith('/') (pre-#9378 had it; #9378 had repurposed that line, so main has no dup). - windmill-parser-ts/src/lib.rs: restore the multi-line new_source_file(...) formatting required by backend/rustfmt.toml (the single-line revert would fail `cargo fmt --check`). Now differs from main only by the $f//$u/ removal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
220cd35cf7 |
feat: support $f/ and $u/ import path aliases for scripts (#9378)
* feat: support $f/ and $u/ import path aliases for scripts $f/ and $u/ are local-friendly aliases for the absolute workspace import paths /f/ and /u/. Unlike the /-prefixed form (which local tools treat as a filesystem-root path), the $-prefixed form is a bare specifier that can be remapped via tsconfig paths / Deno import maps, so the same import resolves on the Windmill worker and in a local editor. - worker: recognize $f//$u/ in the Deno import map and both Bun loaders - dep-map/parser: normalize $f/->f/, $u/->u/ for lockgen + dep tracking - cli: emit $f/$u path aliases in generated tsconfig.json / deno.json - frontend: ATA + Monaco paths resolve $f//$u/ type hints in the editor Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): split generated tsconfig into managed + user file with refresh command Mirror the AGENTS.cli.md/AGENTS.md prompts model for the IDE tsconfig so the recommended settings can evolve without ever clobbering user customizations: - tsconfig.wmill.json: wmill-managed, always refreshed, holds recommended compilerOptions incl. the $f/$u path aliases (Deno: import_map.wmill.json) - tsconfig.json: user-owned, created once, just extends the managed file; warn (never auto-edit) when an existing one doesn't reference it - add 'wmill refresh tsconfig'; init generates it unconditionally (no longer gated behind resource-type namespace / a bound workspace) - regenerate CLI guidance docs for the new subcommand Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): address PR review on $f/ tsconfig generation - handle existing deno.jsonc so we don't shadow it with a new deno.json (P1 identified by cubic) - fix the bun-types hint that pointed users at the managed do-not-edit tsconfig.wmill.json; tell them to install + re-run 'wmill refresh tsconfig' - document the .ts-extension-only local-resolution limitation (cross-flavor .bun.ts/.deno.ts/.fetch.ts scripts won't resolve in a local editor) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): warn when a project's tsconfig isn't wired to tsconfig.wmill.json Mirror the prompts freshness check for the managed tsconfig so users with an existing setup actually discover they're missing $f//$u/ resolution: - embed a version hash in tsconfig.wmill.json (excludes the env-dependent bun-types 'types' entry so it doesn't false-positive) - add warnIfTsconfigStale to the main.ts freshness hook, gated identically to the prompts check (skips init/refresh/help/version). When a tsconfig.json exists it warns one line (stderr) if the managed file is missing, not referenced via extends, or out of date; silent for non-TS projects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): make tsconfig setup equivalent to prompts (auto-wire + stale-only) Unify the two managed-file systems so they behave identically: - auto-wire an existing unlinked tsconfig.json/deno.json on init/refresh (add extends / importMap; merge into an array extends), instead of only warning. Parses JSON and falls back to a warning when it can't round-trip (JSONC comments, or a conflicting deno imports/importMap) — never corrupts. - narrow warnIfTsconfigStale to stale-only, gated on the managed file existing, exactly like warnIfPromptsStale: it no longer nags about a missing or unlinked tsconfig.json, so a deliberately-custom/unlinked setup stays silent and a not-yet-initialized project isn't bothered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): place tsconfig.wmill.json first in extends to preserve user base config Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): migrate legacy tsconfig and require consent for custom configs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): align prompts wiring to the same consent model Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): bump windmill-parser-wasm-ts to 1.714.0 for $f/ $u/ aliases Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(worker): resolve $f/ and $u/ in deno lock generation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: narrow relative-imports lock-gen guard to deno import-map failure Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): sync bun.lock with windmill-parser-wasm-ts 1.714.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): warn when a custom tsconfig's paths would shadow $f/ $u/ aliases Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszel
|
8053266f88 |
fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url (#9428)
* fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): clone user_db for oauth2 refresh and drop advisory ids from comments Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(mcp): disable redirects on MCP client to prevent SSRF bypass Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
89a7a37776 |
fix(backend): authorize single-job read endpoints by job/flow visibility (#9416)
* fix(backend): authorize single-job read endpoints by job/flow visibility Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(jobs): share read links + cached access checks for run visibility - Cache the job read-access RLS probe (size-bounded LRU keyed by the caller's authz-relevant identity + job id; no TTL since job-side inputs are immutable). - Inherit visibility along the full parent_job chain so any flow you can see lets you read its (deeply nested) steps. - Share read links: GET /jobs/job_view_token/{id} mints a stateless HMAC(workspace_key, job_id) token (only if the caller can read the job); the token grants an authenticated member read of that job and its flow subtree via a ?view_token query param or X-View-Token header. Run page gains a Share button and honors a ?view_token link. - Denied-but-existing reads now return 403 with guidance to request a share link (vs 404 for non-existent), and the run page renders that case with instructions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(jobs): address PR review — scope-tag check on mint, constant-time view-token verify - P1 (Codex): get_job_view_token now enforces the caller's if_jobs:filter_tags scope before minting, so a tag-scoped token can't mint a transferable link for a job outside its tags. Adds a scoped-token regression test (allowed + denied). - Constant-time view-token verification (HmacSha256::verify_slice) instead of comparing hex strings (Claude/Pi nit). - get_completed_job_result: an authed reader passing an invalid suspended-secret triple now falls through to the normal visibility gate instead of erroring out (Claude nit); unauthenticated callers still rejected. - Length-prefix the read-access cache key fields so no input values can collide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(api): add job_view_token to openapi spec; use generated client in run page Addresses Codex review nit: the new GET /jobs/job_view_token/{id} endpoint was missing from openapi.yaml (the source the frontend client is generated from). Adds the path + operationId getJobViewToken, and switches the run page's Share button from a raw fetch to JobService.getJobViewToken. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): carry view_token on share-link downloads Addresses Codex review: download actions bypass the request interceptor that adds X-View-Token (downloadViaClient uses raw fetch; cookie-mode downloads use plain hrefs), so a share-link viewer got 403 downloading logs/results/args. Append the view_token query param to the job download paths (result/logs/args/flow-all-logs) via a new appendViewToken() helper, covering both client-fetch and href modes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(jobs): enforce tag scope in require_job_read_access (view-token use side) Addresses Codex P1: the view_token use-side bypassed if_jobs:filter_tags on handlers that don't tag-filter their data query (result_by_id, get_flow_job_debug_info, get_otel_traces) — a tag-scoped token could use someone else's valid share token to read out-of-scope job data. Move the tag-scope check into require_job_read_access (runs before any created_by/view_token/RLS grant), so it applies uniformly to every gated handler; removes the now-redundant explicit check in get_job_view_token. Adds a use-side regression test (scoped token + valid out-of-scope view_token denied on otel/result_by_id; in-scope still allowed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): include workspace in share read link Addresses Codex P1: the copied share URL omitted the workspace. The token is signed with the run's workspace key and the logged layout only switches $workspaceStore when the URL carries workspace=, so a recipient whose persisted active workspace differs would open the link against the wrong workspace and the token would fail validation. Pin workspace= alongside view_token in the link. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(jobs): authorize get_result_maybe get_started branch for queued jobs Addresses Codex P1: get_completed_job_result_maybe only gated when a completed row existed; with ?get_started=true a non-reader reached the fallback branch and got started:true for a running private job. Now fetches created_by and authorizes (created_by/view_token/RLS, or anonymous for unauth) before disclosing running-state; a non-existent job still returns started:false (leaks nothing). Adds a regression test with a queued (no completed row) private job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
73edebc833 |
fix(backend): route //native TypeScript previews to native workers (WIN-2007) (#9407)
* fix(backend): route //native TypeScript previews to native workers Previewing a TypeScript script carrying the `//native` annotation was pushed with `language = bun` (what the editor sends), so the job was tagged `bun` and routed to a regular bun worker. A native-mode worker neither matches the `bun` tag nor accepts a non-native `script_lang` (worker.rs rejects with "cannot execute non-native job with language 'bun'"), so previewing a `//native` script on a native-only worker setup failed — even though the deployed version of the same script runs fine as `bunnative` / tag `nativets`. `push` now reconciles the preview language with the `//native` annotation for `JobPayload::Code`, mirroring the deploy-time logic in `worker_lockfiles`: `bun` + `//native` is promoted to `bunnative` (tag `nativets`), and `bunnative` without `//native` is demoted back to `bun`. This makes a preview run exactly like the deployed script would, and covers every preview entry point (run_preview_script, inline preview, codebase preview) since they all go through `JobPayload::Code`. Adds regression tests asserting the queued job's `script_lang`/`tag` for all four (declared language × annotation) combinations. Fixes WIN-2007 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(backend): add sqlx cache for preview_native_tag test query The regression test's `sqlx::query!` for `v2_job` (tag, script_lang) needs a cached entry so `SQLX_OFFLINE=true` CI compiles it. Adds exactly one new cache file; no existing (OSS or EE) caches removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): trim preview native-tag tests to the essentials Keep the core regression (bun + //native → bunnative/nativets) and the guard that plain bun previews are unaffected. Drop the two bunnative- declared cases, which only re-verified the mirrored demote logic and weren't the reported issue. The shared query is unchanged, so the sqlx cache stays valid. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
b0c3b01d31 |
fix(cli): preserve user drafts on sync push and permissioned-as (#9381)
CLI deploys (sync push, set-permissioned-as) went through the same create/update endpoints as a UI "deploy from draft", which delete the draft at that path. That silently wiped teammates' in-progress drafts on every push. Add a transient skip_draft_deletion deploy flag (mirroring deployment_message) that the CLI sets; the backend then skips the DELETE FROM draft for scripts, flows, and apps. UI deploys are unchanged. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
42d2121af9 |
fix(queue): duration-weighted workspace fairness signal (#9329)
* fix(queue): bump EE ref to include worker_ping fairness signal The current ee-repo-ref.txt pointed to 31cda7c (an unrelated merge commit on the asset-graph-view-ee branch) instead of ddc9e80, which contains the workspace-fairness fix that switches the active-share signal from v2_job_queue.running=true to worker_ping. As a result cloud was still computing overload off the legacy signal, so a workspace with many in-flight/suspended flows (lancom01-prod, with 799 suspended flows × 3 v2_job_queue bookkeeping rows each = 2397 running-true rows) was flagged as 95% of cluster activity despite consuming zero worker slots. Bumping to ddc9e80 picks up the worker_ping-based signal, which naturally excludes (a) suspended jobs (no worker pinging them), (b) zombie running-rows from dead workers, and (c) flow/flownode orchestration rows that never run on a worker in the first place. * test(queue): seed v2_job rows + realistic durations for fairness helpers The new duration-weighted fairness algorithm joins v2_job_queue and v2_job_completed to v2_job for the `kind` filter (excluding flow bookkeeping) and reads `duration_ms` for the completed contribution. Update the test helpers to mirror that schema: * `insert_completed` now inserts a matching v2_job row (kind=script) and writes `duration_ms = 1000` with a 1-second [started_at, completed_at] interval, so each completed row contributes ~1 worker-second when fully inside the refresh window. * `insert_queued` likewise pre-inserts v2_job, sets `started_at` to NOW() - 1s when running=true (so running rows contribute ~1 worker-second by the time the refresh runs), and seeds v2_job_runtime.ping so the running side accrues real-time worker seconds (the algorithm bounds end-of-interval by ping). The zombie/suspended insert helpers are intentionally left without v2_job rows — the new algorithm's INNER JOIN excludes them, so they still correctly contribute zero worker-seconds. * chore(queue): bump EE ref to duration-weighted fairness algorithm Companion to windmill-ee-private#<TBD>: switch the EE workspace fairness aggregation from a count-based UNION (worker_ping snapshot + v2_job_completed count) to a worker-seconds aggregation sourced directly from v2_job_queue and v2_job_completed, with kind/suspend filters mirroring handle_zombie_jobs and per-row defenses against zombie inflation on both halves. * chore(queue): bump EE ref for fairness perf fix (inline window_start) * chore(queue): bump EE ref for fairness perf rewrite (driver-side flip) * update ee ref |
||
|
Ruben Fiszel
|
a6c51b146d |
ignore flaky fairness regression tests in CI (#9328)
`fairness_ignores_zombie_running_rows` and `fairness_ignores_concurrency_suspended_rows` panic intermittently in CI (both Linux and Windows runs). Mark them `#[ignore]` until the underlying flakiness is resolved. |
||
|
Ruben Fiszel
|
896add0350 | fix: improve workspace fairness | ||
|
Ruben Fiszel
|
8bf7fd2c92 | feat(queue): stochastic admission + EE availability of workspace fairness algorithm (#9321) | ||
|
hugocasa
|
2db1c0a1fc | fix: early return should consider failure_module result (#9241) | ||
|
Ruben Fiszel
|
4b1bea8aed |
fix: enforce auth guards on app component preview execution (#9235)
* fix: enforce auth guards on app component preview execution Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: guard previewed runnable path and worker tag in app preview Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: validate app_script id ownership and keep root push isolation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: scope app preview guards to operator check + referenced runnables Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: require jobs:run scope and tag check on app preview (apps:run escalation) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
dfeed9c5c2 |
fix: actionable error when a custom_path is taken by an app in another workspace (#9190)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
 windmill-internal-app[bot]
|
52960ca30a |
fix: reset parent_hash in auto_parent when all versions at path are archived (#9172)
* fix: reset parent_hash in auto_parent when all versions at path are archived * test: regression test for auto_parent with all versions archived --------- Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
f414ffc484 | fix: never mark failure/trigger/approval scripts as auto_kind=lib (#9168) | ||
|
Ruben Fiszel
|
33bf01b627 |
fix(python): preserve strings containing Infinity/NaN in result JSON (#9149)
* fix(python): preserve strings containing Infinity/NaN in result JSON * test(python): add sanity checks for Infinity/NaN in results * refactor(python): use string-aware regex callback for single-pass cleanup * refactor(python): compact regex + handle backslash-escape parity * perf(python): short-circuit cleanup when no Infinity/NaN/NUL in result |
||
|
Ruben Fiszel
|
d666e8431c |
feat: read-only flag on API tokens (#9144)
* feat: read-only flag on API tokens, orthogonal to scopes Add a per-token `read_only` boolean set at creation time. When true, the token can only call HTTP methods classified as Read (GET/HEAD/OPTIONS). Mutating methods and job-run actions are rejected with 403, regardless of which scopes are attached. Surfaced as a prominent toggle in the standard token-creation flow and a discreet `2xs` toggle in MCP mode (where users often want write access, so we don't bias them toward enabling it). MCP enforcement: read-only tokens hide all script/flow/hub tools from `list_tools` and only see endpoint tools whose method is GET, and the runner rejects `call_tool` on anything mutating. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: review fixes for read-only token flag - Exempt /api/mcp/* and /mcp/* paths from the read-only middleware check. MCP transport runs over POST (streamable HTTP / SSE), so otherwise the middleware would 403 every MCP request before the runner could enforce read-only at the tool-call level. - Tighten is_endpoint_read_only to GET only, matching the read_only_hint that create_endpoint_annotations actually emits. - Add unit test for check_read_only_for_route covering GET/HEAD/OPTIONS, mutating methods, and run paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to read-only-trigger-toggle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): make read-only toggle discreet in both modes Match the MCP-mode treatment in standard mode: text-tertiary, 2xs, shared "Read-only" label. The tooltip switches per mode so the explanation still fits the context. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): gate read-only toggle behind Limit token permissions The read-only toggle now only shows when the user has limited the token's scopes (standard mode) or in MCP mode (which always picks an MCP scope). Turning the limit off also resets read-only so it doesn't silently stick. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(frontend): hide incompatible MCP tools when read-only is on When the read-only toggle is on in MCP mode: - Endpoint badges and the custom-mode endpoint MultiSelect filter to GET. - Already-selected non-GET endpoints are pruned from the scope. - The scripts/flows preview is replaced with a note explaining they're hidden (the runner already rejects script/flow runs for read-only). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): place read-only toggle at top of limited scope area The previous gate required at least one scope to be picked before the read-only toggle appeared, which made it look missing while the user was still building their scope list. Move the toggle inside ScopesPicker: - Standard mode: sits directly under the "Limit token permissions" toggle whenever Limit is on, before the scope selector. - MCP mode: sits at the top of the MCP scope block. readOnly is now $bindable on ScopesPicker so CreateToken still owns the value. The auto-reset on un-limit moves into ScopesPicker too. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): nest read-only toggle inside the scope list card Place the read-only toggle at the top of the scope list (between the Selected Scopes summary and the bordered domain list) via a new optional topSlot snippet on ScopeSelector. Keeps ScopeSelector decoupled from read-only specifics; ScopesPicker fills the slot. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a This commit updates the EE repository reference after PR #571 was merged in windmill-ee-private. Previous ee-repo-ref: f53d26e6685dfd60bfa67686fbd7358169cfd130 New ee-repo-ref: 9bc8160be50b3e57a60daf4e1b71c389a6e02b8a Automated by sync-ee-ref workflow. * fix: address CI review for read-only token flag - P1 (Codex): narrow the MCP middleware exemption from "any /api/mcp/*" to just the streamable HTTP transport endpoints (/api/mcp/gateway, /api/mcp/w/{ws}/{mcp,sse,list_tools}). Without this, a read-only token could POST /api/mcp/gateway/oauth/server/approve and mint a follow-on non-read-only MCP token via the OAuth code/token exchange. - P2 (Claude/cubic): fix test comment/assertion mismatch — the run-path assertion now exercises GET (which is what the RUN_PATH_ACTIONS elevation comment describes) in addition to POST. Add a regression assertion for /api/mcp/gateway/oauth/server/approve. - P2 (cubic): short-circuit script/flow/hub-script/resource fetches in MCP list_tools when read_only is on — they would only be discarded below, so skipping the DB and resource fan-out is pure win. - P2 (cubic): when scopes are pre-supplied via the CreateToken prop, the ScopesPicker isn't rendered, which previously hid the read-only toggle entirely. Render it next to the pre-supplied scopes display. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
4d0f2c26a1 |
fix(bun): pass --preserve-symlinks on unbundled execution (#9147)
* fix(bun): pass --preserve-symlinks on unbundled execution Bun 1.2/1.3 moved its global package cache to a content-addressed layout and the installer now creates a single directory symlink from node_modules/<pkg> to the cache entry. Without --preserve-symlinks, Bun resolves modules from each file's realpath, so any require/import inside an installed package walks up from cache_nomount/bun/... and never finds the sibling deps living under <job_dir>/node_modules/. This manifested as e.g. ENOENT while resolving package 'zod/v3' from '/tmp/windmill/cache_nomount/bun/@langchain/core@1.1.44@@@1/dist/...' on //nobundling scripts that pull @langchain/core, even though zod is correctly installed alongside it in node_modules. The bundled execution path already had --preserve-symlinks since #4132 (needed because we symlink the cached bundle file into the job dir). The unbundled path didn't, because at the time Bun installed via per- file hardlinks and the realpath of node_modules entries was the job dir itself. The Bun installer's layout change made the flag necessary on the unbundled path as well. Add the flag to all three unbundled `bun run` invocations: - nsjail unbundled path - non-nsjail unbundled path - dedicated worker (always unbundled) This also fixes a latent bug on the first run of any bun script that imports a package whose internals reference siblings (the build_cache path runs unbundled this round while it builds the bundle for next time). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(bun): regression test for nobundling + transitive require resolution Adds an integration test that mirrors the original failure: a //nobundling script importing @langchain/core, which (in its CJS internals) does require('zod/v3'). Before --preserve-symlinks was added to the unbundled bun run invocations, this failed with: ENOENT while resolving package 'zod/v3' from '.../cache_nomount/bun/@langchain/core@<ver>@@@1/dist/runnables/base.js' The test covers the non-nsjail unbundled path. Reproducibility of the pre-fix failure depends on Bun's installer choosing the directory-symlink layout for the node_modules entry (the default on Bun 1.2/1.3+ with the new content-addressed global cache that produced the user's error). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
 Diego Imbert
|
4427a3d37f |
feat: add workspace-specific flag for resources and variables (#8836)
* feat: add workspace-specific flag for resources and variables Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove set_ws_specific endpoint and fix rust-client compilation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fall back to workspace name for ws_specific file naming When wsNameForFiles is not set (no wmill.yaml workspace config), ws_specific items would not get workspace-suffixed filenames during pull. Now falls back to workspace.name/workspaceId. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use workspace ID instead of CLI name for ws_specific file naming Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: pass workspace ID fallback to elementsToMap for ws_specific push Without this, workspace-specific files (e.g., a.admins.resource.yaml) were not recognized during push when no wmill.yaml or git branch was available, causing spurious deletions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ui nits * nit * Fix variable edit when only editing ws_specific * mark_linked_variables_ws_specific * Helper label * Support json format alongside yaml * Fix file naming push/pull asymetry & ws_specific orphans * Revert all CLI diffs * CLI now appends the remote ws_specific list to the local specificItems * UI for Env switcher * Refactor Resource/Variable editors to use dumb component * Refactor side effects * Editor works with multi workspaces * Fix can_save * Fix As JSON * nit * UI nits * list_ws_specific_versions as pl sql function to avoid round trips * UI Nits * Per-workspace version read-only check * fix: reset session context in list_ws_specific_versions to prevent RLS leakage The function calls set_session_context() in a loop. Although SET LOCAL is transaction-scoped (so settings revert at autocommit), defending against the function being invoked inside a longer outer transaction: - wrap the loop in a sub-block with EXCEPTION WHEN OTHERS that resets the session to a deny-default (windmill_user, empty session.* GUCs) before re-raising, - on the happy path, reset to the same deny-default at the end of the function. * feat: audit auto-marked ws_specific variables When a resource is saved as ws_specific, every variable referenced via $var: inside its value is auto-INSERTed into ws_specific. Previously this happened silently. Now: - mark_linked_variables_ws_specific takes the authed user, - the INSERT uses RETURNING path so we know exactly which variables were freshly flipped (not the ones already ws_specific), - each newly flipped variable gets a 'variables.set_ws_specific' audit entry pointing at the resource that triggered it. * perf: skip mark_linked_variables_ws_specific when nothing relevant changed update_resource was calling mark_linked_variables_ws_specific on every save when the resource was ws_specific, even on a description-only or label-only edit. Gate the call on `ns.value.is_some() || ns.ws_specific == Some(true)` so we only re-mark when the $var: refs could actually have changed or ws_specific was freshly enabled. * docs: explain asymmetric ws_specific toggle in resource tooltip Enabling the resource's 'Workspace specific' toggle silently marks every variable referenced via $var: inside the value as ws_specific, but disabling it does not un-mark those variables (they may be referenced by other resources). Surface this in the tooltip so users know what to expect. * fix: surface non-404 errors when fetching ws_specific items in CLI sync mergeWsSpecificFromServer was catching every error from listWsSpecific and logging it at debug. That's correct for old servers without the endpoint (404), but a 401/403/network failure would silently produce an incomplete sync. Now distinguish 404 (debug, expected) from everything else (warn with status + message) so users notice when the merge fails for real reasons. * perf: collapse compare_two_variables presence checks into one round-trip The early-return path was issuing four sequential EXISTS queries (ws_specific × {source, fork}, variable × {source, fork}). Combine them into a single SELECT so the per-variable diff cost drops ~4x. * sqlx prepare * docs: clarify has_sql_updates invariant in update_variable The else branch of the npath resolution is only reachable for non-rename edits (labels-only, ws_specific-only) because ns.path being Some always forces has_sql_updates=true at the top of the function. Add a debug_assert and a comment explaining the invariant so a future change that decouples ns.path from has_sql_updates trips immediately. Also use `path` directly instead of unwrap_or_default-ing ns.path, since we know it's None here. * chore: drop redundant ws_specific type augmentations ListableResource and ListableVariable from $lib/gen now include `ws_specific?: boolean` after the openapi.yaml additions in this branch were regenerated. The intersection types in resources/+page and variables/+page were duplicating the field — drop them. * Put WsSpecificVersions toggle in top drawer bar * nit size * feat: detect local-only ws_specific items on sync push When wmill.yaml lists a resource/variable in specificItems but the remote isn't yet marked ws_specific for that item, sync push silently dropped the flag because: 1. file-content diff alone never noticed (ws_specific is metadata, not YAML body) — push{Resource,Variable} were never called for those items; 2. even when called, isSuperset(local, remote) returned true and the early-return skipped the API call. Now: - mergeWsSpecificFromServer returns the raw server list alongside the merged config so push can compare 'in local' vs 'in server'; - a new computeWsSpecificFlagOnlyPushes helper walks the local file map, finds ws_specific-flagged paths absent from the server list, and the push function injects them as synthetic 'edited' changes (same before and after content) so the standard display + apply pipeline picks them up; - push{Resource,Variable} no longer early-return when content matches but the ws_specific flag differs. Pull is unaffected — only the push-side caller of mergeWsSpecific takes the new (merged, serverItems) tuple. * getDeployTo for selected ws * refactor: ws_specific kind handling, support .json files The ws_specific helpers had two warts: 1. computeWsSpecificFlagOnlyPushes hardcoded `.resource.{yaml,json}` / `.variable.{yaml,json}` magic strings, even though the existing getTypeStrFromPath / removeType helpers already do that work and already cover both extensions. 2. isSpecificItem / isItemTypeConfigured only matched `.yaml` paths, so users with opts.json local files got no specificItems coverage at all — patterns from wmill.yaml (and from mergeWsSpecificFromServer) are expressed with `.yaml`, and a `.json` file never matched. Changes: - Replace WS_SPECIFIC_KIND_MAP (a closed enum of resource+variable) with configKeyForItemKind, a generic kind→SpecificItemsConfig key mapping. Triggers fold into 'triggers' via the `_trigger` suffix, so adding a kind to the backend's list_ws_specific_versions doesn't require a CLI change. - mergeWsSpecificFromServer now appends `${item.path}.${item.item_kind}.yaml` through the same helper. - computeWsSpecificFlagOnlyPushes uses getTypeStrFromPath + removeType, gated by configKeyForItemKind. No more magic strings. - isSpecificItem and isItemTypeConfigured normalize trailing `.json` to `.yaml` once at the entry, so a single set of patterns covers both extensions for the same logical item. * refactor: dedicated change type for ws_specific flag-only pushes Previously the sync push code injected a synthetic 'edited' Change with before === after to nudge the apply loop into calling pushResource / pushVariable for ws_specific-flag-only diffs, and a guard inside those two functions skipped the early-return when the flag differed. The contract was implicit and easy to break — any future 'skip identical edits' optimization in the change pipeline would silently drop these pushes. Replace with an explicit Change variant: type WsSpecificFlag = { name: 'ws_specific_flag'; path: string; kind: string; wsSpecific: boolean; }; The push apply loop now has a dedicated branch for it that calls wmill.updateResource / updateVariable with just the ws_specific flag. prettyChanges renders it on its own line. The dry-run JSON output picks it up via the existing change.name / change.path passthrough. The defensive wsSpecificMatches check inside push{Resource,Variable} is no longer needed (sync push doesn't go through them for flag-only diffs) and is reverted. * drop folders * feat(cli): warn on remote ws_specific items missing from local config When 'wmill sync pull' fetches the server's ws_specific list, items the server marks as ws_specific but that aren't matched by the local wmill.yaml's specificItems patterns now produce a warning. The merge already preserves correctness (those items are still treated as ws_specific during this pull), but the user's config drifts from the remote — and a later push from another machine without that config would push the item as non-ws_specific. Surface the drift so the user can update wmill.yaml. Also filter ws_specific_flag changes out before preCheckPermissionedAs (it expects added/edited/deleted only and they have no content payload so on_behalf_of resolution doesn't apply). * fix(cli): scope ws_specific drift warning to items in this pull's changes Previously the warning iterated every ws_specific item the server returned, producing log spam for items unrelated to the current pull (items that exist locally with no change, or items the user has nothing to do with this round). Move the loop after compareDynFSElement and only warn for items whose path appears in the changes list — i.e., items the user is actually pulling right now. * fix: clean up linked-side ws_specific rows on resource/variable delete Three places left orphaned ws_specific rows behind: 1. delete_resource deleted the resource's own ws_specific row and the linked variables, but never the ws_specific 'variable' rows that mark_linked_variables_ws_specific had auto-inserted for those variable paths. 2. delete_variable deleted its own ws_specific row and the linked resource at the same path, but never a ws_specific 'resource' row at that path. 3. delete_resources_bulk didn't even cascade to linked variables, let alone clean up their ws_specific rows. A new resource or variable later created at one of those paths would silently inherit a stale ws_specific flag — list_ws_specific would report it as workspace-specific, workspace diffs would treat it as 'no changes', and CLI sync would skip it. Fix: - delete_resource: DELETE FROM ws_specific WHERE item_kind = 'variable' AND path = ANY(linked_var_paths) before the linked-variable delete. - delete_variable: DELETE FROM ws_specific WHERE item_kind = 'resource' AND path = path before the linked-resource delete. - delete_resources_bulk: collect $var: refs from each bulk-deleted resource (mirror of single delete), then delete ws_specific 'variable' rows AND the variable rows themselves. Brings bulk delete in line with single delete semantics, including the orphan cleanup. * fix: gate list_ws_specific by resource/variable RLS The endpoint queried ws_specific directly under user_db, but ws_specific itself has no per-item RLS — only a workspace-level column. Any workspace member could enumerate every ws_specific path including those in folders they lack read access to (e.g. f/finance/prod_db_creds), revealing path existence that list_resources / list_variables would have hidden. Add EXISTS clauses against resource and variable so the same path-based RLS policies that govern those tables (see_own / see_member / see_extra_perms_user / see_extra_perms_groups / see_folder_extra_perms_user) also gate visibility here. The user transaction already establishes the session context; the joins make the policies apply. * only resources and variables * fix(cli): make workspace-specific path mapping handle .json files isSpecificItem() was extended to normalize .json -> .yaml so .json files could be matched against patterns, but the surrounding helpers remained yaml-only: - toWorkspaceSpecificPath only mapped folder.meta.yaml / settings.yaml / .X.yaml — a foo.resource.json went through unchanged, so the workspace-specific filename was never produced. - fromWorkspaceSpecificPath only matched .yaml extensions — pushing foo.dev.resource.json could not map back to foo.resource.json. - isCurrentWorkspaceFile / isWorkspaceSpecificFile regexes ended in \.yaml$, missing every branch-specific .json file. Replace the literal '.yaml' anchors with '(yaml|json)' alternations, preserve the actual extension on round-trips, and rename the helper buildYamlTypePattern -> buildItemTypePattern (it never had anything extension-specific in it). getFileTypeSuffix now returns the matching suffix for either extension. Changed: - getFileTypeSuffix - toWorkspaceSpecificPath / fromWorkspaceSpecificPath - isCurrentWorkspaceFile / isWorkspaceSpecificFile - isTriggerFile / isScheduleFile isItemTypeConfigured / isSpecificItem don't need touching — their checks run after normalizeJsonToYaml(), which already collapses both extensions to .yaml at the entry. * fix: create_resource?update_if_exists=true honors ws_specific=false The upsert path matched on `unwrap_or(false)`, so an explicit `ws_specific: false` and an absent flag were indistinguishable — both fell through with no DELETE on the existing ws_specific row. Callers trying to clear the flag via PUT-with-update_if_exists silently saw their request ignored. Mirror update_resource's three-way handling: Some(true) -> INSERT (+ mark linked variables) Some(false) -> DELETE (only when update_if_exists, since a pure create has no existing row anyway) None -> leave the existing flag alone create_variable doesn't have an upsert path (no ON CONFLICT), so the same bug doesn't apply there. * sqlx prepare * test: cover ws_specific cleanup, RLS filtering, upsert clearing, and CLI .json paths Backend (backend/tests/ws_specific.rs + fixture): - test_linked_delete_cleanup: creates a ws_specific resource that references a variable via $var:, deletes the resource, asserts the cross-kind ws_specific row for the auto-marked variable is also removed. Then does the inverse for delete_variable, verifying the ws_specific 'resource' row at the same path is cleaned by variable delete. - test_list_ws_specific_filters_by_rls: admin creates ws_specific items in u/test-user/ and u/test-user-2/; verifies admin sees both via list_ws_specific while a non-admin (test-user-2) only sees their own path — the RLS see_own policy on the joined resource/variable tables hides the other. - test_create_resource_upsert_clears_ws_specific: covers the three-way Option<bool> handling on the upsert path: Some(true) inserts, Some(false) clears the existing row, None leaves it alone. CLI: - specific_items_unit.test.ts: add 14 tests covering toWorkspaceSpecificPath / fromWorkspaceSpecificPath / isWorkspaceSpecificFile / isCurrentWorkspaceFile / isSpecificItem / isItemTypeConfigured for .json files (variable, resource, trigger, schedule, folder.meta, settings). - ws_specific_flag_only_unit.test.ts (new): covers computeWsSpecificFlagOnlyPushes — emits flag-only changes only for resource and variable kinds (the backend's list_ws_specific_versions scope), does not emit for schedules or triggers, returns empty when serverItems is null (older server), respects existing server entries, preserves .json extension on filePath. - Export computeWsSpecificFlagOnlyPushes so it can be unit-tested. * perf: index workspace_settings.deploy_to for the recursive CTE list_ws_specific_versions's recursive CTE probes WHERE ws.deploy_to = r.ws_id every iteration; without an index on workspace_settings.deploy_to each iteration seq-scans the table — at 10M workspaces with the depth cap of 32 that's up to 320M row reads per call. deploy_to is sparse (most workspaces don't deploy anywhere), so a partial index WHERE deploy_to IS NOT NULL stays small while still covering every probe. Tucked into the existing migration since the function and the index ship together. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
e74f06cb56 |
fix: handle singlestepflow zombies and stop filtering them from runs page (#9055)
* fix: handle singlestepflow zombies and stop filtering them from runs page * fix: support singlestepflow in batch_rerun_jobs Previous PR added singlestepflow to list_selected_job_groups so the BatchReRun pane shows them, but batch_rerun_jobs_inner still joined on kind = 'script' / 'flow' with j.runnable_id (which is NULL for SingleStepFlow), so the rows were silently filtered out — user sees the option, click Re-run, gets zero successes. Mirror the norm_kind CTE projection from list_selected_job_groups inside batch_rerun_jobs_inner: pull the wrapped runnable type and pinned script hash from raw_flow.modules[id='a'], cast back to JOB_KIND so the existing handler dispatch works unchanged. Path-based schema fallback so input_transforms still resolve at rerun time. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: project singlestepflow in batch-rerun schema lookups Codex review pointed out two follow-on regressions from the previous fix: (1) list_selected_job_groups returned schemas with script_hash=null and schema=null for singlestepflow rows because the inner schemas subquery still joined runnable metadata via j.runnable_id (NULL for SingleStepFlow). The BatchReRun pane consumes every selected.schemas entry through mergeSchemasForBatchReruns / buildExtraLibForBatchReruns, both of which assume real schema objects. (2) When use_latest_version=true, batch_rerun_handle_job re-fetched latest_schema from v2_job filtering jb.kind='script' or 'flow' — neither matched singlestepflow, so schema came back NULL and every input_transforms entry silently no-op'd. Both queries now project singlestepflow rows via raw_flow.modules[id='a'] — norm_kind for dispatch and effective_hash for the schemas join, plus a path-based latest-schema fallback so flow-wrapped SSF (no version pinning) and any SSF whose pinned hash has been deleted still resolve. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: add batch_rerun integration tests, fix SSF hash hex parsing Adds 11 integration tests against /jobs/run/batch_rerun_jobs and /jobs/list_selected_job_groups (both endpoints had zero CI coverage). Tests cover the full 4-kind × 3-mode matrix: regular Script and Flow (baseline regression for the SQL refactor), script-wrapped and flow- wrapped SingleStepFlow (regression for the bugs this PR fixes), and a mixed-kind batch. Writing the tests caught a real bug in the previous commit: ScriptHash serializes as a 16-char hex string in raw_flow.modules[a].value.hash (per the custom Serialize impl in windmill-types/scripts.rs), not as an integer. The earlier `(m->'value'->>'hash')::bigint` cast worked on the hand-inserted SQL fixture I'd used for live testing (which embedded the hash as a raw integer) but failed in production where all SSF jobs are pushed via JobPayload::SingleStepFlow's serialized form. Replaced with `('x' || lpad(hex, 16, '0'))::bit(64)::bigint` — preserves the twos-complement bit pattern so both positive and negative i64 hashes round-trip correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update SQLx metadata --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
5bca03eacd |
fix: bubble handle_flow chaining errors to parent flow (#9058)
* fix: bubble handle_flow chaining errors to parent flow * test: while-loop propagates inner forloop iterator failure |
||
|
Ruben Fiszel
|
eebaab9c87 | fix(bun): propagate non-zero exit from generate_bun_bundle on no-DB path (#9051) | ||
|
hugocasa
|
6e5a21a9c7 |
fix(flows): inherit flow_env in sub-flow predicates (#9042)
* fix(flows): inherit flow_env in sub-flow predicates Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(flows): align flow_env lookup with get_root_job_id and tighten gate Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(flows): drop recursive CTE, root_job propagation suffices Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(flows): walk via flow_innermost_root_job to respect imported-flow scope Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(flows): remove flow_env API endpoint, dead code from deno_core era Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
1174d7d77f |
refactor: replace SELECT * with explicit column lists (#9010)
* refactor: replace SELECT * with explicit column lists Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update sqlx offline query cache * chore: update sqlx offline query cache * chore: update sqlx offline query cache with EE support * chore: update sqlx offline query cache, no deletions * chore: update sqlx offline query cache after rebase * fix: correct column names in explicit script query lists - concurrency_limit → concurrent_limit (matches DB column name) - runnable_settings → runnable_settings_handle (matches DB column name) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing delete_after_secs column to script queries Also add integration test covering all explicit-column export queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add workspace export integration test covering all explicit-column queries Covers tarball_workspace (folder, script, resource, resource_type, variable, schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add tarball export integration test covering all explicit-column queries Single test creates one of each entity type and exercises every runtime-checked explicit-column query in tarball_workspace. Uses archive_type=tar to avoid zip feature-gate in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: fix stale verification step and CI contradiction in update-sqlx skill - Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff - Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: remove Co-Authored-By from commit skill template * refactor: extract SCRIPT_COLUMNS const to single source of truth Replaces 5 duplicated 44-column lists with a shared const in windmill-types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
45d959a49e | nit tests | ||
|
Ruben Fiszel
|
28b5e3382c | chore: fix cargo checks | ||
|
Ruben Fiszel
|
96bc00007b | chore: fix cargo checks | ||
|
Ruben Fiszel
|
0f26418f4a | fix cargo checks | ||
|
Ruben Fiszel
|
0c22f52b46 |
feat: support assigning a worker tag to app inline scripts (#9002)
* feat: support assigning a worker tag to app/raw-app inline scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: omit empty tag field from inline script raw_code payload Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: shrink tag popover width --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
aedf369174 |
fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999)
* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(pg): wrap encoder errors with arg context, add fallback test Followups on #8999 review: - Wrap rust-postgres "error serializing parameter N" failures with the arg name, JSON value kind, and asserted Postgres type plus a hint about an explicit cast — so users see actionable context instead of an opaque WrongType. - Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree on the Type for every recognised arg_t when the JSON value matches its natural Rust kind. Catches future drift if either side changes. - Integration test for the prepare + query_raw fallback path: confirms unrecognised arg_t (custom enum) is routed through prepare and the server-resolved type appears in the failure surface — flips into a test failure if a regression accidentally routes unrecognised types through query_typed_raw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add otyp_inferred flag + regex-based placeholder renumbering Two follow-ups from the review of #8999: 1. **Issue #1 (Number/Bool + explicit text decl in WHERE)** Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets it `true` only at the "no info → fall back to text" site (bare `$N`, no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep it `false`. In `convert_val` this flag distinguishes: - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works (`text = text` operator). Pre-#8988 behaviour, restored. - parser-default text (bare `$N`) — bind the value's natural Rust type so the regression case (`Value::Bool` against a real `bool` column via `CAST AS bool`) keeps working. `Arg` is in `windmill-parser`; the new field has `#[serde(default)]` so persisted signatures stay backward-compatible. 2. **Issue #4 ($5/$50 substring rewrite collision)** Replace the per-index `String::replace` chain (which turned `$50` into `$10` when oidx=5 was processed first) with a single regex pass. `\d+` is greedy, so `$5` and `$50` match as distinct units; indices outside the mapping are left intact. 3. Tests: - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline- cast/decl/mixed shapes. - executor unit: `convert_val_bool_against_every_arg_t` and `convert_val_*_number_*` split each text-like target into explicit vs inferred expectations. - executor unit: `renumber_sparse_placeholders_no_collision`. - integration: `test_postgresql_arg_type_combinations` adds 4 cases covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and sparse positional args ($5/$50). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality Backend: 1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s `ToSql for String` / `FromSql for String` reject `Kind::Enum` and `Kind::Domain` even though the wire format is plain UTF-8. The wrapper accepts those kinds in both directions. End result: explicit `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col` results come back as JSON strings instead of erroring at the FromSql layer. 2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text / non-temporal arg_t fell through to `Box<String> + TEXT`, which then failed at the server (no implicit cast text→numeric in expression context). Now strings are parsed into the matching native type with clear error messages on parse failure. 3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering (which fixed the `$5/$50` substring collision but still walked through string literals and comments, mangling `'price: $5'` etc.) with a walk over `parse_pg_statement_arg_positions` — the same string/comment/dollar-quote-aware tokenizer used for index discovery. Adds `parse_pg_statement_arg_positions` to the parser's public API. SDK: 4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now stringifies bigints before serialisation; the executor accepts numeric strings into BIGINT arg slots via the existing `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split so `BigInt` always resolves to `BIGINT` (was reaching `Number.isInteger(BigInt)` which returns false → wrong default). 5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers primitive types only (number / bigint / string / boolean); mixed or nested arrays still fall back to JSON. Mixed int/float widens to `DOUBLE PRECISION[]`. 6. **`.query()` positional bug**: previously the `.query()` method abused the template-tag builder, which appended `$N::TYPE` after the user's literal SQL string instead of binding by position (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()` builds the executor-shaped content directly: a `-- $N argN (TYPE)` declaration block followed by the user's SQL verbatim. Tests: - Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments` asserts string literals, comments, and dollar-quoted blocks don't produce positions (so renumbering doesn't mangle them). - Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling` uses the new position-aware path and includes string-literal + comment + `$$…$$` cases. Existing convert_val tests grow to cover new String→numeric/real/double/oid/bool arms. - Integration: `test_postgresql_arg_type_combinations` adds 13 cases (enum round-trip both directions, string→numeric/real/double/bool/oid, string-literal `$N` non-mangling). The prepare-fallback test now asserts SUCCESS (not failure) for enum encoding via AnyTextValue. - SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests) exhaustively covering inferSqlType primitives + arrays, parseTypeAnnotation, datatable() template tag (with all the new shapes — BigInt, homogeneous arrays, RawSql, schema preamble), datatable().query() positional, and ducklake() shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache) Found while exhaustively probing custom-type DX: every cached-connection reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL` deallocates *all* prepared statements server-side — including the typeinfo statements that tokio_postgres caches per-Client to resolve custom enum / domain Oids. tokio_postgres still held `Statement` objects whose names the server had forgotten, so the next custom-type query failed with intermittent "prepared statement \"sN\" does not exist" errors. The failure was easy to reproduce: any sequence that forced typeinfo lookup for two different custom-type kinds on the same cached connection (e.g. enum followed by domain) would hit it. Replace `DISCARD ALL` with a curated reset that explicitly targets the state we actually care about, *without* touching prepared statements: RESET ALL — GUC parameters (search_path, application _name, statement_timeout, …) RESET SESSION AUTHORIZATION — undoes both `SET SESSION AUTHORIZATION` and `SET ROLE` (RESET ALL does NOT — these aren't GUC parameters, so without this an elevated role from a previous job would silently leak) UNLISTEN * — drops LISTEN registrations CLOSE ALL — closes open cursors Trade-off: temp tables, advisory locks (session-scoped), and user-created PREPARE statements may persist across cached-connection reuse — rare in datatable / PG-script workloads. tokio_postgres's typeinfo cache survives intact, so custom enum / domain queries are fast on subsequent reuse. Tests: - `test_postgresql_custom_types_on_cached_connection` — runs 10× alternating enum + domain queries on a cached connection. Pre-fix this failed with `prepared statement "sN" does not exist` after the first reuse; post-fix passes. - `test_postgresql_set_role_does_not_leak_across_cached_connection` — switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres role, then runs a follow-up job and asserts current_user/session_user are restored. Specifically catches the case where someone might switch back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION AUTHORIZATION) and silently introduce a permission-leak vector. - All existing session-isolation tests (`test_postgresql_cached_connection_resets_session`, `test_postgresql_single_worker_session_isolation`, `test_postgresql_100_jobs_cached`) continue to pass. Found via end-to-end probing of datatable / PG-script DX, not previously covered: the existing isolation tests only did `SET ROLE postgres`, the connecting user, so the leak was invisible. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): address PR #8999 review (cubic + claude) cubic (P1, real bug): - `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but chrono `NaiveTime` only encodes for TIME (same caveat as the scalar arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz` assignment cast at the column site. Add an explicit unit test. claude (#1, silent failure → explicit error): - `Bool` + explicit `(char)` / `(character)` decl previously silently bound BOOL, hoping the server would cast at the use site — but PG has no implicit `bool→char` and the resulting error ("operator does not exist: bool = char") was opaque. Now error at bind time with an actionable hint to use `bool` decl or pass the value as a "t"/"f" string. claude (#2, asymmetry doc): - Object/Array still coerce to text on `matches!(typ, Typ::Str(_))` (covers both explicit AND inferred-default text), unlike Bool/Number which key on `explicit_text_target`. The asymmetry is intentional (no implicit `jsonb → text` cast in expression context vs PG having implicit `bool/int → text` casts) — added a body comment so future maintainers don't try to "align" them. claude (#3, perf): - `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions` walked the SQL tokenizer twice. Fold into a single pass that derives the index set from the position list. claude (#4, fmt drift): - `cargo fmt` over the parser crates I touched with perl scripts in the earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py, rust,graphql,yaml,r}). Net cosmetic. claude (#5, parseTypeAnnotation): - One-line caveat in the SDK's `parseTypeAnnotation` that the returned string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a real PG type, but the only consumer just checks `!== undefined`). While here — discovered + fixed independently while exhaustively probing DX: - **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing intermittent `prepared statement "sN" does not exist` errors on custom-type queries after cached-conn reuse. New regression tests: `test_postgresql_custom_types_on_cached_connection` and `test_postgresql_set_role_does_not_leak_across_cached_connection` (the latter catches the case where someone might switch back to `RESET ALL` alone and silently introduce a permission-leak vector — RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION). - **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00") and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") — neither parseable by `date-fns parseISO`, JavaScript `new Date()` is lenient enough to handle them but several frontend `App*Input.svelte` components use parseISO and fail silently. Switched to ISO-8601 with `T` separator and `+00:00` offset; arg-parsing path still accepts the legacy " UTC" suffix for back-compat. Test coverage: - 17/17 unit (`pg_executor::tests`) - 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`) - 27/27 parser (`windmill-parser-sql`) - 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling Found while probing PG-script DX with millions of numeric cells: 1. **Numeric precision-loss warning**: `numeric` results are still serialised as JSON Number (back-compat — switching to JSON String would silently break user code doing arithmetic on results), but we now detect `Decimal -> f64 -> Decimal` round-trip failure and emit a single job-log warning recommending a `::text` cast in the SQL. Bounded by `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic load + one fetch_sub on the hot path; first lossy value short-circuits to a single load thereafter). Worst-case overhead on a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic loads (vs. ~100ms unbounded). 2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"` (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is what the apps `App*Input.svelte` components use, so timestamp values silently failed to round-trip into date pickers. Switch to ISO-8601 (`T` separator + `+00:00` offset) on the result side; arg-parser continues to accept the legacy `" UTC"`-suffixed format for back-compat. 3. **Float NaN / Infinity results**: `Number::from_f64` returns None for NaN / ±Inf, which `pg_cell_to_json_value` was raising as "invalid json-float" — failing the *entire* query if any cell held one of these special values. Now serialise them as JSON strings ("NaN", "Infinity", "-Infinity") and let the rest of the row come through. Arg-side: `s.parse::<f64>()` already accepts the same strings. Tests: - `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit cases for the precision-loss predicate. - `precision_check_budget_caps_per_query_overhead` — locks in the budget cap and the loss-flag short-circuit. - All 9 PG integration tests + 17 unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults While probing PG-script DX further found three more frictions: 1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;` meant session-scoped advisory locks (`pg_advisory_lock`) leaked across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()` to the chain — `DISCARD ALL` covered this implicitly via `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it in the switch. 2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g. `-- $1 amount (numeric)`) but not provided in the args object was bound as NULL with no error / warning. Misspelling the key in the args object silently produced a row of NULLs — a notorious DX debugging trap. Now: collect the names of declared-but-missing args during dispatch and emit a single one-shot warning to the job logs at end-of-query naming each one. Bound NULL is preserved for back-compat. 3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries `arg.default = Some(Number(5))`, but the dispatch fell straight to NULL when the arg was missing. Now: respect the default — user-supplied value > declaration default > NULL. Also fixes the warning logic above (only warn for args that *don't* have a default). Tests: existing 19 unit + 9 integration pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values Two more frictions found while probing SDK end-to-end against a real datatable resource: 1. **Multi-word array types lose the [] suffix in the parser**. `transform_types_with_spaces` recognises aliases for "double precision", "character varying", "timestamp with time zone", etc. but its return type was `&'a str` — only the bare alias, never with a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at the first space, so the regex's own `(?:\[\])?` array-suffix branch sees only `"double"` (not `"double precision[]"`); the `[]` was silently lost. Result: `$1::double precision[]` (which the SDK now emits for homogeneous float arrays via the new auto-tag) routed through `Value::Array → Type::JSONB` and the server failed with "cannot cast type jsonb to double precision[]". Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>` and re-check the trailing bytes after a multi-word match. If they start with `[]`, return `format!("{alias}[]")` — Owned. Single-word types and the no-match path keep returning Borrowed slices, so no allocation in the hot path. 2. **Array arms in `convert_vec_val` rejected stringified values for numeric / int* / bool / oid / real / double**. The scalar `convert_val` already parses strings into the matching native type for these arg_ts, but the array variant only accepted JSON-native counterparts. Sending `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with "Mixed types in array". Now the array arms mirror the scalar ones — `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes round-trip cleanly. Tests: 19 unit + 9 integration pass; existing parser tests cover the multi-word array forms (the regex-cap behaviour didn't break for single-word types, and Cow plumbing is transparent to all callers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files CI failures: the perl-driven sweep that added `otyp_inferred: false` to every `Arg { ... }` literal when I introduced the field in the parser schema covered `src/lib.rs` files but missed: - parsers/windmill-parser-bash/src/lib.rs (mass-edited but a later format pass un-applied a few sites) - parsers/windmill-parser-go/src/lib.rs (same) - parsers/windmill-parser-graphql/src/lib.rs (same) - parsers/windmill-parser-nu/tests/tests.rs (test file — not swept the first time) - parsers/windmill-parser-ts/tests/tests.rs (test file — same) Also tightened the regex to handle `oidx: None` without the trailing comma (some test files had the field as the last initialiser line). `cargo build --features <CI feature combo> --workspace --all-targets` is clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string Two more frictions found while running the actual SDK end-to-end against a live datatable resource: 1. **JS `Date`** fell into the typeof "object" branch and was tagged `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's `json → text → timestamptz` implicit cast chain, but `${date}` against a `timestamptz` column without a user-supplied cast bound the value as a JSON string and the comparison `timestamptz = json` failed. Now: `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`; `serializeArgValue` emits `Date.toISOString()` so the executor's `Value::String → TIMESTAMPTZ` arm parses it cleanly. 2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)` returns `"null"` per the JS spec, so the value reached the executor as JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL double. Fix: detect non-finite numbers in `serializeArgValue` and stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals directly, and the result-side already renders the values as JSON strings (matching round-trip). SDK unit tests grow from 42 → 44 passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(pg): integration coverage for multi-word arrays + stringified array elements Locks in the two array fixes from the previous commit (`fix(pg): multi-word PG types with [] suffix lost the array-ness`) with end-to-end cases in `test_postgresql_arg_type_combinations`: - `double precision[]`, `character varying[]`, `timestamp without time zone[]` — verifies the parser keeps the `[]` suffix after multi-word alias resolution. - `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies the array arms of `convert_vec_val` apply the same string-coercion the scalar arms do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: fix indentation drift on otyp_inferred lines cargo fmt cleanup of leftover indentation where the perl-driven sweep that introduced the otyp_inferred field landed at the wrong column. No behaviour change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
abcd920964 |
test: isolate WAC v2 python test from stack overflow (#8979)
* test: isolate WAC v2 python test from test-thread stack overflow Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * ci: bump RUST_MIN_STACK to 4MB for backend tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
hugocasa
|
c95642863e |
feat: support restart from steps inside BranchOne, ForLoop, Subflow (#8955)
* feat: support restart from steps inside BranchOne, ForLoop, Subflow Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: preserve original job kind in nested restart, support expanded subflow steps Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: read selected iteration from graph state for nested ForLoop restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: iteration selectors per ForLoop in restart popup, more nested restart tests Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract useNestedRestartState composable Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: cover deployed-subflow + FlowDependencies path in nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update sqlx prepare cache Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: detect BranchOne/ForLoop ancestors inside expanded subflows for nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hide restart button for non-restartable steps (parallel containers, untaken branches) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review feedback on nested restart PR - preview FlowRestartButton: hide nested case (chain UUIDs aren't resolvable in preview path; users can use the run page for nested restart instead) - branchOneAncestorMatchesOriginal: be permissive when status isn't reachable (don't hide the button for BranchOnes nested deeper than top-level) - worker_flow.rs: apply nested_restart_payload swap on the is_simple ForLoop fast path too, so simple iterations don't bypass restart spawn interception - FlowStatusViewer: reset expandedSubflows cache on jobId change; drop $bindable({}) banned pattern for the new prop - API resolver: validate the leaf step exists before returning (fail-fast) - doc fix: branch_or_iteration_n is 0-based, not 1-based - selectedJobStepIsTopLevel reset on early-return in composable - comment iterationCounts collision caveat - new HTTP-level integration tests covering the API endpoint contract: happy path (top-level + nested), unknown step, out-of-range iteration, parallel-loop rejection Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert: remove unreachable nested-restart swap on is_simple ForLoop fast path The swap is unreachable in valid flows: `is_simple_modules` requires the body to be a single `script` / `rawscript` / `flowscript` (per `FlowModule::is_simple`), none of which spawn flow-kind children. Any nested-restart chain targeting a leaf inside such an iteration is rejected by the API at leaf validation. Even if a chain reached the worker via `JobPayload::RawFlow.restarted_from`, the resulting `RestartedFlow` would fail to push (script kind isn't a flow kind). Replaced the swap with an explanatory comment so the next reader knows why the symmetry with the non-simple path was deliberately not added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: handle undefined expandedSubflows + tighten branchOne match check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
581658d881 |
fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor (#8951)
* fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor Three workflow-as-code bug fixes: - #8945: Python WAC template with `@workflow async def main(...)` was not detected as `auto_kind = "wac"`. The detection only ran when no `main` function was found. Hoist the heuristic so it runs whether or not `main` is the entrypoint. - #8946: `scripts/list?kinds=script` filtered out WAC scripts because they set `auto_kind = 'wac'` and the SQL hid everything that wasn't NULL. Allow both NULL and 'wac' (still excluding 'lib' library scripts). - #8947: Preprocessor functions defined alongside a WAC workflow were ignored. Inject the preprocessor invocation into the Python WAC wrapper so it runs before the workflow on the first iteration, then plumb the preprocessed args through `handle_wac_v2_output` so inline child re-runs see the post-preprocessor args via `checkpoint.input_args`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(wac): integration tests for #8946 (scripts/list) and #8947 (preprocessor) - test_scripts_list_includes_wac: hit GET /scripts/list?kinds=script and assert WAC scripts are in the response (would have failed pre-#8946 fix because of the auto_kind IS NULL filter). - test_python_wac_v2_with_preprocessor: deploy a Python WAC script with a preprocessor, run with raw event args, assert the workflow saw the preprocessed shape and v2_job.args/preprocessed were updated. - New wac_preprocessor.sql fixture with auto_kind = 'wac' set explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(wac): address PR review feedback Five review fixes: - python_executor.rs: WAC preprocessor now runs inside the wrapper's `try:` block so failures route through the same `result.json` error serializer as workflow failures. Switched async-coroutine handling from deprecated `asyncio.get_event_loop().run_until_complete(...)` to `asyncio.run(...)` (the recommended primitive on 3.10+). - bun_executor.rs: when copying preprocessed args into `checkpoint.input_args`, surface JSON parse failures via `?` instead of silently coercing to `Value::Null` (which would persist a corrupted arg into every child re-run). Also collapsed the redundant double iteration into a single pass. - windmill-api-scripts/scripts.rs: switched the runnable-script filter from an allow-list (`auto_kind IS NULL OR = 'wac'`) to a deny-list (`<> 'lib'`), so future `auto_kind` values aren't silently filtered from triggers/dropdowns. - windmill-parser-py: aligned the parser's WAC heuristic with the runtime detector `is_wac_v2_py` — `@task` is now optional, matching the runtime which says workflows that only use inline `step()` are still WAC. Added a regression test `test_parse_python_wac_step_only`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5b3913052e |
refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert atomic primitive globals to AtomicBool/AtomicI64 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to arcswap-refactor EE branch commit * refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit * chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private. Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4 New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d Automated by sync-ee-ref workflow. * fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private. Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7 New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c Automated by sync-ee-ref workflow. * fix: secret_backend_integration test — BASE_URL.write().await → .store() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to e587df8 (post-#535 merge) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
64c58c824f |
feat: add deploy restriction rule and fork review requests (#8804)
* feat: add deploy restriction rule and fork review requests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for fork review requests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review comments on fork review requests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename fork review requests to deployment requests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for deployment request rename Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: inline deployment request panel into deploy layout Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: place Request deployment button to the left of Deploy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: inline fork triggers into main deploy list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: open real trigger detail drawer for inline fork triggers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: email notifications for merge completion and reply pings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update deployment_request + protection_rule tables on workspace id rename Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 972893c3870e4c4a70a35748abed282d88904805 This commit updates the EE repository reference after PR #528 was merged in windmill-ee-private. Previous ee-repo-ref: 5684d1c17d930b17849c1e5d7577891e64682d45 New ee-repo-ref: 972893c3870e4c4a70a35748abed282d88904805 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
b3ef4bc26c | perf: add inline-persist fast path for WAC v2 step() (#8807) | ||
|
Ruben Fiszel
|
60211c1d19 |
feat: folder default_permissioned_as rules for ownership defaults on deploy (#8801)
* feat: add folder default_permissioned_as rules for ownership defaults on deploy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unnecessary auth guard on default_permissioned_as — rules are advisory only Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts with new CLI commands Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CI review findings — TOCTOU, race condition, email validation, type coercion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add sqlx offline cache for test queries (fixes cargo_test CI) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review findings — incomplete request bodies, dead code, redundant import Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review findings — full script fields, reactive stores, catch-all validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: app/schedule/trigger set-permissioned-as fetch remote first to avoid data loss Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: app set-permissioned-as avoid creating redundant app version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: compact user/group toggle + select for folder default_permissioned_as rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: collapse default_permissioned_as section by default in folder editor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: include default_permissioned_as in FolderFile CLI type for YAML round-trip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: process folder.meta changes before items in push to apply new rules immediately Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: clone default_permissioned_as on fork/rename + add full lifecycle tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add no-op guarantee test — folder without rules behaves like before Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename cliBehavior to syncBehavior — more accurate scope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
ec9cec1d02 |
fix: treat empty global setting strings as unset (#8793)
* fix: treat empty global setting strings as unset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: close protected-setting whitespace gap in diff and preserve empty ws override Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
4fff89f98c |
fix: hide legacy global_settings.worker_configs ghost row (#8790)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
c57c769dea |
feat: add CI test scripts with auto-trigger on deploy (#8736)
* feat: add CI test scripts with auto-trigger on deploy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: fix annotation parser early return and handle renames correctly Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move CI test results to top of script/flow detail pages Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve CI test results spacing, icon, and remove pass label Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: support one-line annotation and use script/path format Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: move CI test trigger logic to EE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: move CI badge next to New badge and add deduplicated CI summary Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add CI test e2e tests and fix nullable column annotations Add integration tests for CI test annotation parsing (creates/removes ci_test_reference rows) and the CI test results API (single + batch endpoints). Add backend test for auto-trigger on deploy (private+python). Fix sqlx LEFT JOIN LATERAL nullable column annotations in get_ci_test_results and get_ci_test_results_batch queries — sqlx cannot infer nullability from LATERAL subqueries, causing runtime decode errors when no matching job exists. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix build/sqlx * fix * feat: CI test improvements and templates - Fix windmill-dep-map/private feature propagation in worker, api-scripts, and api-flows Cargo.toml so CI test triggers actually fire in EE mode - Clone ci_test_reference rows during workspace fork - Add polling to CiTestResults component (refetch every 3s while running) - Add running state and auto-refresh to ForkWorkspaceBanner CI summary - Add yellow "CI test" badge on script list rows and detail page - Fix Library badge border color (remove indigo border override) - Add CI Test TypeScript and CI Test Python templates in ScriptBuilder - Update sqlx offline cache - Add debug tracing for CI test trigger in worker_lockfiles Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing children prop to WorkspaceDeployLayout Fixes svelte-fast-check type error when passing named snippets as children content inside the component tag. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback - Remove empty wrapper divs around CiTestResults, move mb-4 into component - Add batch endpoint size cap (max 200 items) - Add ON DELETE CASCADE to ci_test_reference workspace FK (new migration) - Downgrade CI test trigger logs from info to debug - Fix false-positive polling: only treat status='running' as running, not null status (CiTestResults, CompareWorkspaces, ForkWorkspaceBanner) - Fix test numbering in integration tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to d9d68c2406df0b59f413ea0b2cb24780a9817d04 This commit updates the EE repository reference after PR #516 was merged in windmill-ee-private. Previous ee-repo-ref: d7ccd9b86da99ec056a0e8708e3637d64290387a New ee-repo-ref: d9d68c2406df0b59f413ea0b2cb24780a9817d04 Automated by sync-ee-ref workflow. * fix: treat queued jobs (job_id set, null status) as running Jobs that have been pushed but not yet picked up by a worker have a job_id but null status. Treat these as 'running' to avoid showing misleading 'pass' badges or '0 passing'. Tests that were never triggered (no job_id, null status) remain neutral/hidden. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
fa668707c0 |
fix: move alert config from config table to global_settings (#8762)
* feat: move alert config from config table to global_settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename alert setting to alert_job_queue_waiting Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add CLI unit tests for pullInstanceConfigs/pushInstanceConfigs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt to merged main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
2d18a68099 |
feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
 Yoaquim Cintrón
|
7bf6ac2b69 |
fix: enrich OTEL spans with job_kind, trigger_kind, trigger, created_by, and script_hash (#8718)
Add five new attributes to the `job` and `job_postprocessing` tracing spans so that OTEL-consuming backends (Sentry, Honeycomb, Datadog, etc.) can filter and group telemetry by how a job was triggered and what type it is. New span attributes: - `job_kind` — Script, Flow, AppScript, AIAgent, Preview, etc. - `created_by` — the user or system identity that queued the job - `trigger_kind` — schedule, webhook, kafka, http, sqs, etc. - `trigger` — the schedule/trigger path (when applicable) - `runnable_id` — the id of the runnable that ran Also adds `JobKind::as_str()` for a consistent lowercase string representation, following the same pattern as `ScriptLang::as_str()`. Existing attributes (job_id, workspace_id, script_path, language, tag, flow_step_id, parent_job, root_job) are unchanged. Note: the EE `full_job` span in `otel_ee.rs` and the log records emitted by `job_logger_ee.rs` would also benefit from these attributes. This PR covers only the public-repo spans; a follow-up EE change would propagate the same fields to logs and the full_job span. |
||
|
Ruben Fiszel
|
6d58d1a74d |
fix: pipeline DISCARD ALL with first query on cached pg connections (#8707)
* perf: pipeline DISCARD ALL with first query on cached pg connections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use RESET ALL instead of DISCARD ALL for lighter session reset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration test for pg session reset on cached connections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: release MutexGuard before caching so pg connection cache actually works The old code shadowed the MutexGuard variable without dropping it, so try_lock() in the post-query caching path always failed — connection caching was effectively dead code. Restructure to explicitly drop the guard before connecting. Also adds a CACHE_HITS counter and clear_pg_cache() helper so the integration test can verify the cached-connection path is exercised. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add single-worker session isolation test for SET ROLE + search_path Pushes 3 jobs into the queue before starting the worker so a single worker processes them all sequentially (matching production). Verifies SET ROLE and SET search_path do not leak between jobs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add RESET ROLE to session reset (RESET ALL does not undo SET ROLE) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use DISCARD ALL for full session reset and retry on stale connections - Switch from pipelined RESET ROLE; RESET ALL to eager DISCARD ALL when validating cached connections. This resets everything: role, GUCs, prepared statements, temp tables, advisory locks, LISTEN registrations. - DISCARD ALL also serves as a health check: if it fails, the stale connection is discarded and a fresh one is created transparently. - Extract new_pg_connection() helper to avoid duplicating the connect + spawn-connection-task logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add 100-job single-worker cache stress test Runs 100 varied PG jobs (plain SELECTs, SET ROLE, SET search_path, multi-statement) through one worker. Verifies all succeed, 99 hit the cache, and no session state leaks between jobs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
adc9fe722d |
fix: gate relock_skip tests on private feature and update ee-repo-ref (#8703)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
ba214709b9 |
fix: add secretKeyRef support for jwt_secret and rsa_keys (#8698)
* feat: add secretKeyRef support for jwt_secret and extra fields (rsa_keys) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update ee-repo-ref to 2c24cf597fdf8c4dccd483f1f1e5c49eb42ef3a3 This commit updates the EE repository reference after PR #508 was merged in windmill-ee-private. Previous ee-repo-ref: ade3bb76f8e0a6e658313b54c7180577fc9efc37 New ee-repo-ref: 2c24cf597fdf8c4dccd483f1f1e5c49eb42ef3a3 Automated by sync-ee-ref workflow. * test: replace unit tests with integration tests for jwt_secret and rsa_keys Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
bffa61e33f |
fix: dedicated worker dispatch, cross-workspace deps, UI improvements (#8689)
* feat: restore bun as default runtime for dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add context comment for bun dedicated worker nodejs migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for flows + add E2E tests - Add workspace_id prefix to dedicated worker map lookup keys - Update ee-repo-ref for dedicated worker path handling fix - Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers - Add 6 E2E tests for dedicated workers: - test_dedicated_flow_rawscript (regression for "Script not found" bug) - test_dedicated_flow_workspace_script - test_dedicated_flow_multiple_steps - test_dedicated_standalone_script - test_dedicated_runner_group - test_dedicated_flow_runners - Add dedicated_flows.sql fixture with scripts, flows, and worker config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always run dependency job for dedicated worker scripts When a script with dedicated_worker=true is deployed with a pre-computed lock (e.g. via wmill sync push), no dependency job was created, so the dedicated worker never detected the update and kept running the old version. Now dedicated worker scripts always generate a dependency job regardless of whether a lock is provided. The dependency job runs on the dedicated worker and triggers a restart so it picks up the new script version. Fixes #8638 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races Dedicated worker tests need non-default worker tags in the global WORKER_CONFIG. When run in parallel (CI uses --test-threads=10), multiple tests clobber each other's config. Use #[serial] to ensure dedicated worker tests run sequentially. Also load worker config from DB via load_worker_config() instead of manually setting WORKER_CONFIG fields, ensuring consistency with the monitor's reload path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests Fix script_path shadowing in bun_executor nodejs branch where the wrapper file path was passed to handle_dedicated_process instead of the logical path, causing "Script not found" for all //nodejs dedicated workers. Add E2E tests for dedicated flows in all supported languages: - test_dedicated_flow_deno - test_dedicated_flow_python - test_dedicated_flow_bunnative (V8 PrewarmedIsolate path) - test_dedicated_flow_bun_nodejs (//nodejs annotation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify dedicated worker dispatch + add serialization and E2E tests - Unified lookup: always use {workspace}:{runnable_path} for dedicated worker dispatch, replacing the flow_step_id iteration approach - Added serialization_semaphore parameter to executor start_worker fns - Added E2E tests: cross-workspace isolation, conflicting flow step IDs, preprocessor on dedicated worker - Added workspace field to RunJob for cross-workspace test support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: cross-workspace workspace dependencies on workers page Add two new instance-level endpoints to the configs router: - GET /configs/list_all_workspace_dependencies - GET /configs/list_all_dedicated_with_deps Both require devops role and return data across all workspaces, enabling the workers page to show a consistent view of which workspace dependencies exist regardless of which workspace the user is browsing. Update DedicatedWorkersSelector to use the new cross-workspace endpoints with fallback to per-workspace calls for non-devops users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include dedicated worker lookup simplification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update ee-repo-ref.txt with new reference * sqlx * fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings - Remove serialization_semaphore from executor start_worker signatures - Remove serialization test and fixtures - Fix DedicatedWorkersSelector to preserve tags from other workspaces when toggling in the picker - Track workspace deps per-workspace for conflict detection - Show warning when dep exists in another workspace but not the script's - Group runner groups per-workspace to prevent cross-workspace merging - Add workspace to dep badge link URL Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify exec protocol — execd: for single-script, exec: for runner groups Add execd:/execd_preprocess: commands to bun/deno/python wrappers for single-script dedicated workers (no path needed). Runner groups keep exec:/exec_preprocess: with path for multi-script disambiguation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add unit tests for execd:/exec: wrapper protocol Verify generate_multi_script_wrapper produces both execd: (single-script) and exec: (runner group) protocol handlers, including preprocessor variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update commit reference in ee-repo-ref.txt * fix: remove beta badge from squash loop, keep tooltip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update protocol tests to use execd: for single-script wrappers Deno and bun single-script protocol tests now send execd:{args} instead of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script (runner group) tests continue to use exec:{path}:{args}. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused TEST_SCRIPT_PATH in deno protocol tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — down migration, push_as workspace, UI improvements - Use regexp_replace in down migration for positional accuracy - Fix push_as() to use self.workspace_id instead of hardcoded value - Remove per-workspace API fallbacks, use cross-workspace endpoints only - Skip devops-only API calls when user is not devops (disabled prop) - Fix duplicate key error for cross-workspace runner groups - Add workspace to RunnerGroup for unique keying - Reuse tagRow snippet for standalone items with expand/collapse - Fix picker alignment: remove empty column for non-expandable items Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive dedicated worker test coverage, fix Python execd_preprocess - Add Python execd_preprocess: handler (was missing for single-script dedicated workers) - Add 10 E2E tests: flow+standalone conflict, mixed lang fallback, unsupported lang flow runners, python runner group, bun/python/deno/bunnative preprocessors, runner group preprocessors, branchone flow - Add 4 Python unit tests for execd:/execd_preprocess: protocol - Update EE ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — migration escaping, deno try/catch, loadRunnables guard - Down migration: use E'...' so \n matches actual newlines - Up migration: anchor regex with ^ to avoid mid-content matches - Deno execd_preprocess: move JSON.parse inside try/catch - DedicatedWorkersSelector: skip devops-only API calls when disabled Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add dedicated worker relative import tests for bun and python Verifies that build_loader's CURRENT_PATH correctly resolves workspace- relative imports when running on a dedicated worker subprocess. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for nested flow structures (branches/loops) - Add extract_flow_root() to strip nesting segments from runnable_path - Dispatch uses flow_root/flow_step_id for nested paths, runnable_path for flat paths — deterministic, O(1) - Fix assert_ran_on_dedicated_worker to BFS all descendants - Fix python mode labels (python vs python3 for runner groups) - Add tests: simple forloop, multi-step forloop, whileloop, branchall, nested branch-in-loop, mixed lang fallback, unsupported lang runners Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: fix ee-repo-ref SHA Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide picker and skip API calls for read-only users, hide empty runner badge - Hide "Add more scripts/flows" section when disabled (read-only) - Skip per-runnable API calls (getScriptByPath, getFlowByPath) for disabled users — just show path info - Hide "0 runners" badge on flows with no eligible steps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 9422b189762ae27edfc346541ae668a4ad728325 This commit updates the EE repository reference after PR #503 was merged in windmill-ee-private. Previous ee-repo-ref: 4c6ba214bfc23fff05d1dc3200ac59e650af3f4f New ee-repo-ref: 9422b189762ae27edfc346541ae668a4ad728325 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
c4c9ef5fd7 |
feat: add optional labels to scripts, flows, apps, schedules, triggers (#8609)
* feat: add optional labels to scripts, flows, apps, raw apps, schedules, and triggers
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update sqlx cache, make labels optional in openapi, regenerate system prompts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add minimal labels input UI to script, flow, and schedule editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce gap between summary and labels input
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to script/flow detail pages and summary/path popover
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move labels inside SummaryPathDisplay trigger for clickable area, reduce gap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: display labels inline to the right of summary, not below
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase gap between summary and labels
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to resources/variables, make labels nullable, add home page label filter badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to workspace export/import, resources, variables + test coverage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make migration idempotent, regenerate sqlx cache after merge
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in script create and flow create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels input UI to resource and variable editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove negative margin from LabelsInput to prevent overlap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add top and left margin to LabelsInput for better spacing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce left margin on LabelsInput
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: widen label input to w-32
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use inline-flex so LabelsInput doesn't stretch full width
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove flex-wrap so label input stays on same line as badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label filter presets to resources, variables, and schedules search
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use max-w-32 on label input to prevent stretching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pull labels closer to summary with negative top margin
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase negative margin to pull labels even closer to summary
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in schedule create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use COALESCE to preserve existing labels when not provided in schedule/flow update
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add labels to CreateResource, EditResource, CreateVariable, EditVariable in OpenAPI spec
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on resource and variable list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on schedule and all trigger list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add folder and label presets to schedules search filter
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: apply user_folders_only filter on all workspaces including admins
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label presets to resources and variables search filters
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: derive folder presets from loaded items, not all workspace folders
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add label query parameter to resource and variable list endpoints in OpenAPI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label filter badges inline with folder filters on home page
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat: display label filter badges inline with folder filters on home page"
This reverts commit
|