mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-19 16:02:14 +00:00
f9cfeb0dbaac53fe92dbc21d99570e69ef8e40bf
532 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
765f50c474 |
feat: folder-level label inheritance for scripts, flows and jobs (#9524)
* feat: folder-level label inheritance for scripts, flows and jobs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use SECURITY DEFINER folder_labels() for RLS-consistent inheritance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: extend folder label inheritance to apps, resources, variables, schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
 hugocasa
|
f2f0812a04 |
feat(flows): opt-in to include the stopping step's result in early-stop errors (#9446)
* feat(flows): early stop can include the stopping step's result in the raised error
When a step uses Early Stop with "Raise an error message if stopped", the
flow result was entirely replaced with a static error object
({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the
stopping step's own output. This made it impossible to stop+fail a flow
while preserving the data the step produced (e.g. an API that returns
HTTP 200 with a userErrors payload).
Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on
the raise-error path, the raised payload becomes
{"error": {...}, "result": <step result>} instead of dropping the result.
Default is false, so existing behavior is unchanged. The option is threaded
through the worker's stop-after-if handling (including stop_after_all_iters_if
for loops/branchall) and exposed in the flow editor's Early Stop panel.
Fixes WIN-2012
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover early-stop error_include_result payload shaping
Add a regression test asserting that a step using Early Stop with a raised
error message and error_include_result=true fails the flow while preserving
the step output as {"error": {..}, "result": <step result>}, and that with
the flag off the result is the bare {"error": {..}} object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(flows): nest early-stop step result inside the error object
Embed the stopping step's result under `error.result` rather than as a
top-level sibling of `error`. This keeps the flow result shape as
`{ "error": { .. } }` — identical to a normal error — so consumers that
key off the top-level shape (single `error` key) keep working, while the
data is still preserved for those that look inside the error object.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): always include the stopping step's result in early-stop errors
Drop the opt-in `error_include_result` gate. Since the step result is nested
inside the error object (`error.result`), the top-level result shape stays
`{ "error": .. }` — identical to a normal error — so consumers that detect or
parse failures by the top-level shape are unaffected. Gating it added schema
surface, plumbing, and a UI toggle for no real compatibility benefit.
Now, whenever a step early-stops with a raised error message, the flow fails
and the raised error embeds the stopping step's own result under
`error.result` (aggregated iteration results for loops/branchall). This
reverts the `StopAfterIf.error_include_result` field, its threading, the
OpenAPI/generated-client surface, and the editor toggle; the "Raise an error
message" tooltip now notes that the step result is included.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(flows): gate early-stop result inclusion behind opt-in flag
Re-introduce the per-step `error_include_result` flag (default off) instead
of always embedding the step result. Although nesting the result under
`error.result` keeps the result *shape* backward-compatible, it does not
address data exposure: a failed flow's result is propagated to synchronous
webhook callers, the flow's failure module, and the workspace/global error
handler (commonly a Slack/email/outbound-webhook notifier). Always including
the step output would surface previously-redacted intermediate data to all of
those sinks for every existing error-stop flow.
Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and
only embeds `error.result` when the flow author explicitly opts in, matching
the original issue's intent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(flows): omit error_include_result when false; refresh generated prompts
- Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result`
so serialized flows are byte-identical when the flag is off. Fixes the
`flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing
flows.
- Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts`
for the new OpenFlow `error_include_result` property. Fixes check-freshness.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(flows): cover error_include_result for the loop "stop after all iters" path
Add a regression test for the stop_after_all_iters_if branch, where `nresult`
already holds the aggregated iteration results — confirming `error.result`
carries each iteration's output (distinct from the per-step fallback path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
73edebc833 |
fix(backend): route //native TypeScript previews to native workers (WIN-2007) (#9407)
* fix(backend): route //native TypeScript previews to native workers Previewing a TypeScript script carrying the `//native` annotation was pushed with `language = bun` (what the editor sends), so the job was tagged `bun` and routed to a regular bun worker. A native-mode worker neither matches the `bun` tag nor accepts a non-native `script_lang` (worker.rs rejects with "cannot execute non-native job with language 'bun'"), so previewing a `//native` script on a native-only worker setup failed — even though the deployed version of the same script runs fine as `bunnative` / tag `nativets`. `push` now reconciles the preview language with the `//native` annotation for `JobPayload::Code`, mirroring the deploy-time logic in `worker_lockfiles`: `bun` + `//native` is promoted to `bunnative` (tag `nativets`), and `bunnative` without `//native` is demoted back to `bun`. This makes a preview run exactly like the deployed script would, and covers every preview entry point (run_preview_script, inline preview, codebase preview) since they all go through `JobPayload::Code`. Adds regression tests asserting the queued job's `script_lang`/`tag` for all four (declared language × annotation) combinations. Fixes WIN-2007 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(backend): add sqlx cache for preview_native_tag test query The regression test's `sqlx::query!` for `v2_job` (tag, script_lang) needs a cached entry so `SQLX_OFFLINE=true` CI compiles it. Adds exactly one new cache file; no existing (OSS or EE) caches removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): trim preview native-tag tests to the essentials Keep the core regression (bun + //native → bunnative/nativets) and the guard that plain bun previews are unaffected. Drop the two bunnative- declared cases, which only re-verified the mirrored demote logic and weren't the reported issue. The shared query is unchanged, so the sqlx cache stays valid. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f0301b1605 |
feat(flows): preserve step/subflow worker tags under a custom-tagged flow (#9375)
* feat(flows): preserve step/subflow worker tags under a custom-tagged flow A flow running on a custom worker tag force-propagates that tag to every descendant step, script and nested sub-flow, overriding their own declared tags. This made it impossible to route a specific step or sub-flow to a different worker group. The new opt-in FlowValue.preserve_step_tags lets a step that declares its own non-empty tag run on it; untagged steps still inherit the flow tag. Defaults off to preserve existing behavior. * chore: regenerate system prompts for preserve_step_tags Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(flows): nest preserve_step_tags toggle under flow worker tag setting The toggle only affects routing when the flow has a custom worker tag, so show it as a sub-setting of the Worker Group tag picker, visible only once a tag is set, instead of as a standalone option. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): allow step worker tag picker when preserve_step_tags is enabled When a flow defines a worker tag, the per-step tag picker was replaced by a read-only "Flow's WG" label. With preserve_step_tags enabled the step's own tag is honored, so the picker must remain editable in that case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): propagate preserve_step_tags to branch and loop bodies payload_from_modules built the synthetic RawFlow for branch/loop bodies with a default FlowValue, dropping preserve_step_tags. Tagged steps inside a branch or loop therefore still inherited the parent flow tag even with the flag enabled. Thread the flag through to the synthetic FlowValue so the behavior is consistent for nested containers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): clear preserve_step_tags when flow worker tag is removed Avoids the flag lingering as invisible state after the flow tag (and its toggle) are removed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): repair preserve_step_tags propagation to branch/loop bodies The previous commit added flow.preserve_step_tags at the payload_from_modules call sites but the parameter and FlowValue field were not actually threaded through (a failed edit left the function unchanged), so the crate did not compile. This completes the change: payload_from_modules takes preserve_step_tags and sets it on the synthetic FlowValue for branch/loop bodies. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): complete preserve_step_tags propagation to branch/loop bodies Previous two commits left windmill-worker uncompilable: payload_from_modules received flow.preserve_step_tags at its call sites but the parameter and the synthetic FlowValue field were not actually added. This adds the parameter, sets preserve_step_tags on the synthetic FlowValue, and threads flow.preserve_step_tags through all five call sites. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(flows): clear preserve_step_tags whenever the flow worker tag is removed The flag was only reset when the Worker Group toggle was switched off, not when the tag was cleared directly in the picker (or via the YAML editor), leaving preserve_step_tags=true as invisible state with the advanced badge still reporting it active. Move the cleanup into the reactive block that already tracks the flow tag so every clear path is covered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
045d12043e |
feat(queue): duration-weighted fairness admission (#9334)
* [ee] feat(queue): duration-weighted fairness admission atomic Add the `WORKSPACE_FAIRNESS_ADMISSION_PPM` atomic that the EE `workspace_fairness_ee::refresh_overloaded` writes on each refresh (see companion EE PR). The atomic is read on every pull by `should_admit_capped` to decide whether the dispatch goes down the standard or fairness path. Defaults to 10_000 (= admit all) so the pre-fairness behaviour is preserved until the first refresh fires. OSS stub in `workspace_fairness.rs` continues to return `true` unconditionally, so non-EE builds are bit-identical. * docs(queue): consolidate full fairness algorithm into workspace_fairness.rs Move the algorithm doc — what "overloaded" means in worker-seconds, the duration-weighted admission derivation, coordinated refresh structure, audit emission, the SQL perf constraints (no params CTE, drive running side from v2_job_runtime), and EE gating — into the OSS surface module where it is readable without EE access. The EE file becomes implementation only. Also bump ee-repo-ref to the EE commit that strips the duplicate doc. * docs(queue): clarify ADMISSION_PPM default is "admit all", not count-based Addresses CI review (claude[bot]): the `10_000` initial value is the "admit all" no-op default that applies before the first refresh classifies an overloaded set — not the count-based value (which would be `target * 10_000`). The count-based form is the empty-bucket fallback inside `compute_admission_ppm`, a different thing. * chore(queue): point ee-repo-ref at EE main (fairness admission merged via #593) * fix(queue): duration-weighted admission uses unclamped service-time window Bumps ee-repo-ref to the EE fix (windmill-ee-private#596) that sources D_c/D_u for the admission probability from a separate 60s service-time window of true `duration_ms`, instead of the occupancy aggregation whose per-job contributions are clamped to the 10s occupancy window. The clamp truncated D_c for capped jobs longer than the window, under-admitting the duration skew (true 34s jobs → ~86% effective share instead of the target 65%). Occupancy worker-seconds still drive overload classification. Updates the algorithm doc in workspace_fairness.rs accordingly. Note: ee-repo-ref points at the EE feature branch; re-point to EE main once #596 merges. |
||
|
Ruben Fiszel
|
8bf7fd2c92 | feat(queue): stochastic admission + EE availability of workspace fairness algorithm (#9321) | ||
|
Ruben Fiszel
|
577a730e90 |
audit-log workspace-fairness cap transitions (#9306)
* feat(queue): audit-log workspace-fairness cap transitions When the cloud per-workspace fairness mechanism adds a workspace to the capped set or releases one, write `workspace_fairness.capped` / `workspace_fairness.uncapped` audit-log entries to the affected workspace. The cluster admin can review the full timeline from the `admins` workspace audit view with `all_workspaces=true`; per-workspace owners see their own events in their normal audit list. Only the per-cycle refresh winner emits entries (matching where the heavy aggregation runs), so a fleet of N workers does not produce N duplicates per transition. The diff is computed against the value already in `background_task_state` rather than the winner's in-memory cache, so a freshly-restarted process winning the claim does not spuriously emit "newly capped" entries for workspaces that were already capped before it started. Audit writes are best-effort: failures are logged via tracing and do not abort the refresh cycle. Fixes WIN-1984 * feat(queue): scope fairness audit to admins workspace + queue-metrics pane - Write `workspace_fairness.capped` / `workspace_fairness.uncapped` to the `admins` workspace (was: per-affected-workspace) with the affected workspace_id moved to the `resource` field. Cluster admins now get the full timeline in one place without `all_workspaces=true`. - Add `GET /workers/workspace_fairness_events` returning the last 100 events. Cloud-gated (returns `[]` on non-cloud) and devops-only. - Add a `WorkspaceFairnessEvents` Section to the Queue Metrics drawer, rendered only when `isCloudHosted()` is true. Shows time / event badge / workspace / parameters with a refresh button. Fixes WIN-1984 |
||
|
Ruben Fiszel
|
de2e243313 |
feat(queue): per-workspace fairness cap on the shared cloud worker pool (#9303)
* feat(queue): cloud-only per-workspace fairness cap on the shared worker pool
On `app.windmill.dev` the cluster runs a single default worker group, so a
single workspace flooding the queue can degrade quality of service for
everyone else. This adds an opt-in mechanism that caps any single workspace
at a configurable share of the shared worker pool when it has been
dominating cluster activity for more than a configurable window.
Detection signal counts both currently-running jobs and jobs completed in
the rolling window, so it catches workspaces hogging slots with long jobs
**and** workspaces spamming many tiny jobs (where no individual job's
started_at is old, but throughput share dominates).
Refresh is coordinated cluster-wide via a single UPDATE on
`background_task_state`: the `WHERE updated_at < now() - interval` predicate
combined with row-level locking means only one process per refresh cycle
actually runs the aggregation, regardless of fleet size. Every other
process gets the freshly written value in the same round trip via
`UNION ALL ... LIMIT 1`. Heavy aggregation rate stays at ~0.2-0.5 qps for
the whole cluster.
Pull queries are split: the existing query string and its bind shape stay
bit-identical to today, so the planner keeps using the same indexes when
fairness is off or no workspace is currently capped. A separate
`WORKER_PULL_QUERIES_FAIRNESS` adds `AND workspace_id <> ALL($2::text[])`
and is only materialized while the feature is enabled.
Hard-gated to `CLOUD_HOSTED=true` + BASE_URL host == app.windmill.dev at
three layers: frontend `cloudonly: true`, API setter rejection in
`set_global_setting_internal`, runtime check in `fairness_active`. Settings
are exposed under Jobs in the instance-settings UI; defaults are off so
the change is a no-op for self-hosted.
Two-pass pull guarantees no worker idling: if every queued job belongs to
a capped workspace, the second pass uses the unmodified pull queries.
Cap re-asserts on the next refresh.
Fixes WIN-1982
* fix(queue): address CI review findings on workspace fairness
Six fixes from the four-reviewer cross-check on #9303:
1. **Aggregation evaluation (Codex P1).** The previous `INSERT ... ON CONFLICT
DO UPDATE WHERE updated_at < ...` had the heavy `v2_job_queue ∪
v2_job_completed` aggregation inlined into `VALUES`, which Postgres
evaluates for every contender to build the proposed row — losing the
"one heavy aggregation per cycle cluster-wide" property the design
advertises. Split into three small statements: (a) cheap claim with
constant `VALUES`, (b) winner-only `UPDATE ... SET value = jsonb_build_object('overloaded', <agg>)`
(Postgres only evaluates `SET` per row matching `WHERE`, so losers never
compute the aggregation), (c) read for everyone. Heavy query now truly
runs ~0.2-0.5 qps cluster-wide regardless of fleet size.
2. **Numeric setting wraparound (cubic P1).** `u64 as u32` and downstream
`u32 as i32` could silently flip sign and feed `make_interval(secs => -N)`,
making `now() - interval` a future timestamp and disabling the
completed-jobs half of the activity signal. Clamp `duration_secs` to
[1, 86400] and `min_total_jobs` to [0, u32::MAX] before storing.
3. **`/instance_config` bypass (cubic/Claude/Codex P2).** Bulk config endpoint
sidestepped `set_global_setting_internal`'s gate; a self-hosted superadmin
could persist `workspace_fairness_*` rows via the bulk path. Mirror the
per-key check in `set_instance_config` upsert flow.
4. **DB error coerced to false (Claude P2).** `load_workspace_fairness_enabled`
collapsed `Err(_)` to `false` and unconditionally swapped the atomic — a
transient DB blip during notify-event propagation toggled the feature off
cluster-wide (and triggered a `store_pull_query` rebuild precisely when load
is highest). Now propagates the error so the atomic stays at its prior value.
5. **Refresh failure cooldown (Claude P2).** Storing `0` removed the rate
limit entirely; every subsequent pull spawned a new refresh task. Leave
`LAST_REFRESH_MICROS` at `now_us` (already written by the CAS) so the
natural interval acts as the cooldown.
6. **Visibility + duplication (Pi P2).** Mark `make_pull_query_fairness` as
`pub(crate)`. Move the duplicated `BASE_URL host == app.windmill.dev`
parser into `windmill-common::worker::is_cloud_production_host` and share
it between the API setter and the runtime path.
Verified locally:
- `POST /api/settings/global/workspace_fairness_enabled` → 400 (per-key gate)
- `PUT /api/settings/instance_config` with fairness key → 400 (bulk gate)
- `cargo check --workspace --features=private,enterprise,quickjs` — clean
Refs WIN-1982.
* fix(queue): second round of CI review nits on workspace fairness
Three issues raised by the Codex/Claude re-review of commit
|
||
|
hugocasa
|
dd5320205f |
feat: parse windmill_failure field to tag run as failure (#9073)
* feat: parse windmill_failure field in job result to tag run as failure * feat: preserve top-level fields when windmill_failure tags a run as failure * fix: address review findings on windmill_manual_failure * refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases * fix: prefer injected ManualFailure error over sibling name/message in OTel |
||
|
Ruben Fiszel
|
e3cc258455 | fix(queue): cap worker pull loop at 10 to avoid DB storm (#9062) | ||
|
Ruben Fiszel
|
1174d7d77f |
refactor: replace SELECT * with explicit column lists (#9010)
* refactor: replace SELECT * with explicit column lists Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update sqlx offline query cache * chore: update sqlx offline query cache * chore: update sqlx offline query cache with EE support * chore: update sqlx offline query cache, no deletions * chore: update sqlx offline query cache after rebase * fix: correct column names in explicit script query lists - concurrency_limit → concurrent_limit (matches DB column name) - runnable_settings → runnable_settings_handle (matches DB column name) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing delete_after_secs column to script queries Also add integration test covering all explicit-column export queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add workspace export integration test covering all explicit-column queries Covers tarball_workspace (folder, script, resource, resource_type, variable, schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add tarball export integration test covering all explicit-column queries Single test creates one of each entity type and exercises every runtime-checked explicit-column query in tarball_workspace. Uses archive_type=tar to avoid zip feature-gate in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: fix stale verification step and CI contradiction in update-sqlx skill - Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff - Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: remove Co-Authored-By from commit skill template * refactor: extract SCRIPT_COLUMNS const to single source of truth Replaces 5 duplicated 44-column lists with a shared const in windmill-types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
0c22f52b46 |
feat: support assigning a worker tag to app inline scripts (#9002)
* feat: support assigning a worker tag to app/raw-app inline scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: omit empty tag field from inline script raw_code payload Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: shrink tag popover width --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
hugocasa
|
c95642863e |
feat: support restart from steps inside BranchOne, ForLoop, Subflow (#8955)
* feat: support restart from steps inside BranchOne, ForLoop, Subflow Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: preserve original job kind in nested restart, support expanded subflow steps Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: read selected iteration from graph state for nested ForLoop restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: iteration selectors per ForLoop in restart popup, more nested restart tests Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract useNestedRestartState composable Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: cover deployed-subflow + FlowDependencies path in nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update sqlx prepare cache Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: detect BranchOne/ForLoop ancestors inside expanded subflows for nested restart Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hide restart button for non-restartable steps (parallel containers, untaken branches) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review feedback on nested restart PR - preview FlowRestartButton: hide nested case (chain UUIDs aren't resolvable in preview path; users can use the run page for nested restart instead) - branchOneAncestorMatchesOriginal: be permissive when status isn't reachable (don't hide the button for BranchOnes nested deeper than top-level) - worker_flow.rs: apply nested_restart_payload swap on the is_simple ForLoop fast path too, so simple iterations don't bypass restart spawn interception - FlowStatusViewer: reset expandedSubflows cache on jobId change; drop $bindable({}) banned pattern for the new prop - API resolver: validate the leaf step exists before returning (fail-fast) - doc fix: branch_or_iteration_n is 0-based, not 1-based - selectedJobStepIsTopLevel reset on early-return in composable - comment iterationCounts collision caveat - new HTTP-level integration tests covering the API endpoint contract: happy path (top-level + nested), unknown step, out-of-range iteration, parallel-loop rejection Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * revert: remove unreachable nested-restart swap on is_simple ForLoop fast path The swap is unreachable in valid flows: `is_simple_modules` requires the body to be a single `script` / `rawscript` / `flowscript` (per `FlowModule::is_simple`), none of which spawn flow-kind children. Any nested-restart chain targeting a leaf inside such an iteration is rejected by the API at leaf validation. Even if a chain reached the worker via `JobPayload::RawFlow.restarted_from`, the resulting `RestartedFlow` would fail to push (script kind isn't a flow kind). Replaced the swap with an explanatory comment so the next reader knows why the symmetry with the non-simple path was deliberately not added. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: handle undefined expandedSubflows + tighten branchOne match check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
 wendrul
|
0773b5bc5d |
fix: workspace specfic tags compatibility with forked workspaces (#8850)
* fix: workspace specfic tags compatibility with forked workspaces * Rename _db to db and use saved WM_FORK_PREFIX * Add ttl cache for mapping fork id to parent workspace id * Change second option to just have a -fork suffix |
||
|
Ruben Fiszel
|
0798719256 | nit key check | ||
|
Ruben Fiszel
|
362ae248fe |
fix: per-branch concurrency key for promotion-mode git sync (#8844)
* [ee] fix: per-branch concurrency key for promotion-mode git sync jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref for per-branch concurrency key Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 7bc0fdb8647268c7afa67b4b0bed69c897eaf92a This commit updates the EE repository reference after PR #537 was merged in windmill-ee-private. Previous ee-repo-ref: b933874649a63c5266a33360a95e3c163acc6b5f New ee-repo-ref: 7bc0fdb8647268c7afa67b4b0bed69c897eaf92a Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
3aa279cfd7 | nit tx commit cj | ||
|
Ruben Fiszel
|
5b3913052e |
refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert atomic primitive globals to AtomicBool/AtomicI64 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to arcswap-refactor EE branch commit * refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit * chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private. Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4 New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d Automated by sync-ee-ref workflow. * fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private. Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7 New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c Automated by sync-ee-ref workflow. * fix: secret_backend_integration test — BASE_URL.write().await → .store() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to e587df8 (post-#535 merge) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
2d18a68099 |
feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
wendrul
|
5125263859 |
add tracings for long debounce key errors (#8747)
* Add messages about debug key * update ee repo ref * undo b * update merged ee repo ref |
||
|
wendrul
|
eb32206940 | cloud debounce keys potential 'value too long' error (#8750) | ||
|
Ruben Fiszel
|
c4c9ef5fd7 |
feat: add optional labels to scripts, flows, apps, schedules, triggers (#8609)
* feat: add optional labels to scripts, flows, apps, raw apps, schedules, and triggers
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update sqlx cache, make labels optional in openapi, regenerate system prompts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add minimal labels input UI to script, flow, and schedule editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce gap between summary and labels input
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to script/flow detail pages and summary/path popover
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move labels inside SummaryPathDisplay trigger for clickable area, reduce gap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: display labels inline to the right of summary, not below
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase gap between summary and labels
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to resources/variables, make labels nullable, add home page label filter badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to workspace export/import, resources, variables + test coverage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make migration idempotent, regenerate sqlx cache after merge
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in script create and flow create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels input UI to resource and variable editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove negative margin from LabelsInput to prevent overlap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add top and left margin to LabelsInput for better spacing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce left margin on LabelsInput
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: widen label input to w-32
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use inline-flex so LabelsInput doesn't stretch full width
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove flex-wrap so label input stays on same line as badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label filter presets to resources, variables, and schedules search
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use max-w-32 on label input to prevent stretching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pull labels closer to summary with negative top margin
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase negative margin to pull labels even closer to summary
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in schedule create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use COALESCE to preserve existing labels when not provided in schedule/flow update
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add labels to CreateResource, EditResource, CreateVariable, EditVariable in OpenAPI spec
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on resource and variable list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on schedule and all trigger list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add folder and label presets to schedules search filter
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: apply user_folders_only filter on all workspaces including admins
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label presets to resources and variables search filters
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: derive folder presets from loaded items, not all workspace folders
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add label query parameter to resource and variable list endpoints in OpenAPI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label filter badges inline with folder filters on home page
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat: display label filter badges inline with folder filters on home page"
This reverts commit
|
||
|
Ruben Fiszel
|
da8886be85 |
feat: add configurable preview job tag override in default tags settings (#8649)
* feat: add configurable preview job tag override in default tags settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: skip re-tagging for FlowPreview jobs when preview override is active Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
 Pyra
|
010753c73a |
fix: skip debounce arg accumulation when batch table is empty (CE) (#8485)
On CE (without private feature), v2_job_debounce_batch is never populated because maybe_debounce_post_preprocessing is EE-only. The accumulation query returns zero rows, producing an empty array that replaces the original nodes_to_relock value. This causes flow modules to never get relocked when triggered by relative imports. Fix: only replace the original value when the batch query actually returned entries to accumulate. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
hugocasa
|
efb4a27d51 |
fix: replace email with permissioned_as for triggers/schedules (#8439)
* refactor: replace email with permissioned_as for triggers/schedules
Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.
- Migration: add permissioned_as to all 9 trigger tables + schedule,
drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix sqlx/build
* update ee ref
* refactor: simplify resolve_edited_by to always use authed username
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix compile + migration
* update ref
* test: add trigger trait method tests for permissioned_as queries
Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* update sqlx
* fix: use permissioned_as directly for schedules and fix audit RLS for groups
- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as field
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix build
* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule
- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
explaining it carries emails for flows/scripts and permissioned_as for
triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
just backwards compat, only permissioned_as matters)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: preserve email column when permissioned_as is preserved on schedule edit
Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update deploy UI labels from "edited by" to "run as" for triggers
Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use u/username format for custom trigger/schedule deploy selection
When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: show u/username format for "me" option in trigger deploy selector
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: simplify OnBehalfOfSelector to return the right format per kind
OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username
The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: always show u/username format in OnBehalfOfSelector for all kinds
Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: replace email with permissioned_as in http_trigger test insert
The email column was dropped from trigger tables in the migration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: review fixes — migration, app policy, capture cleanup, naming
- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to all test schedule INSERTs
Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: strip permissioned_as from exports/sync, fix OpenAPI required field
- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove accidentally committed generated files
These directories are gitignored and should not be tracked.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: regenerate system prompts for permissioned_as schema changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove permissioned_as from CLI TriggerFile Omit list
Already stripped in workspace export, no need to also omit from the type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: optimize email cache key and revert TriggerFile Omit change
- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: zero-allocation email cache lookups using Equivalent trait
Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add permissioned_as to Schedule required fields in OpenAPI spec
The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: handle group- prefix in migration UPDATE statements
edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "fix: handle group- prefix in migration UPDATE statements"
This reverts commit
|
||
|
Ruben Fiszel
|
7de98c0df4 |
feat: add OTel metrics support (#8442)
* [ee] feat: add OTel metrics support Add OpenTelemetry metrics export for Windmill operational metrics. When the OTel metrics toggle is enabled in instance settings (EE), Windmill exports 16 metrics to any OTLP-compatible collector, letting users observe queue depths, worker execution, DB pool state, and health without a separate Prometheus setup. Changes: - otel_oss.rs: no-op stubs for OSS builds - monitor.rs: queue count/running count gauges, zombie counters, DB pool monitoring (shared single DB query and loop with Prometheus) - worker.rs: execution count/duration, worker busy, pull duration - jobs.rs: queue push/delete/pull counters - health.rs: DB latency gauge - main.rs: call monitor_pool_otel unconditionally - InstanceSetting.svelte: enable metrics toggle for EE licenses Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for OTel metrics Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add worker_started, worker_uptime, health_status, health_db_unresponsive OTel metrics Wire up 5 additional metrics to reach parity with Prometheus: - worker_execution_failed: wired in add_completed_job_error (was defined but unused) - worker.started: incremented on worker startup - worker.uptime: recorded each loop iteration - health.status: phase gauge (healthy/degraded/unhealthy) - health.db_unresponsive: flag (0/1) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to fbe68e4aa621e30378995cfd328a6ccf74176614 This commit updates the EE repository reference after PR #469 was merged in windmill-ee-private. Previous ee-repo-ref: 6fa1881aafdfb60f4abf11a37f01f6fedaecb3ec New ee-repo-ref: fbe68e4aa621e30378995cfd328a6ccf74176614 Automated by sync-ee-ref workflow. * fix: remove duplicate cfg attr and duplicate OTel pool reporting - Remove duplicate #[cfg(feature = "prometheus")] on monitor_pool - Remove OTel block from monitor_pool; monitor_pool_otel is the sole OTel reporter, eliminating duplicate windmill.db.pool.* metrics in EE builds - Simplify monitor_pool back to its original Prometheus-only structure Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
bee928276e |
fix: show cancelled WAC jobs as done in workflow timeline (#8436)
* fix: show cancelled WAC jobs as done in workflow timeline Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: keep polling after cancel so WAC timeline updates to completed state Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: treat WAC as done in LogPanel when loader stops after cancel Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: show preview badge and hide _MODULES arg in run history Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show preview badge alongside status dot, not instead of it Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
 Diego Imbert
|
ad03a5dbd7 |
fix: cleanup job debounce batch (#8420)
* delete from v2_job_debounce on accumulate * sqlx prepare * Unconditionally remove from v2_job_debounce_batch * sqlx prepare |
||
|
Ruben Fiszel
|
31d6660d56 |
feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Diego Imbert
|
e0e78442b7 |
Support T | T[] in debounce (#8340)
* Detect union types in TS * display union type arguments * Handle single values at accumulation time * nit propagate otyp * Python support * npm package update |
||
|
Ruben Fiszel
|
48bc3e2445 |
fix: prevent zombie jobs from looping forever (#8313)
* fix: always increment zombie job counter to prevent infinite loop at restart limit Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: force-complete zombie jobs when handle_job_error fails Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: revert counter increment change, keep re-detection for robustness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: disable schedule when permissioned_as user is not found Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
83be59e0e8 |
fix: debounce webhook arg accumulation with max_count/max_time limits (#8307)
* fix: correct debounce max_total_debounces_amount semantics and complete previous job on limit exceeded Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: enable debounce arg accumulation for post-preprocessing flows Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add debounce accumulation tests for max_count and max_time limits Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add push-time max_count and max_time accumulation tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * sqlx --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
438f609a78 |
fix: delete debounce_key on post-preprocessing limit exceeded (#8299)
* fix: delete debounce_key entry when post-preprocessing limits exceeded For preprocessor flows, the runnable_settings_handle has debounce_delay_s = None, so maybe_apply_debouncing at pull-time won't clean up stale debounce_key entries. Previously we only reset the entry (UPDATE), but since the flow executes immediately without rescheduling, a stale entry would cause the next incoming flow to incorrectly try to debounce against an already-executing job. Change from UPDATE (reset) to DELETE so the entry is fully removed. Update tests to expect deletion instead of reset. Companion EE PR: https://github.com/windmill-labs/windmill-ee-private/pull/448 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: persist accumulated debounce args to v2_job for flows The in-memory arg accumulation in maybe_apply_debouncing was not persisted back to v2_job. For scripts this is fine (single execution), but for flows, subsequent steps re-read args from the DB via get_mini_pulled_job and would see the original (non-accumulated) value. Also improve the job log message to show both original and accumulated argument values for clarity. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref to d1c14355026151ecdd31adda8e2c60ecd1b5ad65 This commit updates the EE repository reference after PR #448 was merged in windmill-ee-private. Previous ee-repo-ref: bff784002a3335af7c10982599c8f03e536d5abf New ee-repo-ref: d1c14355026151ecdd31adda8e2c60ecd1b5ad65 Automated by sync-ee-ref workflow. * test: assert accumulated debounce args are persisted to v2_job Add DB persistence assertions to accumulation tests to prevent regressions on the fix that writes accumulated args back to v2_job. Without this, flow steps re-reading args from the DB would see the original (non-accumulated) value. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * chore: update ee-repo-ref.txt to ee-private main Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
a6d4390790 |
feat: workflow-as-code (WAC) v2 (#8172)
* feat: workflow-as-code v2 with @task decorator API
Replace ctx.step("name", "script") API with @task decorators where
functions are called directly. Users no longer need to pass WorkflowCtx
or use string-based step names/script paths.
Python: @task decorator with contextvars-based implicit context
TypeScript: task() wrapper with module-level context variable
Parsers: detect @task function calls instead of ctx.step() calls
Worker: updated wrappers to set implicit context
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: WAC v2 checkpoint/replay with _executing_key child dispatch
- Rust-side orchestration: parent dispatches child jobs, suspends, resumes on completion
- _executing_key in checkpoint tells child which step to execute directly
- task() throws StepSuspend(mode="step_complete") after executing target step
- result_processor handles child completion and updates parent checkpoint
- WacGraph.svelte for runtime execution visualization
- Sequential and parallel workflows tested end-to-end
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WAC v2 bundle cache, globalThis ctx sharing, description optional
- Disable bun bundle caching for WAC v2 scripts (wrapper needs
windmill-client from node_modules, not available in bundle mode)
- Use Reflect.set/get(globalThis, "__wmill_wf_ctx") to share workflow
context across dual module instances (wrapper vs user script)
- Never-resolving thenable for non-matching steps in child job mode
prevents Promise.all race conditions
- Make description field optional in NewScript API (defaults to "")
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add step() primitive for inline checkpointed steps
step() executes a function inline (no child job) and persists the result
to the checkpoint. On replay, the cached value is returned — ensuring
deterministic behavior for non-deterministic operations like Date.now()
or Math.random().
- TypeScript: step(name, fn) — executes inline, throws StepSuspend with
mode "inline_checkpoint" to persist before continuing
- Rust: InlineCheckpoint variant in WacOutput, saves to checkpoint and
resets running=false for immediate re-pickup (no zombie wait)
- Shared step counter between task() and step() via _allocKey()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Python WAC v2 support with task(), step(), workflow()
- Python SDK: WorkflowCtx with _executing_key child mode, _alloc_key
shared counter, _run_inline_step for step(), _execute_directly and
_never_resolve for child mode, step() async function
- Python executor: WAC v2 detection, checkpoint.json writing, WAC
wrapper.py generation calling _run_workflow(), post-execution hook
into shared handle_wac_v2_output()
- Make handle_wac_v2_output pub so both bun and python executors share
the same dispatch/suspend/inline-checkpoint logic
- 17 Python tests covering dispatch, replay, parallel, conditional,
inline checkpoint, and child mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update sqlx prepared queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WacGraph Tooltip→Popover, simplify wacToFlow parsers
- Fix type error: Tooltip doesn't accept text snippet, use Popover
- Extract shared helpers for task matching and block collection
- Replace linear tasks.find() with Map lookups
- Remove mutable module-level counter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Box::pin WAC v2 output handler to prevent stack overflow
handle_python_job's async state machine was too large when combined
with handle_wac_v2_output. Box::pin heap-allocates the future.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: merge WAC v1 and v2 task decorators to preserve backward compat
The v2 @task decorator was shadowing the v1 one, breaking WAC v1
scripts that rely on HTTP-based dispatch via /workflow_as_code/ API.
The merged decorator handles three modes:
- v2: inside @workflow context → checkpoint/replay dispatch
- v1: WM_JOB_ID set, no @workflow → HTTP API dispatch + wait_job
- standalone: no Windmill env → execute function body directly
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip no_main_func detection for WAC v2 scripts in TS and Python parsers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent empty/noop dispatch causing infinite requeue loop
- Validate steps.len() > 0 in WAC dispatch handler (issue 3)
- Replace noop StepSuspend throw with never-resolving promise so it
can't reach the backend as an empty dispatch (issue 4)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Python task wrapper now converts positional args to kwargs in v2 mode
Previously only **kwargs were passed to _next_step(), silently dropping
positional arguments. Extract shared _merge_args() helper used by both
v1 and v2 paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace unwrap() with proper error propagation in WAC arg serialization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add workspace_id filter to v2_job queries in WAC dispatch
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent race condition in WAC child dispatch
Restructure dispatch to save checkpoint + suspend parent + seed child
checkpoints in a single transaction BEFORE pushing child jobs. This
ensures a fast child can't complete before the parent is suspended.
Also wrap InlineCheckpoint save + running reset in a transaction to
prevent corrupted state on crash.
Use ULID for pre-generated child job IDs (consistent with rest of API).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: include step key and child job ID in WAC error propagation
Move step_key lookup before the success check so failed child errors
include which task failed, the child job ID, and the original error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document WAC determinism contract and step dispatch semantics
- Document that workflow functions must be deterministic across replays
- Document that WacStepDispatch.script/args are metadata, not dispatch targets
- Add comments on counter-based key allocation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: tighten WAC v2 detection to reduce false positives
Replace naive substring matching with line-aware checks that skip
comments and look for specific patterns:
- TS: import from "windmill-client" containing workflow/task
- Python: @workflow and @task decorators with wmill import
Extracted shared helpers in wac_executor.rs used by both executors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: show failed steps in WacGraph when workflow completes with errors
When flowDone is true and a pending step isn't in completedSteps,
mark it as 'failed' instead of 'running'. The failed state CSS and
XCircle icon were already defined but never triggered.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: unsuspend and fail parent when WAC child push fails
Previously if a child push failed mid-batch, the parent remained
suspended with suspend = num_steps but fewer children, hanging until
the 14-day timeout. Now the push loop catches errors and unsuspends
the parent before returning the error.
Also adds source hash validation: if the script content changes between
replays, the job fails with a clear error instead of silently feeding
stale checkpoint data into wrong steps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: clear suspend_until when unsuspending WAC parent
Set suspend_until = NULL alongside suspend = 0 in both the child
failure and all-children-complete paths, so the parent doesn't rely
on subtle pull query invariants to be re-picked-up.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add exhaustive edge case tests for WAC v2 SDK
fix: make TS task wrapper non-async to fix unawaited task flush
The async wrapper caused microtask-based thenable auto-resolution that
fired .then() and threw StepSuspend before _flushPending() could capture
unawaited steps — making the flush mechanism completely broken. Now the
thenable is returned directly without async wrapping. Backward compatible
with v1 (all code paths still return awaitables).
Tests added (59 TS + 66 Python) covering: full sequential lifecycle,
step after parallel, parallel after parallel, conditional on step result,
empty/single-task workflows, 10+ steps, falsy value preservation, inline
steps, mixed step/task, unawaited flush, child mode with parallel,
key determinism, large parallel groups, and complex mixed patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: atomic checkpoint updates to prevent parallel child race condition
Replace read-modify-write pattern in handle_wac_child_completion with
atomic SQL operations:
- completed_steps merged via jsonb_set(... || jsonb_build_object(...))
so concurrent children on different workers don't overwrite each other
- suspend counter decremented atomically with RETURNING to determine
"all done" condition (instead of checking completed_steps in memory)
- suspend_until cleared in the same atomic decrement statement
Before this fix, two parallel children completing simultaneously could
both load the same checkpoint, each add their step, and save — the
second write would overwrite the first, silently losing a child result
and leaving the parent suspended forever.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: cancel already-pushed children on partial WAC dispatch failure
When pushing child jobs sequentially, if pushing child N fails, children
1..N-1 are already running. Previously the error handler only unsuspended
the parent, leaving orphaned children that would complete and corrupt the
checkpoint state (decrementing suspend on an already-unsuspended parent,
potentially causing duplicate step execution on re-run).
Now on partial failure:
1. Cancel all already-pushed children (prevents them from completing
and corrupting checkpoint state)
2. Clear pending_steps from checkpoint (so parent doesn't think
children are outstanding on re-run)
3. Then unsuspend parent (so the error propagates)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip WAC duration write and child check for non-WAC parents
The duration write to workflow_as_code_status was running for every
non-flow child with a parent (error handlers, success handlers,
run_script children), even though it was only intended for WAC jobs.
Add WHERE workflow_as_code_status IS NOT NULL to skip non-WAC parents
entirely. Piggyback RETURNING pending_steps.job_ids on the same query
so WAC v2 child completion needs zero extra DB round-trips on the
success path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: seed child checkpoint in same transaction as push
The child checkpoint insert was happening before the child job was
pushed, violating the FK constraint on v2_job_status. Move it into
the push transaction so the job row exists and the child can't be
picked up before its checkpoint is ready.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: set running=false when WAC parent suspends for child dispatch
The parent job kept running=true after suspending, so workers wouldn't
pick it up when children completed and suspend reached 0. The parent
only advanced when the zombie job detector reset it (~90s). Now the
dispatch suspend sets running=false so the parent is immediately
eligible for pickup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WAC parent suspend/unsuspend lifecycle
Keep running=true when suspending the parent so the normal pull query
(WHERE running=false) never picks it up. Keep suspend_until non-null
when decrementing suspend to 0 so the suspended pull query
(WHERE suspend_until IS NOT NULL AND suspend<=0) picks it up.
Previously: setting running=false caused infinite restart loops because
the normal pull query has no suspend check and would immediately re-pick
the parent. Clearing suspend_until on the last child prevented the
suspended pull from ever seeing it, requiring the 90s zombie detector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add approval primitive, flow child completion, timeline fixes for WAC v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add error propagation, task options, sleep, and parallel for WAC v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: fix python SDK tests to use name-based keys and add new test coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address WAC v2 review findings (sleep timing, error marker, atomicity)
- Fix sleep using suspend=1 instead of 0 to enforce actual delay
- Add approval/sleep resume injection to Python executor
- Fix TS SDK concurrency_limit mapping (was reading wrong property)
- Namespace error marker as __wmill_error to avoid user data collision
- Wrap child completion SQL in transaction for atomicity
- Decrement suspend even when step key is missing (prevents hang)
- Expand TASK_RE to handle export const, let, var, generics
- Validate step key uniqueness before dispatch
- Log warning on checkpoint deserialization failure
- Remove unimplemented delete_after_use from SDKs
- Add TaskError exception class to Python SDK with diagnostic context
- Fix extra positional args handling and add functools.wraps
- Improve getParamNames to handle typed/destructured params
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* sqlx
* test: add WAC v1 e2e integration tests for TS and Python
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: revert fake test versions in typescript-client
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove unused WacGraph component and strip wacToFlow to isWorkflowAsCode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract shared approval/sleep resume logic into wac_executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Diego Imbert
|
7d9d16a6a3 |
feat: runScript inline for path and hash (#8019)
* runScript inline for path and hash * Update backend/windmill-api/src/jobs.rs Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * refactor: unify inline script param structs and deduplicate closures - Replace RunInlineScriptByPathFnParams and RunInlineScriptByHashFnParams with a single RunInlineScriptFnParams using InlineScriptTarget enum - Collapse two nearly-identical closures in worker.rs into one - Merge duplicate InlineByPath/InlineByHash into InlineScriptArgs - Extract shared run_inline_script_inner helper in API handler - Add missing check_scopes to run_inline_script_by_hash endpoint - Fix duplicate lines from prior commit in run_inline_script_by_path - Change tag from "inline_preview" to "inline" for deployed scripts Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> * Integration tests * rm * rename feature to run_inline * Run inline integration tests * Fix tests * check path scope * openapi fix * nits * remove register_potential_assets_on_inline_execution * unused variable * refactor * Pass user_db to check script permission --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> |
||
|
Ruben Fiszel
|
a00927b300 |
fix: preserve debouncing settings for flows with preprocessors (#8043)
* fix: preserve debouncing settings for flows with preprocessors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Revert "fix: preserve debouncing settings for flows with preprocessors"
This reverts commit
|
||
|
Ruben Fiszel
|
a91c532eca |
fix: make WM_FLOW_PATH available in flow step previews (#8042)
* fix: pass flow path in flow step preview for AI agent modules JobLoader.runFlowPreview was missing the path parameter, causing WM_FLOW_PATH to be unavailable when using the Run button on individual flow steps. Test up to here worked correctly because it uses a different code path (utils.svelte.ts) that already passed the path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: make WM_FLOW_PATH available for rawscript/script step previews Inject the flow path as `_flow_path` in the job args when running a script preview from the flow editor. The SQL pull queries now use COALESCE to fall back to this arg when no parent runnable path exists, making WM_FLOW_PATH available for individual step "Run" previews. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: rename _flow_path args key to _FLOW_PATH Match existing convention used by _ENTRYPOINT_OVERRIDE. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
fd5ebc2fda |
fix: tag bunnative dependency jobs as bun instead of nativets (#8045)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
hugocasa
|
aaa1b92300 |
fix: hash long dedicated worker tags (#7914)
* fix: hash long dedicated worker tags * Update frontend/src/lib/components/dedicated_worker.ts Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
1456f73c16 | nit test | ||
|
Ruben Fiszel
|
4343b73485 | fix: reduce DB pool contention by eliminating dual-connection patterns (#7861) | ||
|
Ruben Fiszel
|
894d8a94f8 |
reuse existing transaction in push to reduce pool pressure (#7858)
* fix: reuse existing transaction in push instead of acquiring new connection In push_inner, fetch_authed_from_permissioned_as was acquiring a new connection from the pool to fetch job permissions, even though a transaction was already open. Use fetch_authed_from_permissioned_as_conn with the existing transaction instead, reducing pool pressure when many jobs are pushed concurrently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * improve contention * improve contention --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5e63d31b92 | nit | ||
|
Ruben Fiszel
|
67c8aef9fa |
fix: box push() future to prevent stack overflow in nested async chains
The push() function generates a ~13KB async state machine that gets inlined into every caller's future. In deeply nested async chains (e.g. flow execution), this causes stack overflows. Boxing the future at the definition site via a thin wrapper reduces each caller's stack footprint to a single pointer. This also reverts the RUST_MIN_STACK workaround from CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
ccc4806b9a |
refactor: extract windmill-types crate from windmill-common (#7851)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
c580572252 |
refactor: extract windmill-api-scripts and windmill-api-users subcrates (#7850)
* refactor: extract windmill-api-scripts and windmill-api-users subcrates Split the monolithic windmill-api crate by extracting scripts.rs, flows.rs, users.rs, and users_oss.rs into dedicated subcrates. This reduces incremental rebuild times when editing these modules. Changes: - Create windmill-api-scripts crate (scripts.rs + flows.rs, ~4.3K lines) - Create windmill-api-users crate (users.rs + users_oss.rs, ~2.4K lines) - Move clear_schedule to windmill-queue (shared by scripts, flows, workspaces) - Move username utilities (VALID_USERNAME, INVALID_USERNAME_CHARS, generate_instance_wide_unique_username) to windmill-common/src/usernames.rs - Move COOKIE_DOMAIN, IS_SECURE, WithStarredInfoQuery, BulkDeleteRequest, WebhookShared to windmill-common for cross-crate access - Original files in windmill-api become thin stubs with pub use re-exports - EE-dependent route handlers remain in windmill-api (create_user, rename_user, set_password, reset_password, etc.) - Feature forwarding for enterprise, private, parquet, no_auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract windmill-api-workspaces subcrate (Step 3) Move workspaces.rs, workspaces_extra.rs, workspaces_oss.rs, and workspaces_ee.rs into a new windmill-api-workspaces crate (~7K lines). Routes that depend on windmill-api internals (AI copilot, teams, tarball export, critical alerts, stripe) remain in the windmill-api stub. The subcrate handles all other workspace management routes. Also moved send_email_if_possible to windmill-common/email_oss.rs to make it available across subcrates without circular deps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * refactor: extract windmill-api-groups subcrate (groups.rs + folders.rs) Extract groups.rs (1,093 lines) and folders.rs (833 lines) into a new windmill-api-groups subcrate. Both modules had clean dependencies on already-extracted crates (windmill-api-auth, windmill-common, windmill-api-workspaces). Also removes unused re-exports of get_instance_username_or_create_pending and INVALID_USERNAME_CHARS from windmill-api/src/utils.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: add granular_acls.rs and folder_history.rs to windmill-api-groups Extract granular_acls.rs (395 lines) and folder_history.rs (68 lines) into the windmill-api-groups subcrate. Both modules only depend on already-extracted crates and belong to the same access-control domain as groups and folders. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove unused imports and dead code from subcrate extraction - Remove unused BASE_URL import from lib.rs - Remove workspaces_extra.rs and workspaces_oss.rs re-export stubs (no consumers in windmill-api) - Remove dead send_email_if_possible OSS stub (callers moved to windmill-api-users) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * chore: bust CI cargo cache for subcrate split Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: re-export BASE_URL for EE files that use crate::BASE_URL Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: forward no_auth feature to windmill-api-users, remove dead code - Add "windmill-api-users/no_auth" to windmill-api's no_auth feature so the login bypass in users.rs:1600 activates correctly - Remove dead send_email_if_possible from windmill-api-users/users_oss.rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: re-enable cargo cache for backend tests Cache was disabled to bust stale entries from before subcrate split. Now that a clean build has run, re-enable for faster CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: install mold+clang in CI workflows The .cargo/config.toml uses mold linker for x86_64-linux. Build scripts require linking even during cargo check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: increase cargo test timeout to 30 min Exit code 143 (SIGTERM) means the 20-min timeout was hit during compilation without cache. Bump to 30 min as safety net. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: limit cargo build jobs to 4 to prevent OOM in CI Exit code 143 (SIGTERM) after 8 min = OOM kill during compilation. 8 parallel LLVM codegen jobs exhaust memory on ubicloud-standard-8. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
bbb397b6ad |
fix: improve scheduling reliability in extreme pool contention conditions (#7825)
* fix: reuse outer tx for schedule push in commit_completed_job Instead of calling handle_maybe_scheduled_job(db) which opens its own connections (peak=3), inline the schedule push using a savepoint on the outer transaction. Auth is fetched via the tx connection using fetch_authed_from_permissioned_as_conn, and push_scheduled_job runs on a savepoint so failures roll back only the push, not the completion. On push failure: savepoint rolls back, schedule is disabled on the outer tx, and the zombie return path is preserved if disabling also fails. Peak connections drop from 3 to 1 (or 2 on cold RunnableSettings cache). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * fix: extract shared try_schedule_next_job to unify schedule push paths Replace the two diverging schedule-push implementations (inlined in commit_completed_job and standalone handle_maybe_scheduled_job) with a single try_schedule_next_job that reuses the caller's transaction via savepoints. This eliminates extra pool connection usage in the worker_flow.rs path and ensures consistent retry/error semantics. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add failpoint markers to try_schedule_next_job Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove plan.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove inner retry loop from try_schedule_next_job, add caller-level retries The 10-retry x 5s-sleep loop inside try_schedule_next_job held locks on v2_job_completed/v2_job_queue for up to ~45s when running inside the outer commit_completed_job transaction. Now try_schedule_next_job makes a single attempt and returns errors to the caller. Non-retryable errors (QuotaExceeded, NotFound) disable the schedule immediately inside the function. Transient errors are returned for the caller to retry: - commit_completed_job path: outer backon retry (10x3s) retries the entire transaction including the schedule push, so no locks are held during sleep. - handle_flow path: new backon retry (10x3s) wraps begin/push/commit with a fresh transaction per attempt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: clear push_err after successful schedule disable to prevent stuck schedules When try_schedule_next_job disables the schedule for non-retryable errors (NotFound, QuotaExceeded), clear the error so the caller commits the tx (persisting the disable). Previously, the error propagated up, causing the tx to be dropped and rolling back the disable — leaving the schedule permanently enabled but broken. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add 5s timeout on push_scheduled_job, clean up handle_flow error handling - Add tokio::time::timeout(5s) around push_scheduled_job inside try_schedule_next_job to bound worst-case lock holding per attempt - Remove unreachable QuotaExceeded/NotFound match arms in handle_flow (these errors are handled internally by try_schedule_next_job) - Add report_error_to_workspace_handler_or_critical_side_channel in handle_flow when post-exhaustion schedule disable fails Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: return SchedulePushZombieError when both schedule push and disable fail When handle_flow cannot push the next scheduled job AND cannot disable the schedule, return a SchedulePushZombieError so the worker leaves the flow job in the queue for zombie detection to restart. This prevents stuck schedules where neither the next tick was pushed nor the schedule was disabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
e655a06563 |
fix: prevent schedule pool connection exhaustion (#7821)
* fix: prevent schedule pool connection exhaustion Schedules were disabling themselves after upgrading from v1.605.0 to v1.614.0 due to pool connection deadlock. The root cause was fetch_authed_from_permissioned_as acquiring a pool connection inside push() while a transaction already held one, exhausting the pool under pressure. Fix: pre-compute Authed before db.begin() for the normal path, and reuse the transaction connection via fetch_authed_from_permissioned_as_conn for the on_behalf_of_email path. Peak pool usage drops from 2 to 1 for all schedule push paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace pool exhaustion tests with comprehensive schedule push tests Replace the 16 pool-specific tests with 19 tests covering all schedule push code paths: script/flow scheduling, on_behalf_of_email (script and flow), retry wrapping, duplicate detection, invalid timezone/cron/args, script/flow not found, paused schedules, clock shift detection, disabled schedule, path mismatch, push failure disabling schedule, and trigger metadata. Also simplify the obo_authed pattern in push_scheduled_job to use a single match assignment instead of two bindings with .or() chaining. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
 claude[bot]
|
05fa3cd013 |
feat: add workspace setting to disable error handler for u/ scripts/flows (#7634)
* feat: add workspace setting to disable error handler for u/ scripts/flows Add a new workspace setting `error_handler_muted_on_user_path` that allows disabling the error handler for jobs related to scripts/flows that start with the "u/" prefix (user scripts/flows). Changes: - Add migration to add `error_handler_muted_on_user_path` column to workspace_settings - Update WorkspaceSettings struct and get_settings query - Update EditErrorHandler struct and edit_error_handler API endpoint - Update error handler cache to include the new setting - Add skip logic in send_error_to_workspace_handler for u/ paths - Add toggle in workspace settings UI The implementation uses the existing cached settings to avoid additional database queries, making it efficient. Closes #7633 Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * all * update ee private * combine migrations * sqlx * fix: migration fixes for auto_invite JSONB column - Fix TEXT[] to JSONB conversion using to_jsonb() - Add GIN index on auto_invite for efficient ? operator queries - Add index cleanup to down migration Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * sqlx * all * all * all --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Pyra
|
7249b82dba | feat: better mixed versions handling (#7628) |