Commit Graph

7 Commits

Author SHA1 Message Date
hugocasa f2f0812a04 feat(flows): opt-in to include the stopping step's result in early-stop errors (#9446)
* feat(flows): early stop can include the stopping step's result in the raised error

When a step uses Early Stop with "Raise an error message if stopped", the
flow result was entirely replaced with a static error object
({"error": {"name": "EarlyStopError", "message": "..."}}), discarding the
stopping step's own output. This made it impossible to stop+fail a flow
while preserving the data the step produced (e.g. an API that returns
HTTP 200 with a userErrors payload).

Add an opt-in `error_include_result` flag on StopAfterIf. When enabled on
the raise-error path, the raised payload becomes
{"error": {...}, "result": <step result>} instead of dropping the result.
Default is false, so existing behavior is unchanged. The option is threaded
through the worker's stop-after-if handling (including stop_after_all_iters_if
for loops/branchall) and exposed in the flow editor's Early Stop panel.

Fixes WIN-2012

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(flows): cover early-stop error_include_result payload shaping

Add a regression test asserting that a step using Early Stop with a raised
error message and error_include_result=true fails the flow while preserving
the step output as {"error": {..}, "result": <step result>}, and that with
the flag off the result is the bare {"error": {..}} object.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(flows): nest early-stop step result inside the error object

Embed the stopping step's result under `error.result` rather than as a
top-level sibling of `error`. This keeps the flow result shape as
`{ "error": { .. } }` — identical to a normal error — so consumers that
key off the top-level shape (single `error` key) keep working, while the
data is still preserved for those that look inside the error object.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(flows): always include the stopping step's result in early-stop errors

Drop the opt-in `error_include_result` gate. Since the step result is nested
inside the error object (`error.result`), the top-level result shape stays
`{ "error": .. }` — identical to a normal error — so consumers that detect or
parse failures by the top-level shape are unaffected. Gating it added schema
surface, plumbing, and a UI toggle for no real compatibility benefit.

Now, whenever a step early-stops with a raised error message, the flow fails
and the raised error embeds the stopping step's own result under
`error.result` (aggregated iteration results for loops/branchall). This
reverts the `StopAfterIf.error_include_result` field, its threading, the
OpenAPI/generated-client surface, and the editor toggle; the "Raise an error
message" tooltip now notes that the step result is included.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(flows): gate early-stop result inclusion behind opt-in flag

Re-introduce the per-step `error_include_result` flag (default off) instead
of always embedding the step result. Although nesting the result under
`error.result` keeps the result *shape* backward-compatible, it does not
address data exposure: a failed flow's result is propagated to synchronous
webhook callers, the flow's failure module, and the workspace/global error
handler (commonly a Slack/email/outbound-webhook notifier). Always including
the step output would surface previously-redacted intermediate data to all of
those sinks for every existing error-stop flow.

Gating keeps the existing behavior (bare `{ "error": .. }`) as the default and
only embeds `error.result` when the flow author explicitly opts in, matching
the original issue's intent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(flows): omit error_include_result when false; refresh generated prompts

- Add `skip_serializing_if = "is_false"` to `StopAfterIf.error_include_result`
  so serialized flows are byte-identical when the flag is off. Fixes the
  `flowmodule_serde` round-trip test (cargo_test) and avoids churn on existing
  flows.
- Regenerate `system_prompts/auto-generated/` and `cli/src/guidance/skills.gen.ts`
  for the new OpenFlow `error_include_result` property. Fixes check-freshness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(flows): cover error_include_result for the loop "stop after all iters" path

Add a regression test for the stop_after_all_iters_if branch, where `nresult`
already holds the aggregated iteration results — confirming `error.result`
carries each iteration's output (distinct from the per-step fallback path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 18:57:05 +00:00
hugocasa 6e5a21a9c7 fix(flows): inherit flow_env in sub-flow predicates (#9042)
* fix(flows): inherit flow_env in sub-flow predicates

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): align flow_env lookup with get_root_job_id and tighten gate

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): drop recursive CTE, root_job propagation suffices

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(flows): walk via flow_innermost_root_job to respect imported-flow scope

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): remove flow_env API endpoint, dead code from deno_core era

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 20:11:38 +00:00
Ruben Fiszel 2d18a68099 feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period

Extends delete_after_use with delete_after_secs to enable configurable
retention periods for job args/result/logs. At completion, jobs can be
scheduled for future deletion via a new job_delete_schedule table,
processed by a monitor task. Supports per-script, per-flow, and
per-flow-step configuration. Backward compatible.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add integration tests, revert query! macros, fix review issues

- Add integration tests for resolve_delete_after_secs, schedule_job_deletion,
  flow-level and module-level delete_after_secs, backward compat
- Revert sqlx::query() back to sqlx::query!() macros for compile-time safety
- Regenerate sqlx offline cache
- Fix FlowModule/NewScript/FlowValue constructions in all test files
- Fix autoscaling_ee.rs for updated script_path_to_payload return type

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for autoscaling_ee fix

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate cleanup_scheduled_job_deletions behind enterprise feature

Prevents dead_code warning (which CI treats as error via -D warnings)
when compiling without enterprise feature.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate sqlx cache after merge with main

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback on scheduled deletion

- Monitor: roll back transaction on any cleanup error so schedule rows
  survive for retry on next cycle (instead of best-effort then discard)
- Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id
  to prevent orphan rows when jobs are deleted through other means
- Simplify bool-to-Option conversion with .then_some(true)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: stop setting delete_after_use alongside delete_after_secs

No mixed-version deployment scenario exists, so delete_after_secs alone
is sufficient. The backend's resolve_delete_after_secs handles
(None, Some(secs)) correctly without needing delete_after_use set.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove delete_after_use from public API surface

Remove delete_after_use from OpenAPI spec, API client, runtime client,
and workspace export. Only delete_after_secs is exposed going forward.

The field remains in Rust backend types with #[serde(skip_serializing)]
for backward-compatible deserialization of existing scripts/flows that
were saved with delete_after_use: true.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806

This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private.

Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3

New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806

Automated by sync-ee-ref workflow.

* fix: regenerate system prompts, remove unused import

- Regenerate auto-generated system prompts after openflow schema change
- Remove unused serde_json::json import in test file (CI -D warnings)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: insert dummy v2_job row in schedule tests for FK constraint

The job_delete_schedule table has a FK to v2_job, so tests need a
real v2_job row before inserting into the schedule table.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: trigger CI re-run

* fix: remove heavy flow integration tests to avoid CI worker contention

The flow integration tests spawn workers that compete for CPU with
the existing relock_skip tests under --test-threads=10, causing
consistent 60s timeouts in CI. Keep only the lightweight unit tests
and DB integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: restore correct ee-repo-ref for our branch

The ref was overwritten to main's EE ref during a rebase. Restore to
our branch's EE commit that includes the autoscaling tuple fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: retrigger CI on fresh runner

* fix: remove FK constraint from job_delete_schedule to unblock CI

The FK with ON DELETE CASCADE to v2_job may have caused performance
overhead during test DB setup (each sqlx::test creates a fresh DB
with all migrations). Remove the FK — orphan schedule rows are
harmlessly cleaned by the monitor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ee-ref

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-08 04:15:28 +00:00
Diego Imbert 5d1c54d9b3 feat: Debounce node (#8324)
* Debounce node works

* sqlx prepare

* sqlx prepare

* fix: address PR review issues for flow node debouncing

- Add sibling check in parent-walking loop to avoid killing branchall siblings
- Remove stale .sqlx cache files from earlier iterations
- Remove single-variant FlowNodeDebounceResult enum, use Result<()>
- Parse flow value once in version guard, recurse into nested modules
- Fix Svelte reactivity when switching selected flow modules
- Fix Tab indentation in FlowModuleComponent
- Use integer types in OpenAPI spec for debounce fields

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ee repo ref

* nit sqlx

* add Debouncing: None

* ee repo ref

* ee repo

* sqlx update

* fix: reject node-level debouncing inside branches (branchall/branchone)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: reject node-level debouncing inside branches (branchall/branchone)"

This reverts commit fa4820dde2.

* ee repo

* sqlx prepare

* sqlx prepare

* feat: add MIN_VERSION_SUPPORTS_NODE_DEBOUNCING (1.658.0) version guard

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: mark node-level debouncing as EE only in openflow schema

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: guard node debouncing against parallel steps (len > 1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* generate system prompts

* system prompts

* chore: update ee-repo-ref to c04f3851c03758662e4936ff4b6e71bc56dbae7e

This commit updates the EE repository reference after PR #451 was merged in windmill-ee-private.

Previous ee-repo-ref: d140bb8944dfe3efb23cf8c12f556eacf30e2f87

New ee-repo-ref: c04f3851c03758662e4936ff4b6e71bc56dbae7e

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-24 09:47:41 +00:00
Ruben Fiszel ea52a8b8ce refactor: move integration tests to subcrates to reduce recompilation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:04:54 +00:00
Ruben Fiszel 22cce51db5 fix: return null for non-existent step access in flow expressions
Previously, accessing a non-existent step via results.nonexistent would
throw an error. This fix makes both Deno Core and QuickJS return null
instead, enabling patterns like:

- results.nonexistent ?? 'default'
- results.nonexistent?.value ?? 'default'

The fix was applied to:
- js_eval.rs: handle_full_regex fast-path now uses .ok().flatten()
- js_eval_quickjs.rs: fallback path now uses .ok().unwrap_or(null)

Added flow engine test to verify the behavior.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 07:48:46 +00:00
Ruben Fiszel 5c20b37a53 feat: add QuickJS as alternative JS engine for flow expression evaluation (#7664)
* feat: add QuickJS as alternative JS engine for flow expression evaluation

Add rquickjs as an optional alternative to deno_core for evaluating
JavaScript expressions in flow transformations. QuickJS offers ~8-16x
faster startup times for simple expressions, making it ideal for
evaluating many small expressions in flows.

Key changes:
- Add new `quickjs` feature flag for windmill-worker
- Implement js_eval_quickjs.rs with true async Rust callbacks for
  variable(), resource(), and results.xxx access (no pre-fetching)
- Share expression transformation logic (replace_with_await,
  replace_with_await_result) between both implementations
- Add USE_QUICKJS_FOR_FLOW_EVAL env var to switch engines at runtime
- When only quickjs feature is enabled (no deno_core), QuickJS is
  automatically used
- Add comprehensive parity tests comparing QuickJS and deno_core output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* quickjs

* quickjs

* all

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:44:40 +00:00