mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
dda59767c2b997e675ffe799bc6b1dcc79c2a52d
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
3b95c947d4 |
test(wac): pin the failure record with one corpus both SDKs read (#10385)
* test(wac): pin the failure record with one corpus both SDKs read Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(wac): add the behaviour matrix that verified the failure record Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(wac): record how to exercise an unreleased SDK change Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): guard the whole extra pair, not just its value Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): never rehash an untrusted extra key Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): walk only a real __dict__ when collecting extra Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(wac): name the divergence the corpus cannot pin Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(wac): state the extra-encoding constraints in four lines Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
aeaea57ca1 |
fix(wac): one failure record for tasks and steps, in every round (#10368)
* fix(wac): hand a caught task and step failure the same shape in every round Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(wac): decide the failure record once, server-side Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): leave a legacy SDK's failure marker untouched, and ship wacError to jsr Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): carry a step's custom error fields, and bound the stack in bytes Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): keep a step's extra fields serializable and bounded Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): record a non-Error throw the way a task records it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): guard the last unguarded throw site in the step marker Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): make failure reporting non-throwing on both clients Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): take the step traceback the way the executor takes it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): contain the reads that happen before a failure is checkpointed Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): fall back to the checkpointed marker, not the live one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): keep non-finite fields and hostile proxies out of the checkpoint path Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): keep the snapshot that passed the serialization probe Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(wac): keep the failure-record module's surface to what is used Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
044ce39e5f |
fix(wac): return the checkpointed value from step(), not the live object (#10367)
* fix(wac): return the checkpointed value from step(), not the live object Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(wac): regenerate system prompts and narrow the round-trip claim Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * style(wac): condense the round-trip comments and fix the fallback note Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(sdk): type step() as the JSON round trip of its body's result Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(sdk): apply the JSON round trip to task() and the standalone paths Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): encode bigint, keep unknown as unknown, align dropped-key results Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): null out results whose key JSON.stringify would drop Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): normalize only the top-level result, keeping nested keys as they were Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(wac): normalize a child task's result so a deployed job cannot fail to parse Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(sdk): pin non-finite number behavior in Jsonified and its tests Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): admit undefined for keys whose value JSON.stringify may omit Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): make a key JSON.stringify may omit optional, not just nullable Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(sdk): treat a class-valued property as dropped, like any other function Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
727d22b9a1 |
fix(wac): report a task failure the child round's body catches (#10366)
* fix(wac): report a task failure the child round's body catches * chore(wac): state the child-round failure invariant once * test(wac): pin the catch-then-continue re-raise in the child round |
||
|
Ruben Fiszel
|
be5e3bbfc4 |
fix(wac): checkpoint step errors so a caught exception does not hang replay (#10348)
* fix(wac): checkpoint step errors so a caught exception does not hang replay Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(wac): honour a step suspend the workflow body caught and swallowed Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(wac): park every suspend, not only those from a failing step Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(wac): keep the generated bun wrapper comment-free Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(wac): park the child task-completion suspend and align error identity Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(wac): pin the TaskError identity of replayed step and task failures Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
9cef724ff2 |
feat: bind WAC approval urls to a named wait_for_approval step (#10317)
* feat: bind WAC approval urls to a named wait_for_approval step Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: reject duplicate WAC approval step keys instead of renaming them Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: reject WAC approval links minted for a step that is not awaiting approval Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: bind WAC approval links to the awaiting step and stop step key aliasing Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: reject empty approval keys and scope minted-key writes to the workspace Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: enforce WAC approval binding at consumption and reject colliding keys Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: make WAC approval binding and collision checks atomic, harden TS step keys Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: decrement WAC suspend atomically instead of from a pre-lock snapshot Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: add sqlx cache entry for the atomic WAC suspend decrement Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: omit empty approver param from python get_approval_urls Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: pin the suspend-snapshot decrement and the colliding-mint race Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: drop the suspend-snapshot interleave test, it cannot both be stable and discriminate Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix: reject step keys that cannot be minted as a URL path segment Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a6d4390790 |
feat: workflow-as-code (WAC) v2 (#8172)
* feat: workflow-as-code v2 with @task decorator API
Replace ctx.step("name", "script") API with @task decorators where
functions are called directly. Users no longer need to pass WorkflowCtx
or use string-based step names/script paths.
Python: @task decorator with contextvars-based implicit context
TypeScript: task() wrapper with module-level context variable
Parsers: detect @task function calls instead of ctx.step() calls
Worker: updated wrappers to set implicit context
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: WAC v2 checkpoint/replay with _executing_key child dispatch
- Rust-side orchestration: parent dispatches child jobs, suspends, resumes on completion
- _executing_key in checkpoint tells child which step to execute directly
- task() throws StepSuspend(mode="step_complete") after executing target step
- result_processor handles child completion and updates parent checkpoint
- WacGraph.svelte for runtime execution visualization
- Sequential and parallel workflows tested end-to-end
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WAC v2 bundle cache, globalThis ctx sharing, description optional
- Disable bun bundle caching for WAC v2 scripts (wrapper needs
windmill-client from node_modules, not available in bundle mode)
- Use Reflect.set/get(globalThis, "__wmill_wf_ctx") to share workflow
context across dual module instances (wrapper vs user script)
- Never-resolving thenable for non-matching steps in child job mode
prevents Promise.all race conditions
- Make description field optional in NewScript API (defaults to "")
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add step() primitive for inline checkpointed steps
step() executes a function inline (no child job) and persists the result
to the checkpoint. On replay, the cached value is returned — ensuring
deterministic behavior for non-deterministic operations like Date.now()
or Math.random().
- TypeScript: step(name, fn) — executes inline, throws StepSuspend with
mode "inline_checkpoint" to persist before continuing
- Rust: InlineCheckpoint variant in WacOutput, saves to checkpoint and
resets running=false for immediate re-pickup (no zombie wait)
- Shared step counter between task() and step() via _allocKey()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Python WAC v2 support with task(), step(), workflow()
- Python SDK: WorkflowCtx with _executing_key child mode, _alloc_key
shared counter, _run_inline_step for step(), _execute_directly and
_never_resolve for child mode, step() async function
- Python executor: WAC v2 detection, checkpoint.json writing, WAC
wrapper.py generation calling _run_workflow(), post-execution hook
into shared handle_wac_v2_output()
- Make handle_wac_v2_output pub so both bun and python executors share
the same dispatch/suspend/inline-checkpoint logic
- 17 Python tests covering dispatch, replay, parallel, conditional,
inline checkpoint, and child mode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: update sqlx prepared queries
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WacGraph Tooltip→Popover, simplify wacToFlow parsers
- Fix type error: Tooltip doesn't accept text snippet, use Popover
- Extract shared helpers for task matching and block collection
- Replace linear tasks.find() with Map lookups
- Remove mutable module-level counter
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Box::pin WAC v2 output handler to prevent stack overflow
handle_python_job's async state machine was too large when combined
with handle_wac_v2_output. Box::pin heap-allocates the future.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: merge WAC v1 and v2 task decorators to preserve backward compat
The v2 @task decorator was shadowing the v1 one, breaking WAC v1
scripts that rely on HTTP-based dispatch via /workflow_as_code/ API.
The merged decorator handles three modes:
- v2: inside @workflow context → checkpoint/replay dispatch
- v1: WM_JOB_ID set, no @workflow → HTTP API dispatch + wait_job
- standalone: no Windmill env → execute function body directly
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip no_main_func detection for WAC v2 scripts in TS and Python parsers
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent empty/noop dispatch causing infinite requeue loop
- Validate steps.len() > 0 in WAC dispatch handler (issue 3)
- Replace noop StepSuspend throw with never-resolving promise so it
can't reach the backend as an empty dispatch (issue 4)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: Python task wrapper now converts positional args to kwargs in v2 mode
Previously only **kwargs were passed to _next_step(), silently dropping
positional arguments. Extract shared _merge_args() helper used by both
v1 and v2 paths.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: replace unwrap() with proper error propagation in WAC arg serialization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add workspace_id filter to v2_job queries in WAC dispatch
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent race condition in WAC child dispatch
Restructure dispatch to save checkpoint + suspend parent + seed child
checkpoints in a single transaction BEFORE pushing child jobs. This
ensures a fast child can't complete before the parent is suspended.
Also wrap InlineCheckpoint save + running reset in a transaction to
prevent corrupted state on crash.
Use ULID for pre-generated child job IDs (consistent with rest of API).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: include step key and child job ID in WAC error propagation
Move step_key lookup before the success check so failed child errors
include which task failed, the child job ID, and the original error.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: document WAC determinism contract and step dispatch semantics
- Document that workflow functions must be deterministic across replays
- Document that WacStepDispatch.script/args are metadata, not dispatch targets
- Add comments on counter-based key allocation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: tighten WAC v2 detection to reduce false positives
Replace naive substring matching with line-aware checks that skip
comments and look for specific patterns:
- TS: import from "windmill-client" containing workflow/task
- Python: @workflow and @task decorators with wmill import
Extracted shared helpers in wac_executor.rs used by both executors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: show failed steps in WacGraph when workflow completes with errors
When flowDone is true and a pending step isn't in completedSteps,
mark it as 'failed' instead of 'running'. The failed state CSS and
XCircle icon were already defined but never triggered.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: unsuspend and fail parent when WAC child push fails
Previously if a child push failed mid-batch, the parent remained
suspended with suspend = num_steps but fewer children, hanging until
the 14-day timeout. Now the push loop catches errors and unsuspends
the parent before returning the error.
Also adds source hash validation: if the script content changes between
replays, the job fails with a clear error instead of silently feeding
stale checkpoint data into wrong steps.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: clear suspend_until when unsuspending WAC parent
Set suspend_until = NULL alongside suspend = 0 in both the child
failure and all-children-complete paths, so the parent doesn't rely
on subtle pull query invariants to be re-picked-up.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add exhaustive edge case tests for WAC v2 SDK
fix: make TS task wrapper non-async to fix unawaited task flush
The async wrapper caused microtask-based thenable auto-resolution that
fired .then() and threw StepSuspend before _flushPending() could capture
unawaited steps — making the flush mechanism completely broken. Now the
thenable is returned directly without async wrapping. Backward compatible
with v1 (all code paths still return awaitables).
Tests added (59 TS + 66 Python) covering: full sequential lifecycle,
step after parallel, parallel after parallel, conditional on step result,
empty/single-task workflows, 10+ steps, falsy value preservation, inline
steps, mixed step/task, unawaited flush, child mode with parallel,
key determinism, large parallel groups, and complex mixed patterns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: atomic checkpoint updates to prevent parallel child race condition
Replace read-modify-write pattern in handle_wac_child_completion with
atomic SQL operations:
- completed_steps merged via jsonb_set(... || jsonb_build_object(...))
so concurrent children on different workers don't overwrite each other
- suspend counter decremented atomically with RETURNING to determine
"all done" condition (instead of checking completed_steps in memory)
- suspend_until cleared in the same atomic decrement statement
Before this fix, two parallel children completing simultaneously could
both load the same checkpoint, each add their step, and save — the
second write would overwrite the first, silently losing a child result
and leaving the parent suspended forever.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: cancel already-pushed children on partial WAC dispatch failure
When pushing child jobs sequentially, if pushing child N fails, children
1..N-1 are already running. Previously the error handler only unsuspended
the parent, leaving orphaned children that would complete and corrupt the
checkpoint state (decrementing suspend on an already-unsuspended parent,
potentially causing duplicate step execution on re-run).
Now on partial failure:
1. Cancel all already-pushed children (prevents them from completing
and corrupting checkpoint state)
2. Clear pending_steps from checkpoint (so parent doesn't think
children are outstanding on re-run)
3. Then unsuspend parent (so the error propagates)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: skip WAC duration write and child check for non-WAC parents
The duration write to workflow_as_code_status was running for every
non-flow child with a parent (error handlers, success handlers,
run_script children), even though it was only intended for WAC jobs.
Add WHERE workflow_as_code_status IS NOT NULL to skip non-WAC parents
entirely. Piggyback RETURNING pending_steps.job_ids on the same query
so WAC v2 child completion needs zero extra DB round-trips on the
success path.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: seed child checkpoint in same transaction as push
The child checkpoint insert was happening before the child job was
pushed, violating the FK constraint on v2_job_status. Move it into
the push transaction so the job row exists and the child can't be
picked up before its checkpoint is ready.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: set running=false when WAC parent suspends for child dispatch
The parent job kept running=true after suspending, so workers wouldn't
pick it up when children completed and suspend reached 0. The parent
only advanced when the zombie job detector reset it (~90s). Now the
dispatch suspend sets running=false so the parent is immediately
eligible for pickup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: WAC parent suspend/unsuspend lifecycle
Keep running=true when suspending the parent so the normal pull query
(WHERE running=false) never picks it up. Keep suspend_until non-null
when decrementing suspend to 0 so the suspended pull query
(WHERE suspend_until IS NOT NULL AND suspend<=0) picks it up.
Previously: setting running=false caused infinite restart loops because
the normal pull query has no suspend check and would immediately re-pick
the parent. Clearing suspend_until on the last child prevented the
suspended pull from ever seeing it, requiring the 90s zombie detector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add approval primitive, flow child completion, timeline fixes for WAC v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add error propagation, task options, sleep, and parallel for WAC v2
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: fix python SDK tests to use name-based keys and add new test coverage
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address WAC v2 review findings (sleep timing, error marker, atomicity)
- Fix sleep using suspend=1 instead of 0 to enforce actual delay
- Add approval/sleep resume injection to Python executor
- Fix TS SDK concurrency_limit mapping (was reading wrong property)
- Namespace error marker as __wmill_error to avoid user data collision
- Wrap child completion SQL in transaction for atomicity
- Decrement suspend even when step key is missing (prevents hang)
- Expand TASK_RE to handle export const, let, var, generics
- Validate step key uniqueness before dispatch
- Log warning on checkpoint deserialization failure
- Remove unimplemented delete_after_use from SDKs
- Add TaskError exception class to Python SDK with diagnostic context
- Fix extra positional args handling and add functools.wraps
- Improve getParamNames to handle typed/destructured params
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* sqlx
* sqlx
* test: add WAC v1 e2e integration tests for TS and Python
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: revert fake test versions in typescript-client
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove unused WacGraph component and strip wacToFlow to isWorkflowAsCode
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: extract shared approval/sleep resume logic into wac_executor
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
|