Commit Graph
1761 Commits
Author SHA1 Message Date
Ruben FiszelandClaude Opus 4.7 750904fa29 fix(nsjail): gate unix-symlink test behind cfg(unix) for Windows build (#9280)
The disk_backed_refuses_preexisting_symlink_at_jail_tmp test calls
std::os::unix::fs::symlink directly, which doesn't exist on Windows
targets. Without a cfg gate, `cargo check --tests` fails on Windows
with E0433. Other symlink call sites in this crate (php_executor,
bun_executor, rust_executor, etc.) already follow this pattern.

Fixes WIN-1972

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:10:02 +02:00
Ruben Fiszel 6676284a9e feat(nsjail): optional disk-backed /tmp via instance setting (#9272)
* feat(nsjail): optional disk-backed /tmp via instance setting

* test(nsjail): unit-test tmp mount resolver and narrow visibility

* refactor(nsjail): switch tmp backing to select + conditional UI

* ui(nsjail): make tmpfs the visible default in /tmp backing select

* fix(nsjail): refuse preexisting jail_tmp to block symlink escape

* fix(nsjail): allow jail_tmp reuse on sequential nsjail calls

Codex flagged that python/ruby/rust executors invoke nsjail twice per
job_dir (install then run). The previous resolver treated any preexisting
jail_tmp as hostile and silently fell back to tmpfs on the second call,
so disk-backed mode never reached the main script run for those langs.

Use symlink_metadata().is_dir() to distinguish a real directory left by
an earlier call in the same job_dir (safe to reuse) from a symlink or
other entity (still refused, as the codebase-tar escape requires).

Also loosen the frontend visibility predicate: only hide nsjail settings
when job_isolation is explicitly 'none' or 'unshare', so deployments
that enable nsjail via DISABLE_NSJAIL=false with no DB setting can
still see the controls.
2026-05-25 12:10:02 +02:00
Ruben FiszelandClaude Opus 4.7 03cd1c97b6 feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting (#9271)
* feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting

Allows operators to point `uv python install` at a private mirror of the
python-build-standalone releases. Configurable via the
`UV_PYTHON_INSTALL_MIRROR` env var or the `uv_python_install_mirror`
instance setting, with the env var as the boot fallback and the instance
setting taking precedence at reload.

Fixes WIN-1966

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: hoist uv_python_install_mirror binding above sandboxing branch

The non-sandboxed uv pip install branch referenced a binding that was
only declared inside the sandboxed branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: neutral placeholder for uv_python_install_mirror

The previous placeholder was the default public URL the setting is meant
to redirect away from. A neutral example mirror URL is clearer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 12:10:02 +02:00
Ruben Fiszel 257cfebdbf oom_adj nit 2026-05-25 12:10:02 +02:00
Diego Imbertandclaude[bot] 45dc3194e7 feat: CLI datatable serve / psql (#9267)
* feat(cli): add datatable list and run commands

* feat(cli): render datatable query results as a table

* feat(cli): serve datatables as a postgres-wire endpoint

* feat(cli): add 'datatable psql' to launch psql against the proxy

* feat(cli): route datatable serve by client-supplied database name

* override database list + password option

* fix: support extended queries in datatable serve

* fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma

* refactor: extract raw_output envelope encoding into pg_raw_output module

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
2026-05-25 12:10:02 +02:00
Ruben Fiszel 69dd3cd4a5 chore: merge main into asset graph view 2026-05-20 15:16:31 +00:00
Ruben Fiszel 00221128cb fix: cgroup-aware DuckDB memory_limit + allocator memory release (#9245) 2026-05-20 14:05:16 +00:00
Ruben FiszelandClaude Opus 4.7 9111f8908d feat(nsjail): make tmpfs size configurable via instance setting (#9261)
* feat(nsjail): make tmpfs size configurable via instance setting

Adds a new `nsjail_tmpfs_size_mb` instance setting that overrides the
size of the `/tmp` tmpfs mount inside the nsjail sandbox across all
languages. When unset, the existing per-language defaults (500MB or
800MB) continue to apply, so no behavior change for existing
deployments.

The setting is exposed under Settings → Jobs and is read at job
execution time, so changes take effect on the next job without a
restart.

Fixes WIN-1963

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(nsjail): unify default tmpfs size to 800MB

Previously each executor passed its own per-language default (500MB or
800MB) to resolve_nsjail_tmpfs_size. Unify on a single
DEFAULT_NSJAIL_TMPFS_SIZE_BYTES constant (800MB) so the placeholder
behavior is consistent across languages.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(nsjail): resolve tmpfs size outside ruby download closure

The download.ruby config render runs inside a sync closure passed to
par_install_language_dependencies_seq, so `.await` on
resolve_nsjail_tmpfs_size() was a compile error under the `ruby`
feature. Resolve the size once before the closure and capture the
string instead.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(nsjail): rename resolver to *_bytes and clarify fallback

Addresses CI review feedback:
- Rename `resolve_nsjail_tmpfs_size` to `resolve_nsjail_tmpfs_size_bytes`
  so the returned unit is unambiguous at the call site (cubic P2).
- Fix the `NSJAIL_TMPFS_SIZE_MB` doc comment that still said "per-language
  default" — there is no per-language fallback anymore, all unset
  values resolve to the unified 800MB `DEFAULT_NSJAIL_TMPFS_SIZE_BYTES`
  (codex/pi P2).
- Expand the resolver doc to call out that `Some(0)` and negative values
  also fall back, since the match arm is `Some(mb) if mb > 0`.

No behavior change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 13:24:49 +00:00
2a780ad87a feat: resolve relative imports from local content in script/flow preview (#9233)
* feat: thread temp_script_refs into preview jobs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: resolve python preview relative imports from temp script refs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: use local relative imports in wmill script preview

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: use local relative imports in wmill flow preview

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add temp_script_refs to Preview and FlowPreview openapi schemas

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: pass temp_script_refs to bun lockfile gen for no-lock preview

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: route script preview through shared buildPreviewTempScriptRefs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: resolve local relative imports in wmill app dev inline scripts

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address cubic review — bundle cache key, preview-mode gate, error masking

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf: skip dep-tree build when previewed script has no relative imports; narrow old-backend classifier

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address review issues (preview-only gate, bundle preview, app dev cwd)

Three P1s flagged in repeated codex/pi reviews on PR #9233:

- Gate _TEMP_SCRIPT_REFS extraction on JobKind::Preview (bun + python
  executors) and propagation in worker_flow on JobKind::FlowPreview. job.args
  includes caller-controlled request args, so honoring this key on deployed
  runs would let a caller swap import resolution to local content uploaded
  via /raw_temp.
- run_bundle_preview_script now injects temp_script_refs into PushArgs.extra,
  mirroring run_preview_script — closes the silent data drop for the bundle
  preview path.
- wmill app dev chdirs to the wmill.yaml root before buildPreviewTempScriptRefs
  and restores after, so the `cd <app>__raw_app && wmill app dev` invocation
  (cwd is the raw_app folder, no app_folder arg) still walks sibling workspace
  scripts like f/lib.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ee): bump ee-repo-ref to 5b347d6 (handle_python_deps arity fix)

Picks up the EE arity fix so cargo_test + check_ee_full compile cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ee): bump ee-repo-ref to 52e273d (agent-workers bundle path arity fix)

Picks up windmill-ee-private 52e273d which adds the missing &None arg to
compute_bundle_local_and_remote_path in windmill-api-agent-workers/src/ee.rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ee): bump ee-repo-ref to 2d6ffd3 (EE main merged in)

Previous bump pinned an older EE commit, missing the audit-log object-store
export module (EE PR #579, commit ec3cd35) and other EE main updates. The
CE backend's `crate::ee_oss::anchor_audit_logs_s3_checkpoint_env_var` and
`export_audit_logs_to_object_store` references need the new EE definitions.
Merged origin/main into the EE branch and pinned the merge commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to b0c87b1272c25dca4aa9148c87fb024a9d9ef322

This commit updates the EE repository reference after PR #583 was merged in windmill-ee-private.

Previous ee-repo-ref: 2d6ffd32c99bd93e79cf78675cb89499a81b17e1

New ee-repo-ref: b0c87b1272c25dca4aa9148c87fb024a9d9ef322

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-20 12:44:29 +00:00
Ruben FiszelandClaude Opus 4.7 aa12c66c25 feat(snowflake): derive public key from private key when omitted (WIN-1959) (#9251)
* feat(snowflake): derive public key from private key when omitted (WIN-1959)

Snowflake key-pair auth needs a SHA256 fingerprint of the public key for
the JWT iss claim, but the public key is mathematically derivable from
the RSA private key. Other tools (e.g. Power BI) only require the
private key, so requiring users to supply both is redundant. When
public_key is missing, fall back to deriving it from private_key (PKCS#8
or PKCS#1 PEM) instead of erroring out.

Fixes WIN-1959

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(snowflake): treat empty public_key/private_key as missing

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 06:48:10 +00:00
Ruben Fiszel 631582f6e3 Merge branch 'main' into feat/asset-graph-view 2026-05-19 17:53:31 +00:00
Ruben FiszelandClaude Opus 4.7 26f3cbef25 fix: bound resource/variable interpolation recursion depth (WIN-1957) (#9243)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 15:39:42 +00:00
88c1493145 feat: add flow_user_state(key) to QuickJS input transform sandbox (WIN-1947) (#9093)
* Add flow_user_state(key) to QuickJS input transform sandbox

* fix: use root flow id for flow_user_state in QuickJS sandbox

* fix: url-encode key in get_flow_user_state

* fix: stub flow_user_state in eval contexts without by_id

---------

Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
2026-05-19 13:49:57 +00:00
Ruben FiszelandClaude Opus 4.7 8b7f7b37bd fix: don't fail flow on AlreadyCompleted after zombie restart (#9214)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 23:11:37 +00:00
Ruben FiszelandClaude Opus 4.7 bd05bcadde fix: validate entrypoint override to prevent worker code injection (GHSA-wxjq-w5pj-jqhx) (#9204)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 09:01:05 +00:00
Ruben FiszelandClaude Opus 4.7 f8467f38c8 fix: prevent cross-tenant DNS poisoning via writable /etc in nsjail (#9194)
* fix: bind /etc resolver files read-only in nsjail sandboxes

* docs(nsjail): explain why per-file /etc resolver binds are load-bearing

The explicit /etc/hosts, /etc/resolv.conf and /etc/hostname binds look
like removable duplication of the read-only /etc bind above them. They
are not: on Kubernetes those files are separate kubelet bind-mounts on
top of /etc and nsjail's read-only remount is non-recursive, so without
these shadow binds they stay writable and a job can persist cross-tenant
DNS poisoning for the pod lifetime. Comment guards against a future
"dedup cleanup" silently reintroducing the vulnerability.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(nsjail): shorten the load-bearing-bind comment to 3 lines

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-05-17 12:54:38 +00:00
Ruben FiszelandClaude Opus 4.7 19a386ce55 fix: atomic AND-join gate + preserve resolved partition; drop scratch artifacts
Addresses local-review findings before PR:
- P1: record_and_check_join_slot was a non-atomic check-then-act on a
  pooled connection; concurrent completion of a subscriber's last two
  partition-bearing inputs on different workers could double-dispatch.
  Now one transaction guarded by a tx-scoped advisory lock keyed on
  (workspace, subscriber, partition) so the gate fires exactly once.
- P2: the preprocessed-args overwrite in result_processor replaced args
  wholesale, dropping a partition resolved by resolve_partition_for_job;
  the UPDATE now preserves an existing persisted partition key.
- P2: gate resolve_partition_for_job on a cheap code.contains check so
  non-pipeline script jobs skip the annotation scan on the hot path.
- P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a
  local scheduler lock accidentally committed; gitignore the lock.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 08:13:29 +00:00
Ruben FiszelandClaude Opus 4.7 a888ef27f1 feat: resolve pipeline partition at job execution time
Stage C: in handle_code_execution_job, once the script content is loaded,
parse the // partitioned annotation (free here) and resolve the concrete
partition once — schedule fire-time (scheduled_for anchor, not wall-clock)
for time kinds, triggering payload for dynamic. The value is injected
into the in-memory args the body sees (via a shadowed job clone) and
persisted back to v2_job.args so dispatch_asset_triggers propagates the
same value down the cascade. Already-set (explicit/backfill/cascade)
partitions are never re-resolved (run identity immutable); unresolvable
partitioned runs fail with a clear error. Integration test exercises the
full worker loop + cascade propagation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-16 19:41:21 +00:00
Ruben FiszelandClaude Opus 4.7 81b5736106 fix: atomic bundle cache writes to prevent parallel cold-load race (#9186)
* fix: atomic bundle cache writes to prevent parallel cold-load race

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: trust-but-replace in atomic_publish_dir to never trust a stale partial cache dir

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: simplify atomic_publish_dir and add content-addressed rename-failure fallback

Revert the destroy-then-recreate dir swap (introduced concurrent-publisher
edge cases: spurious Err under a real herd, EACCES masking a stale partial)
back to a single atomic rename. Add the content-addressed exists-fallback to
atomic_write_file_bytes/atomic_copy_file so the loser of a publish race (and
Windows, where rename cannot replace an open/existing destination) treats the
already-published identical cache as success instead of failing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 21:04:16 +00:00
Ruben Fiszel b37e6830fe merge 2026-05-15 14:16:30 +00:00
69b3141e03 fix: apply pip_local_dependencies filtering to deployed scripts with populated lockfiles (#9178)
* fix: apply pip_local_dependencies filtering to deployed scripts with populated lockfiles

* refactor: share pip_local_dependencies filtering helper, log ignored deps

* test: split pure filter core out for unit testing, cover #-preservation

---------

Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-05-15 12:46:37 +00:00
e1819313e1 fix: aggregate wait time should target the true root job, not flow_innermost_root_job (#9177)
* fix: aggregate wait time should target the true root job, not flow_innermost_root_job

* refactor: reuse get_root_job_id helper for wait-time aggregation

Instead of duplicating the root_job → flow_innermost_root_job →
parent_job fallback chain inline, call the existing get_root_job_id()
helper (the same one used by push_next_flow_job) and filter out the
self-id case so standalone scripts still skip aggregate insertion.
Behaviorally identical, single source of truth.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 12:24:29 +00:00
Ruben Fiszel d48d61cc79 feat(otel-tracing-proxy): configurable tracing MITM NO_PROXY hosts (#9169)
* feat(otel-tracing-proxy): configurable NO_PROXY hosts

* refactor(otel-tracing-proxy): NO_PROXY only governs job-side bypass

* fix(otel-tracing-proxy): restore empty NO_PROXY default

* test(otel-tracing-proxy): unit tests for NO_PROXY normalization

* fix(otel-tracing-proxy): gate normalize_no_proxy_hosts to EE features
2026-05-14 14:03:47 +00:00
Ruben Fiszel 33bf01b627 fix(python): preserve strings containing Infinity/NaN in result JSON (#9149)
* fix(python): preserve strings containing Infinity/NaN in result JSON

* test(python): add sanity checks for Infinity/NaN in results

* refactor(python): use string-aware regex callback for single-pass cleanup

* refactor(python): compact regex + handle backslash-escape parity

* perf(python): short-circuit cleanup when no Infinity/NaN/NUL in result
2026-05-13 15:49:36 +00:00
Ruben FiszelandClaude Opus 4.7 4d0f2c26a1 fix(bun): pass --preserve-symlinks on unbundled execution (#9147)
* fix(bun): pass --preserve-symlinks on unbundled execution

Bun 1.2/1.3 moved its global package cache to a content-addressed
layout and the installer now creates a single directory symlink from
node_modules/<pkg> to the cache entry. Without --preserve-symlinks,
Bun resolves modules from each file's realpath, so any require/import
inside an installed package walks up from cache_nomount/bun/... and
never finds the sibling deps living under <job_dir>/node_modules/.

This manifested as e.g.
  ENOENT while resolving package 'zod/v3' from
  '/tmp/windmill/cache_nomount/bun/@langchain/core@1.1.44@@@1/dist/...'
on //nobundling scripts that pull @langchain/core, even though zod is
correctly installed alongside it in node_modules.

The bundled execution path already had --preserve-symlinks since #4132
(needed because we symlink the cached bundle file into the job dir).
The unbundled path didn't, because at the time Bun installed via per-
file hardlinks and the realpath of node_modules entries was the job
dir itself. The Bun installer's layout change made the flag necessary
on the unbundled path as well.

Add the flag to all three unbundled `bun run` invocations:
- nsjail unbundled path
- non-nsjail unbundled path
- dedicated worker (always unbundled)

This also fixes a latent bug on the first run of any bun script that
imports a package whose internals reference siblings (the build_cache
path runs unbundled this round while it builds the bundle for next
time).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(bun): regression test for nobundling + transitive require resolution

Adds an integration test that mirrors the original failure: a //nobundling
script importing @langchain/core, which (in its CJS internals) does
require('zod/v3'). Before --preserve-symlinks was added to the unbundled
bun run invocations, this failed with:

  ENOENT while resolving package 'zod/v3' from
  '.../cache_nomount/bun/@langchain/core@<ver>@@@1/dist/runnables/base.js'

The test covers the non-nsjail unbundled path. Reproducibility of the
pre-fix failure depends on Bun's installer choosing the directory-symlink
layout for the node_modules entry (the default on Bun 1.2/1.3+ with the
new content-addressed global cache that produced the user's error).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 13:04:31 +00:00
818cb31fbc fix: send flow push-loop ping outside transaction so zombie monitor sees it (#9136)
* fix: send flow push-loop ping outside transaction so zombie monitor sees it

* fix: keep flow push-loop ping using now() with reusable sqlx cache

---------

Co-authored-by: windmill-internal-app[bot] <1429786+windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-05-13 12:32:35 +00:00
centdix 17cf538a2d refactor: move ai providers to windmill-ai (#9120) 2026-05-12 14:09:31 +00:00
Ruben Fiszel e7cdc994ef Merge branch 'main' into feat/asset-graph-view 2026-05-11 22:38:47 +00:00
centdix 6f7d31e56b refactor: move ai image handling to windmill-ai (#9098) 2026-05-11 14:59:40 +00:00
Ruben Fiszel 8809b4ea83 all 2026-05-11 14:49:13 +00:00
centdixandwindmill-internal-app[bot] 27acbbf3d5 refactor: move ai sse plumbing to windmill-ai (#9059)
* docs: refine windmill ai refactor plan

* refactor: move ai sse plumbing to windmill-ai

* refactor: remove ai re-export shims

* fix: update ee ai memory ref

* chore: update ee-repo-ref to d3bc7fa85195b46b7a38d43c2f806520bf8b5454

This commit updates the EE repository reference after PR #560 was merged in windmill-ee-private.

Previous ee-repo-ref: ff35bf7cc198e13884b33654e1d6dbd8a8b314d3

New ee-repo-ref: d3bc7fa85195b46b7a38d43c2f806520bf8b5454

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 10:01:45 +00:00
Ruben FiszelandClaude Opus 4.7 2527c12511 fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090)
The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.

Three changes that together stop the propagation:

1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
   confirm every listed path exists on disk before writing
   `.valid.windmill`. On failure, wipe the directory and fall through
   to a fresh local install — the next install also self-heals the
   broken object-store entry by pushing a fresh tar.

2. After `uv pip install` succeeds, run the same RECORD check before
   queuing the piptar upload or writing `.valid.windmill`. A bad install
   never becomes the source of a broken tar in the object store.

3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
   for upload, so we never push an unfinalized archive (no end-of-archive
   marker) to the object store.

Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 21:21:45 +00:00
hugocasa b3a4874240 feat: parse windmill_failure field to tag run as failure (#9073)
* feat: parse windmill_failure field in job result to tag run as failure

* feat: preserve top-level fields when windmill_failure tags a run as failure

* fix: address review findings on windmill_manual_failure

* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases

* fix: prefer injected ManualFailure error over sibling name/message in OTel
2026-05-09 21:21:45 +00:00
hugocasa ae630e01e1 perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085) 2026-05-09 21:21:45 +00:00
Ruben Fiszel a49c35c755 fix: reject root-rooted paths in ansible playbook validator on windows (#9081) 2026-05-09 21:21:45 +00:00
Ruben Fiszel e4d675a006 perf(flows): cache resolved flow_env per flow execution (#9079)
* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures
2026-05-09 21:21:45 +00:00
Ruben Fiszel 18736fad34 perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078) 2026-05-09 21:21:45 +00:00
Ruben FiszelandClaude Opus 4.7 98ff146cfa fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090)
The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.

Three changes that together stop the propagation:

1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
   confirm every listed path exists on disk before writing
   `.valid.windmill`. On failure, wipe the directory and fall through
   to a fresh local install — the next install also self-heals the
   broken object-store entry by pushing a fresh tar.

2. After `uv pip install` succeeds, run the same RECORD check before
   queuing the piptar upload or writing `.valid.windmill`. A bad install
   never becomes the source of a broken tar in the object store.

3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
   for upload, so we never push an unfinalized archive (no end-of-archive
   marker) to the object store.

Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 17:56:51 +00:00
hugocasa dd5320205f feat: parse windmill_failure field to tag run as failure (#9073)
* feat: parse windmill_failure field in job result to tag run as failure

* feat: preserve top-level fields when windmill_failure tags a run as failure

* fix: address review findings on windmill_manual_failure

* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases

* fix: prefer injected ManualFailure error over sibling name/message in OTel
2026-05-08 15:57:34 +00:00
hugocasa 23af6c2ea3 perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085) 2026-05-08 15:57:17 +00:00
Ruben Fiszel d37277d234 fix: reject root-rooted paths in ansible playbook validator on windows (#9081) 2026-05-08 09:12:33 +02:00
Ruben Fiszel e1a7c75e19 perf(flows): cache resolved flow_env per flow execution (#9079)
* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures
2026-05-08 07:43:46 +02:00
Ruben Fiszel 2067e0719f perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078) 2026-05-08 04:43:07 +00:00
Ruben Fiszel 5bca03eacd fix: bubble handle_flow chaining errors to parent flow (#9058)
* fix: bubble handle_flow chaining errors to parent flow

* test: while-loop propagates inner forloop iterator failure
2026-05-06 23:47:13 +00:00
Ruben Fiszel c6f1c5e623 fix(bun): make hub script cache resilient to malformed lockfiles (#9063) 2026-05-06 23:44:11 +00:00
Ruben Fiszel eebaab9c87 fix(bun): propagate non-zero exit from generate_bun_bundle on no-DB path (#9051) 2026-05-06 10:00:46 +00:00
hugocasaandClaude Opus 4.7 6e5a21a9c7 fix(flows): inherit flow_env in sub-flow predicates (#9042)
* fix(flows): inherit flow_env in sub-flow predicates

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): align flow_env lookup with get_root_job_id and tighten gate

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): drop recursive CTE, root_job propagation suffices

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(flows): walk via flow_innermost_root_job to respect imported-flow scope

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(flows): remove flow_env API endpoint, dead code from deno_core era

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-05 20:11:38 +00:00
Ruben FiszelandClaude Opus 4.7 192866d519 fix(flows): don't bubble error when continue_on_error is on the last step (#9029)
* fix(flows): don't bubble error when continue_on_error is on the last step

When the last step of a flow (or branch/forloop) failed with continue_on_error
or skip_failures enabled, should_continue_flow resolved to false (because the
flow was at its last step), and the flow was completed with success=false.
This made parent flows / subflows treat the run as a failure even though the
user explicitly asked to continue past errors.

Detect this case and set success=true so the failure is captured in the
result but not propagated.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: explain why success is overridden post should_continue_flow

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 16:54:53 +00:00
Ruben FiszelandClaude Opus 4.7 1be62ea926 fix: stop sequential whileloop on iteration failure (#9028)
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 16:51:59 +00:00
70a5880d36 fix: distinguish AlreadyCompleted from execution failure on OTLP job span (#9004)
* fix: distinguish job-already-completed from execution failure on OTLP span

handle_queued_job's bool return type conflated two distinct Ok(false)
cases: a real race with another worker (Error::AlreadyCompleted) and any
job execution that returned an error via process_result. The outer "job"
span recorded "job already completed by another worker" for both, so
every failed Python/bun/etc. script ended up with that misleading
otel.status_message even though the real error was correctly recorded
on the inner job_postprocessing span.

Replace the bool with a JobOutcome enum (Completed / Failed { description }
/ AlreadyCompleted). Failed carries the truncated error string, so the
outer span's Status.message now reflects the actual cause.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: extract user-facing error from error_value for span description

Cubic flagged that capturing e.to_string() before the match meant
ExitStatus failures (the most common failure mode for script jobs)
ended up with the generic "exit status: …" string rather than the
script error extracted from job logs by extract_error_value.

Move the description capture to after error_value is built and
deserialise it as ErrorMessage to pull out the structured message.
Falls back to "Job failed" if the value isn't shaped that way.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: use permissive description extraction for OTLP status message

Cubic flagged that strict ErrorMessage parsing downgraded real failures
to a generic "Job failed" whenever the result wasn't shaped as
{message, name} — e.g. agent-worker's "See logs for more details" raw
string, or runtime-written result.json files with a different shape.

Switch to parsing as serde_json::Value and pulling .message out if it's
a string, falling back to the whole value if it's a bare string. Only
fall back to "Job failed" when neither is available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-05-01 18:25:18 +00:00