22 Commits

Author SHA1 Message Date
Pyra ab3c0206d7 fix: support @typechecked decorator in Python relative imports (#8495)
WindmillFinder's ModuleSpec lacked origin, so __file__ was never set on
loaded modules. inspect.getfile() then raised "is a built-in module",
breaking typeguard's @typechecked and anything else that introspects
module source. Use spec_from_file_location() which sets origin correctly.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
2026-08-17 12:30:25 +02:00
Ruben Fiszel d97f380c87 test: stop stranding sqlx pool permits in run_in_isolated_thread (#10707) 2026-08-14 21:15:10 +02:00
Ruben Fiszel 7155a0bb96 feat: Data Pipelines alpha (#9193)
* feat: add workspace asset graph view

Workspace-wide canvas of assets and their producer/consumer scripts,
reachable from the assets page. Left-to-right layered layout via
d3-dag sugiyama, rendered with @xyflow/svelte (same stack as the
flow editor). GET /w/:ws/assets/graph returns deduped nodes + edges.

Follow-ups: filters (kind/folder/search), node detail drawer, inline
script edit from a clicked node.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* all

* all

* update

* all

* all

* all

* feat(pipeline): output-kind picker and per-(lang, output) templates

Add a third stage to PipelineInsertMenu that asks what kind of asset the
new script will produce (datatable / ducklake / s3 parquet / s3 object /
none). The picked kind drives a real wmill SDK skeleton — typed
datatable inserts, ducklake CREATE+INSERT, s3 parquet COPY, etc. — with
the upstream asset auto-wired as the input source when added from an
asset node. Reorder languages to bun → duckdb → python → sql so
data-shaped languages surface first.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* chore(main): release 1.693.4 (#8994)

* chore(main): release 1.693.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit (#8997)

* feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: include .yaml variants in collections/roles requirements lookup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries (#9000)

* fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries

PR #8940 stopped lowercasing in sanitizeForFilesystem to fix #8939, where
a raw-app runnableId like CamelCaseTSRunnable produced a CamelCase YAML
metadata file but a lowercased code file, making them desync and
register as duplicate runnables on push.

That fix overshot. sanitizeForFilesystem is also reached by
newPathAssigner, which serves normal apps and flows where the input is
the script's human summary ("Get Users Data") rather than an identifier.
There the on-disk filename is the only artifact — there's no companion
YAML to keep in sync — so lowercasing was the right behavior. Removing
it changed both the on-disk filename and the !inline reference in
app.yaml / flow.yaml from get_users_data.inline_script.ts to
Get_Users_Data.inline_script.ts on the next pull, surfacing as
unwanted case churn for users updating to 1.693.x.

Add a preserveCase option to sanitizeForFilesystem (default false →
lowercase). newRawAppPathAssigner opts in; newPathAssigner stays on
the default. Update unit tests accordingly and add an end-to-end
raw-app round-trip in raw_app_sync.test.ts that pushes a CamelCase
backend runnable, pulls it back, and asserts both YAML and code file
preserve case with no lowercase orphan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(cli): use readdir for exact-case orphan check on Windows

The CamelCase round-trip test used fileExists("camelcasetsrunnable.ts")
to assert no lowercase orphan was produced, which false-positives on
Windows since the filesystem is case-insensitive and resolves the
lookup to the existing CamelCaseTSRunnable.ts. Switch to readdir +
toContain so the exact on-disk casing is compared identically on Linux
and Windows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): wmill-lock.yaml auto-fill + --rehash-only + path-prefix dedup (#8978)

* fix(cli): canonical lockfile hashes + lock upgrade migration to v3

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): use __app_hash subpath in rehash missing-entry check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): run sync pull lockfile auto-fill regardless of changes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: regenerate system prompts for new lock and rehash-only commands

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on lock upgrade

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): drop v3 marker; always run fallback; fail-fast on unknown lockfile version

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): drop yaml-round-trip legacy hash variant; recover via --rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): include legacy hash in script push staleness warning check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert(cli): drop canonical hash formula; keep raw-bytes hashing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* perf(cli): reuse change-tracker map for sync pull lockfile auto-fill

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): pin lockfile hash + yaml format and cover regression cases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): byte-stable snapshot tests for flow.yaml format

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add app and script-metadata yaml snapshot fixtures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address claude review on rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(cli): factorize script-path to remote-path derivation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address claude + cubic review (dry-run mutation, rehash short-circuit)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(cli): make rehash a subcommand and factorize fs walks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): normalize line endings in yaml snapshot tests for windows ci

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on rehash + auto-fill

- Flat-layout scripts now clearGlobalLock before rehash write so legacy
  ./-prefixed duplicates get cleaned up (matches flow/app behavior).
- Add MalformedLockfileError; sync pull auto-fill re-throws it alongside
  UnknownLockVersionError instead of silently warning + continuing.
- Document the legacy step-removal false-negative in
  isFlowDirectlyStale / isAppDirectlyStale and the categorizeLocalFiles
  ignore-filter invariant.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use otel.status_message for OTLP Status.message on failed jobs (#8995)

tracing-opentelemetry only recognizes otel.status_code and
otel.status_message as fields that map to the OTLP Status proto.
The previously-used otel.status_description fell through to the
generic attribute recorder, leaving Status.message unset and
preventing OTLP consumers from filtering spans on error status.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: route email trigger path through standard info channel (#8996)

* docs(skill): document email triggers and S3 attachments

Add an "Email triggers" section to the triggers skill covering the
local-part config, the parsed_email/raw_email/email_extra_args payload,
the URL-style extras convention, where to find trigger_path (only with
a preprocessor, at event.trigger_path), and — most importantly — that
binary attachments are uploaded to the workspace S3 bucket and surface
as `{ s3: "windmill_emails/<job_id>/attachments/<filename>" }`. Scripts
must use wmill.loadS3File / wmill.load_s3_file to read them.

Also pulls EmailTrigger into the schema mappings so a real
`email_trigger.schema.yaml` is generated, and adds Email/Azure to the
trigger kinds list in the CLI agent guidance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref for email trigger path fix

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 26184ab7a4aadfc529dcedf038aa08d36c7ad381

This commit updates the EE repository reference after PR #553 was merged in windmill-ee-private.

Previous ee-repo-ref: 318a46897a605dc9be3817901f35ba5a99a0a525

New ee-repo-ref: 26184ab7a4aadfc529dcedf038aa08d36c7ad381

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update git sync version to 1.693.5

* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999)

* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pg): wrap encoder errors with arg context, add fallback test

Followups on #8999 review:

- Wrap rust-postgres "error serializing parameter N" failures with the arg
  name, JSON value kind, and asserted Postgres type plus a hint about an
  explicit cast — so users see actionable context instead of an opaque
  WrongType.
- Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree
  on the Type for every recognised arg_t when the JSON value matches its
  natural Rust kind. Catches future drift if either side changes.
- Integration test for the prepare + query_raw fallback path: confirms
  unrecognised arg_t (custom enum) is routed through prepare and the
  server-resolved type appears in the failure surface — flips into a
  test failure if a regression accidentally routes unrecognised types
  through query_typed_raw.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add otyp_inferred flag + regex-based placeholder renumbering

Two follow-ups from the review of #8999:

1. **Issue #1 (Number/Bool + explicit text decl in WHERE)**

   Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets
   it `true` only at the "no info → fall back to text" site (bare `$N`,
   no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep
   it `false`.

   In `convert_val` this flag distinguishes:
   - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce
     `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works
     (`text = text` operator). Pre-#8988 behaviour, restored.
   - parser-default text (bare `$N`) — bind the value's natural Rust
     type so the regression case (`Value::Bool` against a real `bool`
     column via `CAST AS bool`) keeps working.

   `Arg` is in `windmill-parser`; the new field has `#[serde(default)]`
   so persisted signatures stay backward-compatible.

2. **Issue #4 ($5/$50 substring rewrite collision)**

   Replace the per-index `String::replace` chain (which turned `$50`
   into `$10` when oidx=5 was processed first) with a single regex
   pass. `\d+` is greedy, so `$5` and `$50` match as distinct units;
   indices outside the mapping are left intact.

3. Tests:
   - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline-
     cast/decl/mixed shapes.
   - executor unit: `convert_val_bool_against_every_arg_t` and
     `convert_val_*_number_*` split each text-like target into explicit
     vs inferred expectations.
   - executor unit: `renumber_sparse_placeholders_no_collision`.
   - integration: `test_postgresql_arg_type_combinations` adds 4 cases
     covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and
     sparse positional args ($5/$50).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality

Backend:

1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s
   `ToSql for String` / `FromSql for String` reject `Kind::Enum` and
   `Kind::Domain` even though the wire format is plain UTF-8. The wrapper
   accepts those kinds in both directions. End result: explicit
   `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the
   ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col`
   results come back as JSON strings instead of erroring at the FromSql
   layer.

2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these
   arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text /
   non-temporal arg_t fell through to `Box<String> + TEXT`, which then
   failed at the server (no implicit cast text→numeric in expression
   context). Now strings are parsed into the matching native type with
   clear error messages on parse failure.

3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering
   (which fixed the `$5/$50` substring collision but still walked through
   string literals and comments, mangling `'price: $5'` etc.) with a
   walk over `parse_pg_statement_arg_positions` — the same
   string/comment/dollar-quote-aware tokenizer used for index discovery.
   Adds `parse_pg_statement_arg_positions` to the parser's public API.

SDK:

4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now
   stringifies bigints before serialisation; the executor accepts
   numeric strings into BIGINT arg slots via the existing
   `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split
   so `BigInt` always resolves to `BIGINT` (was reaching
   `Number.isInteger(BigInt)` which returns false → wrong default).

5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column
   now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers
   primitive types only (number / bigint / string / boolean); mixed or
   nested arrays still fall back to JSON. Mixed int/float widens to
   `DOUBLE PRECISION[]`.

6. **`.query()` positional bug**: previously the `.query()` method
   abused the template-tag builder, which appended `$N::TYPE` after the
   user's literal SQL string instead of binding by position
   (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()`
   builds the executor-shaped content directly: a `-- $N argN (TYPE)`
   declaration block followed by the user's SQL verbatim.

Tests:

- Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments`
  asserts string literals, comments, and dollar-quoted blocks don't
  produce positions (so renumbering doesn't mangle them).
- Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling`
  uses the new position-aware path and includes string-literal + comment
  + `$$…$$` cases. Existing convert_val tests grow to cover new
  String→numeric/real/double/oid/bool arms.
- Integration: `test_postgresql_arg_type_combinations` adds 13 cases
  (enum round-trip both directions, string→numeric/real/double/bool/oid,
  string-literal `$N` non-mangling). The prepare-fallback test now
  asserts SUCCESS (not failure) for enum encoding via AnyTextValue.
- SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests)
  exhaustively covering inferSqlType primitives + arrays,
  parseTypeAnnotation, datatable() template tag (with all the new
  shapes — BigInt, homogeneous arrays, RawSql, schema preamble),
  datatable().query() positional, and ducklake() shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache)

Found while exhaustively probing custom-type DX: every cached-connection
reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL`
deallocates *all* prepared statements server-side — including the typeinfo
statements that tokio_postgres caches per-Client to resolve custom enum /
domain Oids. tokio_postgres still held `Statement` objects whose names
the server had forgotten, so the next custom-type query failed with
intermittent "prepared statement \"sN\" does not exist" errors. The
failure was easy to reproduce: any sequence that forced typeinfo lookup
for two different custom-type kinds on the same cached connection (e.g.
enum followed by domain) would hit it.

Replace `DISCARD ALL` with a curated reset that explicitly targets the
state we actually care about, *without* touching prepared statements:

  RESET ALL                     — GUC parameters (search_path, application
                                  _name, statement_timeout, …)
  RESET SESSION AUTHORIZATION   — undoes both `SET SESSION AUTHORIZATION`
                                  and `SET ROLE` (RESET ALL does NOT —
                                  these aren't GUC parameters, so without
                                  this an elevated role from a previous
                                  job would silently leak)
  UNLISTEN *                    — drops LISTEN registrations
  CLOSE ALL                     — closes open cursors

Trade-off: temp tables, advisory locks (session-scoped), and user-created
PREPARE statements may persist across cached-connection reuse — rare in
datatable / PG-script workloads. tokio_postgres's typeinfo cache survives
intact, so custom enum / domain queries are fast on subsequent reuse.

Tests:
- `test_postgresql_custom_types_on_cached_connection` — runs 10×
  alternating enum + domain queries on a cached connection. Pre-fix this
  failed with `prepared statement "sN" does not exist` after the first
  reuse; post-fix passes.
- `test_postgresql_set_role_does_not_leak_across_cached_connection` —
  switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres
  role, then runs a follow-up job and asserts current_user/session_user
  are restored. Specifically catches the case where someone might switch
  back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION
  AUTHORIZATION) and silently introduce a permission-leak vector.
- All existing session-isolation tests
  (`test_postgresql_cached_connection_resets_session`,
   `test_postgresql_single_worker_session_isolation`,
   `test_postgresql_100_jobs_cached`) continue to pass.

Found via end-to-end probing of datatable / PG-script DX, not previously
covered: the existing isolation tests only did `SET ROLE postgres`, the
connecting user, so the leak was invisible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): address PR #8999 review (cubic + claude)

cubic (P1, real bug):
- `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but
  chrono `NaiveTime` only encodes for TIME (same caveat as the scalar
  arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz`
  assignment cast at the column site. Add an explicit unit test.

claude (#1, silent failure → explicit error):
- `Bool` + explicit `(char)` / `(character)` decl previously silently
  bound BOOL, hoping the server would cast at the use site — but PG has
  no implicit `bool→char` and the resulting error
  ("operator does not exist: bool = char") was opaque. Now error at
  bind time with an actionable hint to use `bool` decl or pass the
  value as a "t"/"f" string.

claude (#2, asymmetry doc):
- Object/Array still coerce to text on `matches!(typ, Typ::Str(_))`
  (covers both explicit AND inferred-default text), unlike Bool/Number
  which key on `explicit_text_target`. The asymmetry is intentional
  (no implicit `jsonb → text` cast in expression context vs PG having
  implicit `bool/int → text` casts) — added a body comment so future
  maintainers don't try to "align" them.

claude (#3, perf):
- `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions`
  walked the SQL tokenizer twice. Fold into a single pass that derives
  the index set from the position list.

claude (#4, fmt drift):
- `cargo fmt` over the parser crates I touched with perl scripts in the
  earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py,
  rust,graphql,yaml,r}). Net cosmetic.

claude (#5, parseTypeAnnotation):
- One-line caveat in the SDK's `parseTypeAnnotation` that the returned
  string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns
  `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a
  real PG type, but the only consumer just checks `!== undefined`).

While here — discovered + fixed independently while exhaustively probing
DX:

- **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET
  SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's
  `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing
  intermittent `prepared statement "sN" does not exist` errors on
  custom-type queries after cached-conn reuse. New regression tests:
  `test_postgresql_custom_types_on_cached_connection` and
  `test_postgresql_set_role_does_not_leak_across_cached_connection`
  (the latter catches the case where someone might switch back to
  `RESET ALL` alone and silently introduce a permission-leak vector —
  RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION).

- **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix
  `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00")
  and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") —
  neither parseable by `date-fns parseISO`, JavaScript `new Date()` is
  lenient enough to handle them but several frontend `App*Input.svelte`
  components use parseISO and fail silently. Switched to ISO-8601 with
  `T` separator and `+00:00` offset; arg-parsing path still accepts the
  legacy " UTC" suffix for back-compat.

Test coverage:
- 17/17 unit (`pg_executor::tests`)
- 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`)
- 27/27 parser (`windmill-parser-sql`)
- 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling

Found while probing PG-script DX with millions of numeric cells:

1. **Numeric precision-loss warning**: `numeric` results are still serialised
   as JSON Number (back-compat — switching to JSON String would silently
   break user code doing arithmetic on results), but we now detect
   `Decimal -> f64 -> Decimal` round-trip failure and emit a single
   job-log warning recommending a `::text` cast in the SQL. Bounded by
   `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic
   load + one fetch_sub on the hot path; first lossy value
   short-circuits to a single load thereafter). Worst-case overhead on
   a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic
   loads (vs. ~100ms unbounded).

2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned
   `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"`
   (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is
   what the apps `App*Input.svelte` components use, so timestamp values
   silently failed to round-trip into date pickers. Switch to ISO-8601
   (`T` separator + `+00:00` offset) on the result side; arg-parser
   continues to accept the legacy `" UTC"`-suffixed format for
   back-compat.

3. **Float NaN / Infinity results**: `Number::from_f64` returns None for
   NaN / ±Inf, which `pg_cell_to_json_value` was raising as
   "invalid json-float" — failing the *entire* query if any cell held
   one of these special values. Now serialise them as JSON strings
   ("NaN", "Infinity", "-Infinity") and let the rest of the row come
   through. Arg-side: `s.parse::<f64>()` already accepts the same
   strings.

Tests:
- `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit
  cases for the precision-loss predicate.
- `precision_check_budget_caps_per_query_overhead` — locks in the
  budget cap and the loss-flag short-circuit.
- All 9 PG integration tests + 17 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults

While probing PG-script DX further found three more frictions:

1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to
   `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`
   meant session-scoped advisory locks (`pg_advisory_lock`) leaked
   across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()`
   to the chain — `DISCARD ALL` covered this implicitly via
   `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it
   in the switch.

2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g.
   `-- $1 amount (numeric)`) but not provided in the args object was
   bound as NULL with no error / warning. Misspelling the key in the
   args object silently produced a row of NULLs — a notorious DX
   debugging trap. Now: collect the names of declared-but-missing
   args during dispatch and emit a single one-shot warning to the job
   logs at end-of-query naming each one. Bound NULL is preserved for
   back-compat.

3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries
   `arg.default = Some(Number(5))`, but the dispatch fell straight to
   NULL when the arg was missing. Now: respect the default —
   user-supplied value > declaration default > NULL. Also fixes the
   warning logic above (only warn for args that *don't* have a default).

Tests: existing 19 unit + 9 integration pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values

Two more frictions found while probing SDK end-to-end against a real
datatable resource:

1. **Multi-word array types lose the [] suffix in the parser**.
   `transform_types_with_spaces` recognises aliases for "double
   precision", "character varying", "timestamp with time zone", etc.
   but its return type was `&'a str` — only the bare alias, never with
   a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at
   the first space, so the regex's own `(?:\[\])?` array-suffix branch
   sees only `"double"` (not `"double precision[]"`); the `[]` was
   silently lost. Result: `$1::double precision[]` (which the SDK now
   emits for homogeneous float arrays via the new auto-tag) routed
   through `Value::Array → Type::JSONB` and the server failed with
   "cannot cast type jsonb to double precision[]".

   Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>`
   and re-check the trailing bytes after a multi-word match. If they
   start with `[]`, return `format!("{alias}[]")` — Owned. Single-word
   types and the no-match path keep returning Borrowed slices, so no
   allocation in the hot path.

2. **Array arms in `convert_vec_val` rejected stringified values for
   numeric / int* / bool / oid / real / double**. The scalar `convert_val`
   already parses strings into the matching native type for these arg_ts,
   but the array variant only accepted JSON-native counterparts. Sending
   `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for
   bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with
   "Mixed types in array". Now the array arms mirror the scalar ones —
   `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes
   round-trip cleanly.

Tests: 19 unit + 9 integration pass; existing parser tests cover the
multi-word array forms (the regex-cap behaviour didn't break for
single-word types, and Cow plumbing is transparent to all callers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files

CI failures: the perl-driven sweep that added `otyp_inferred: false` to
every `Arg { ... }` literal when I introduced the field in the parser
schema covered `src/lib.rs` files but missed:

  - parsers/windmill-parser-bash/src/lib.rs       (mass-edited but a
    later format pass un-applied a few sites)
  - parsers/windmill-parser-go/src/lib.rs         (same)
  - parsers/windmill-parser-graphql/src/lib.rs    (same)
  - parsers/windmill-parser-nu/tests/tests.rs     (test file — not
    swept the first time)
  - parsers/windmill-parser-ts/tests/tests.rs     (test file — same)

Also tightened the regex to handle `oidx: None` without the trailing
comma (some test files had the field as the last initialiser line).

`cargo build --features <CI feature combo> --workspace --all-targets`
is clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string

Two more frictions found while running the actual SDK end-to-end against
a live datatable resource:

1. **JS `Date`** fell into the typeof "object" branch and was tagged
   `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's
   `json → text → timestamptz` implicit cast chain, but `${date}` against
   a `timestamptz` column without a user-supplied cast bound the value
   as a JSON string and the comparison `timestamptz = json` failed. Now:
   `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`;
   `serializeArgValue` emits `Date.toISOString()` so the executor's
   `Value::String → TIMESTAMPTZ` arm parses it cleanly.

2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)`
   returns `"null"` per the JS spec, so the value reached the executor as
   JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL
   double. Fix: detect non-finite numbers in `serializeArgValue` and
   stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's
   `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals
   directly, and the result-side already renders the values as JSON
   strings (matching round-trip).

SDK unit tests grow from 42 → 44 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(pg): integration coverage for multi-word arrays + stringified array elements

Locks in the two array fixes from the previous commit
(`fix(pg): multi-word PG types with [] suffix lost the array-ness`)
with end-to-end cases in `test_postgresql_arg_type_combinations`:

- `double precision[]`, `character varying[]`, `timestamp without time
  zone[]` — verifies the parser keeps the `[]` suffix after multi-word
  alias resolution.
- `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies
  the array arms of `convert_vec_val` apply the same string-coercion
  the scalar arms do.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: fix indentation drift on otyp_inferred lines

cargo fmt cleanup of leftover indentation where the perl-driven sweep
that introduced the otyp_inferred field landed at the wrong column.
No behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: support assigning a worker tag to app inline scripts (#9002)

* feat: support assigning a worker tag to app/raw-app inline scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: omit empty tag field from inline script raw_code payload

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: shrink tag popover width

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat(pipeline): 2-col picker, draft path edit, save-all + leave guard

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* all

* update

* fix(cli): forward HEADERS env var on every backend fetch call (#9075)

Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped
the `HEADERS` env var, causing requests to fail behind auth gateways like
Cloudflare Access (same shape as #6421):

- `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed
  in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw
  `fetch` for the `skip_if_noop` query param.
- Script preview `/jobs/run/preview_bundle`.
- App dev `/jobs_u/getupdate_sse` SSE stream.
- `wmill docs` `/api/inkeep`.

All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()`
so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque
JSON parse failure.

Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that
403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and
otherwise reverse-proxies to the test backend, then runs `wmill sync push`
of a fresh script through the proxy. Negative case (no `HEADERS` env)
verifies the proxy actually gates; positive case asserts every request
including `/scripts/create` reaches the backend with the headers attached.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): add --parallel flag to generate-metadata (#9074)

* feat(cli): add --parallel flag to generate-metadata

* fix(cli): validate --parallel input and harden flush ordering

* perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078)

* fix(cli-tests): stabilize flow lock-gen race + Windows path (#9080)

* fix(cli-tests): stabilize flow lock-gen race + Windows path

Three CLI test failures on the latest main, all flaky on CI:

1. `Mixed Case Paths: pull and push flow with capitalized folder` and
   `Integration: Mixed scripts and flows with nonDottedPaths are
   idempotent`: flow create/update queues an async FlowDependencies job
   that fills inline-script lockfiles and rewrites flow.value. The tests
   pulled/pushed before the worker finished, so dry-run idempotency saw
   phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a
   `waitForFlowDependencyJob` helper that polls `/flows/get` for the
   latest `dependency_job` and `/jobs_u/completed/get` until it lands,
   and called it after each API/CLI flow write in both tests.

2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only,
   added in #9075): the new test built the CLI entrypoint via
   `new URL("..", import.meta.url).pathname`, which yields `/C:/...` on
   Windows and `Bun.spawn` rejected before reaching the proxy, leaving
   `rejectedRequests.length` at 0. Switched to
   `fileURLToPath` + `node:path.join` to match `cargo_backend.ts`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli-tests): use /flows/deployment_status to actually wait for dep job

CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob`
as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`,
but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60)
do not include that field. The helper exited on the first iteration
without polling.

Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns
`{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID
written into `deployment_metadata` in the same tx as the dep-job push
(backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by
the time the create/update API call returns, the response carries the
latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as
before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to
~32s, confirming the helper now actually waits instead of returning
immediately. The 404 short-circuit in `sync_pull_push.test.ts` still
works — `get_deployment_status` returns 404 when the flow is absent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(flows): cache resolved flow_env per flow execution (#9079)

* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures

* chore(main): release 1.698.0 (#9076)

* chore(main): release 1.698.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: reject root-rooted paths in ansible playbook validator on windows (#9081)

* fix(native-triggers): serialize Google channel renewal across replicas (#9060)

* fix(native-triggers): serialize Google channel renewal across replicas

`sync_all_triggers` runs every 5 minutes on every windmill-app replica
with no leader election. Multiple replicas were each rotating the
webhook token, creating a new Google watch channel, and racing the
trigger UPDATE — leaving the loser's new token (in `token`) and channel
(in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week
without the silent best-effort `delete_token_by_hash` ever logging a
warning.

Wrap each per-trigger renewal in a transaction and acquire the row with
`SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row
instead of duplicating the work. The lock spans `rotate_webhook_token`
→ Google API call → `update_native_trigger_service_config` and is only
released on commit. Re-checks `should_renew_channel` after acquiring
the lock so a replica that committed seconds earlier doesn't trigger a
duplicate renewal.

The pattern matches existing batch-cleanup paths in `monitor.rs`
(job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites.

Also logs at `debug!` when `delete_token_by_hash` finds no matching row,
so future investigations can distinguish "deleted" from "not found"
without changing the `Ok(false)` contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Address claude review:
- #5: per-skip log info -> debug (expected outcome under SKIP LOCKED)
- #2: warn moved out of delete_token_by_hash to the call site that knows the
  expected state (try_renew_channel_locked); other callers are race-prone and
  shouldn't warn
- #3: NULL service_config now warns (anomalous case)
- #4: post-Google-API DB-update + commit failures log distinctly so the
  channel-orphan case is grep-able

Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration,
mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the
existing 'ephemeral-' filter excludes them from user-token email/critical-alert
paths (no filter changes in 3 places). Orphans now self-clean via the existing
expiry sweep in monitor.rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Address second-round review:
- Claude #1 (P2): username_override_from_label now strips the 'ephemeral-'
  prefix for ephemeral-webhook-* labels, so created_by stays
  webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-...
  (preserves audit/job-list filter compatibility)
- Codex (P2): updated renew_channel doc — labels are no longer copied; rotate
  mints fresh ephemeral-webhook-google-{rd5} with 14d expiration
- Claude #3 (optional): test_rotate_webhook_token now asserts the rotated
  Google token has an ephemeral-webhook-google-* label and a populated
  expiration

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Reconsider the previous fixup: stripping the 'ephemeral-' prefix made
created_by no longer match token.label exactly, defeating the linking
purpose. Just allowlist 'ephemeral-webhook-' alongside the other
recognized webhook/email/ws prefixes — created_by becomes
ephemeral-webhook-google-XXXXX, matching token.label exactly. The
'ephemeral-' substring also informs operators that this is a
system-managed auto-expiring token vs a user-managed webhook trigger.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): bump svelte version in `wmill app new` template (#9084)

* fix(cli): bump svelte version in `wmill app new` template

The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte
compiler bundled in `wmill app dev` emits `$.delegated('click', ...)`
calls. The `delegated` export was added later, so 5.45.2 doesn't have
it — esbuild warns `Import "delegated" will always be undefined`,
replaces the call with `void 0`, and the page crashes at first
event-handler bind (white screen).

Bump to `^5.55.5` so the compiler and runtime stay in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(frontend): bump svelte version in raw_apps UI template

Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a
package.json with `svelte: "5.45.2"`. That works today only because
the bundled rolldown worker also pins 5.45.2 — when the worker is
upgraded past 5.51.1, the compiler will emit `$.delegated()` and the
runtime won't have it, producing the same white-page crash that hit
the CLI.

5.55.5 still exports `event` (used by the current bundled compiler),
so this is forward-compatible: it works with the 5.45.2 compiler now
and won't break when the worker is upgraded.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085)

* feat: parse windmill_failure field to tag run as failure (#9073)

* feat: parse windmill_failure field in job result to tag run as failure

* feat: preserve top-level fields when windmill_failure tags a run as failure

* fix: address review findings on windmill_manual_failure

* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases

* fix: prefer injected ManualFailure error over sibling name/message in OTel

* fix: hide _ENTRYPOINT_OVERRIDE jobs from script/flow history panel (#9088)

* fix(flows): populate error handler input args from failure picker (#9087)

* fix(flows): populate error handler input args from failure picker

* style(flows): fix indentation in failure-step branch

* fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090)

The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.

Three changes that together stop the propagation:

1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
   confirm every listed path exists on disk before writing
   `.valid.windmill`. On failure, wipe the directory and fall through
   to a fresh local install — the next install also self-heals the
   broken object-store entry by pushing a fresh tar.

2. After `uv pip install` succeeds, run the same RECORD check before
   queuing the piptar upload or writing `.valid.windmill`. A bad install
   never becomes the source of a broken tar in the object store.

3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
   for upload, so we never push an unfinalized archive (no end-of-archive
   marker) to the object store.

Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(main): release 1.699.0 (#9082)

* chore(main): release 1.699.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat(cli): auto-infer args for `wmill app push` (#9091)

Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`)
with no args. The local path defaults to CWD, and the remote path is
derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/
`__app`/`__raw_app` suffixes stripped. Either, both, or neither
positional argument can be passed.

Also resolves `file_path` against the user's original CWD before
`resolveWorkspace` may chdir to the wmill.yaml root, so a relative
`file_path` argument is interpreted from where the user invoked the
command (previously it could resolve against the wrong directory).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* fix(pipeline): live-update graph for annotations and body assets

* fix(pipeline): persist draft body edits across node switches

* fix(pipeline): persist live writes per draft to keep output node fresh after switch

* feat(pipeline): animate graph edges only while a runnable is executing

* feat(pipeline): add run button on script nodes + recomputing hint on preview

* feat(pipeline): compact preview layout, two-way Test/Run sync

* fix(pipeline): test button cross-browser placement (no overflow trick)

* style(log-viewer): replace took/mem-peak labels with timer/cpu icons

* style(log-viewer): hyphenate Auto-scroll label and prevent wrapping

* style(log-viewer): lowercase auto-scroll label, force vertical scrollbar

* style(log-viewer): force horizontal scrollbar instead of vertical

* fix(log-viewer): scope overflow-x to top bar so pre doesn't drive panel width

* fix(pipeline): overlay live body-asset writes for persisted scripts too

* fix(pipeline): persist inferred body assets at save so edges survive page reload

* fix(pipeline): snapshot live draft writes at persist time so they survive reload

* fix(pipeline): keep inferred body writes on the canvas across selection changes

* fix(pipeline): untrack inferredWrites cache mutation to break effect loop

* fix(pipeline): refetch asset graph after persisted-script save

* feat(pipeline): optional AI prompt when creating a pipeline script

* all

* all

* test: cover asset-trigger dispatch end-to-end through worker

* feat(pipeline): split-button Test with optional downstream cascade

* feat(pipeline): cascade option on graph Run + match button heights

* style(pipeline): match caret bg/text to Test button's accent-secondary

* feat(pipeline): split Run pill on graph node exposes cascade option

* feat: live run activity + status badges in pipeline asset graph

- folder-scoped queue poll lights up the downstream asset-trigger
  cascade (not just the launched script); zero requests at rest,
  catch-up for fast hops, auto-disarm when idle
- per-runnable node badge: last-run status + session run count
- animate unsaved/live-parsed edges (was unconditionally suppressed)
- background-pane click no longer clears selection
- run-bridge guarded so node selection/save no longer triggers a test

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: live activity log, optimistic badges, node-avoiding graph edges

- collapsible folder activity log (PipelineEventLog): live job feed,
  polls only while open/active, slow idle cadence, capped + pruned
- composable: observe mode + events list + run-count anchored to
  graph-open time (pre-existing history excluded)
- optimistic node badge: launched script shows running instantly via
  the zero-latency activeRunnable hint, keeps the polled run count
- activity pane height capped (min(18rem,40vh)) then scrolls
- route asset-graph edges through sugiyama-computed waypoints so they
  go around nodes instead of under them; bezier fallback for
  adjacent-layer / draft-overlay edges

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: prefetch all folder script assets so graph is stable on load

On pipeline load, eagerly infer body assets for every persisted folder
script and seed the existing inferredWritesByPath overlay, instead of
only filling it when a node is selected. Scripts whose persisted asset
rows are missing (e.g. object-form writeS3File) now have their edges
from first paint, so clicking a node no longer re-layouts the graph.
One-shot per (workspace, base-graph) load, untracked map reads,
generation-cancelled, pool-capped fetches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf: guard no-op poll re-layout; dedupe write-asset extraction

- skip reactive ids/states/events reassignment when unchanged, so an
  idle poll tick no longer re-runs the full sugiyama layout every 3-6s
- bound countedJobIds (rebuilt from eventsById in lockstep with prune)
- extract shared extractWrites() helper, replacing 4 copy-pasted
  write-asset filter/map blocks in the pipeline page
- compute activeRunnable node-id once, reuse for the active-edge set
  and the optimistic badge (flattened ternary); trim narrating docs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: live read-lineage overlay for inferred body assets

Renaming e.g. duckdb read_parquet('s3://...') / loadS3File now updates
the asset->reader edge live instead of only after Save re-derives the
persisted asset rows.

- extractReads() (+ shared refsByAccess) mirroring extractWrites
- inferredReadsByPath sticky cache, filled by handleAssetsChange and
  the load prefetch alongside writes
- replace the write-only overlay loop with one overlayLineage(map,
  access) helper invoked for both 'w' and 'r' (net DRY)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: detect S3 assets passed as SDK object arg in ts parser

Mirrors merged PR #9181 so feat/asset-graph-view is self-contained
(local origin/main is stale and lacks it). Object/{ s3, storage }
form of writeS3File/loadS3File is now detected, not only the bare
s3:// string literal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate wasm Cargo.lock + frontend package-lock

Lockfile churn from local wasm-pack (asset target) + npm operations
during the asset-graph work. No source/dependency-intent change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: revert to bezier graph edges; add parsing-assets hint

The sugiyama-waypoint routing looked worse than the original; revert
AssetGraphEdge/assetGraphLayout to the pre-routing bezier logic (same
as the flow editor's BaseEdge) and drop the now-unused route plumbing
from the canvas. Add a small 'Parsing assets…' hint shown while the
load-time prefetch sweep is still inferring folder scripts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: extract pure resolveGraph merge + unit tests

Move the ~230-line graphWithDraft precedence/merge (base < session-
inferred < draft-seeded < open-script-live, +read/write/annotation
overlays, +dedup) out of the 1648-line route into a pure, testable
resolveGraph() module; the route's graphWithDraft is now a thin
$derived. Behaviour extracted verbatim. 10 unit tests cover the
precedence matrix. Phase 1 of the state/render split.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: graph controls top-right, lift minimap, hide Save when unchanged

Controls -> top-right horizontal, no lock toggle; MiniMap !mb-10 so
it clears the activity bar; hide the per-script Save button when the
script is already at its latest save point (drafts still show Create).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: scope runtime-asset prune by id to spare static lineage rows

prune_runtime_assets deleted by (workspace_id, path, kind) tuple, so
trimming surplus usage_kind='job' rows for an s3 path also wiped the
static usage_kind='script'/'flow' producer rows for the same path —
silently breaking the asset-trigger cascade (fetch_producer_writes
found no writes; downstream never dispatched; required band-aid
re-syncs). Delete the surplus job rows by id instead; the inner query
is already scoped to usage_kind='job'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: don't re-pulse already-running jobs after they finish

The catch-up pulse re-added a completed job to the active set if its
start was within the (lagging) lookback window — even one we'd already
animated the whole time it ran — keeping its edges lit ~a poll
interval past completion (~5s after a 3.5s test). Track job ids seen
in-flight and skip the pulse for them; it still fires for hops whose
whole lifetime fell between two polls. Bound the set in lockstep with
eventsById; cleared on dispose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: don't catch-up-pulse the runnable launched from the graph

If the poll never sampled a launched run's in-flight window, the
catch-up pulse re-flashed its edges one tick after it correctly
stopped (the page already animated it zero-latency via activeRunnable).
arm(launchedId) records the launched runnable id; catch-up skips it.
Cascade hops (other ids) still pulse. launchedIds cleared on stop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: nudge graph controls left to clear panel toggle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: partition value resolver + asset-cascade propagation

windmill-common/partition: pure resolver — time kinds (tz/format/start
anchor) + dynamic $.a.b JSONPath; 9 unit tests. asset_dispatch:
read the producer's resolved partition and thread it into every
cascaded subscriber's args + trigger.partition, so a chain resolves
once at the top. No migration (cascade needs no spec lookup). Stage
1+3 of pipeline partition runtime; run-start resolution is Stage 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: show args form in compact pipeline preview when script has inputs

AssetGraphDetailsPane keeps the compact (hideArgs) preview but, via a new
previewPanel.argsAboveLogs flag, renders a compact SchemaForm between the
floating Test button and the logs/result panel when the script declares
inputs (e.g. a partitioned script needing a `partition` arg). The preview
pane also grows ~18pts so the args form doesn't shrink logs/result.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: parser join-mode (`// trigger all`) + script_trigger.join_all

Stage A: JoinMode{Any(default),All} + `// trigger any|all` directive in
parse_pipeline_annotations; TriggerSpec::is_partition_bearing() (path
contains {partition}); join_mode threaded through all 4 asset-parser
crates (ts/py/sql/yaml). Stage B: reversible migration adds
script_trigger.join_all; insert_script_trigger writes it; deploy path
sets it from the parsed annotation. No reader yet (AND-join dispatch is
the next stage) so runtime behaviour is unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: resolve pipeline partition at job execution time

Stage C: in handle_code_execution_job, once the script content is loaded,
parse the // partitioned annotation (free here) and resolve the concrete
partition once — schedule fire-time (scheduled_for anchor, not wall-clock)
for time kinds, triggering payload for dynamic. The value is injected
into the in-memory args the body sees (via a shadowed job clone) and
persisted back to v2_job.args so dispatch_asset_triggers propagates the
same value down the cascade. Already-set (explicit/backfill/cascade)
partitions are never re-resolved (run identity immutable); unresolvable
partitioned runs fail with a clear error. Integration test exercises the
full worker loop + cascade propagation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: AND-join barrier for partitioned pipeline subscribers

Stage D: a // trigger all subscriber no longer fires on any input. New
join_pending_inputs slot table keyed (workspace, subscriber, partition);
fetch_subscribers now returns join_all and the dispatch loop records each
partition-bearing input arrival, pushing the subscriber once only when
every partition-bearing input it declares is present for that partition.
Per-partition slots, cleared on fire (re-accumulate, no double-fire),
skew-immune (unlike debounce). Case-3 guard: an unpartitioned producer or
a reference (non-{partition}) input never fires a partitioned join.
Integration test covers wait/fire/isolation/no-double-fire.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: opt-in // debounce for asset-cascade subscribers (parser + schema)

Stage E1+E2. Parser: script-level // debounce <dur> + per-// on
debounce=<dur> override (edge wins, else script default, else none =
fan-out, unchanged); TriggerSpec::Asset carries the per-edge override;
split_trailing_kv_opts separates the ref from trailing key=val opts.
Schema/deploy: reversible migration adds script_trigger.debounce_s;
parse_duration_secs (bare int or <n>s|m|h|d, fail-safe on garbage)
resolves the effective per-edge window at deploy and writes it per row.
No reader yet (dispatch wiring is E3) so runtime is unchanged. New unit
tests for the parser directive and duration parsing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: apply opt-in debounce to asset-cascade subscriber dispatch

Stage E3. fetch_subscribers now also returns debounce_s; push_subscriber
builds real DebouncingSettings (delay + a (subscriber, partition) key,
so distinct partitions never collapse and latest-in-window falls out)
instead of ::default() when the edge opted in. Default stays no-debounce
(fan-out — the prior deliberate behaviour, now overridable rather than
reversed). Wiring test asserts the dispatched job carries the configured
window/key and an undebounced edge carries none.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: atomic AND-join gate + preserve resolved partition; drop scratch artifacts

Addresses local-review findings before PR:
- P1: record_and_check_join_slot was a non-atomic check-then-act on a
  pooled connection; concurrent completion of a subscriber's last two
  partition-bearing inputs on different workers could double-dispatch.
  Now one transaction guarded by a tx-scoped advisory lock keyed on
  (workspace, subscriber, partition) so the gate fires exactly once.
- P2: the preprocessed-args overwrite in result_processor replaced args
  wholesale, dropping a partition resolved by resolve_partition_for_job;
  the UPDATE now preserves an existing persisted partition key.
- P2: gate resolve_partition_for_job on a cheap code.contains check so
  non-pipeline script jobs skip the annotation scan on the hot path.
- P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a
  local scheduler lock accidentally committed; gitignore the lock.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: AND-join fires once under concurrent upstream completion

Regression for the check-then-act race fixed by the advisory-locked
transactional gate: releases N producer dispatches simultaneously via a
barrier and asserts the AND subscriber is pushed exactly once and the
slot is cleared. The invariant holds for the correct gate regardless of
interleaving; a non-atomic regression fails it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: fuller partitioned join + multi-hop pipeline coverage

Exercises a complex pipeline combining options end to end: two
partitioned producers fanning into a // trigger all join, then a
multi-hop downstream chain. Asserts the resolved partition propagates
unchanged at every hop, chain depth increments per hop, the AND barrier
fires exactly once, and a second partition opens an independent slot
with no cross-partition bleed across the whole graph.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor: simplify pipeline code per review (dedup, single-parse, constant)

- ParseAssetsOutput::new() collapses the 6-line annotation copy-paste
  across the 4 asset-parser crates to one call site.
- asset_dispatch: parse the cascade trigger object once and pass it to
  the depth/partition readers instead of deserializing it twice; add a
  TRIGGER_ARG constant for the previously stringly-typed key (3 sites).
- scripts deploy: drop a redundant debounce_default clone.
No behavior change; 29 parser + 6 dispatch integration tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: reap abandoned AND-join slots after a TTL (default 60d, per-slot)

join_pending_inputs slots are normally cleared when the join fires;
partial slots whose inputs never all arrive (upstream removed/renamed,
one-off dynamic partition key, permanent skew) would otherwise leak.
windmill_queue::asset_dispatch::reap_stale_join_slots, called from the
monitor's delete_expired_items loop, deletes a (workspace, subscriber,
partition) slot only when its MOST RECENT row is older than
JOIN_SLOT_TTL_SECS (60d) — per-slot, never per-row, so a legitimately
slow join is not corrupted mid-accumulation. Conservative default;
per-join configurable TTL via the annotation is a planned follow-up.
Test covers stale-reaped / fresh-kept / mixed-slot-kept.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* update

* feat: path-less native trigger markers + missing-trigger placeholder

* feat: pipeline // tag and // retry annotations + dispatch_event log

* fix: derive test-pane min from split-axis dimension (height in bottom layout)

* feat: show last run logs/result when a script node is selected

* fix: backfill asset rows from script.assets for pre-feature scripts

* feat: job-id link + dispatch popover above script log/result

* style: drop 'dispatched' label, keep just the check icon

* fix: drop tag picker from pipeline script editor (set via // tag annotation)

* Nicer UI

* refactor: move google ai proxy handling to windmill-ai (#9260)

* refactor: add ai proxy execution mode

* refactor: move google ai proxy handling

* refactor: share google ai request building

* fix: early return should consider failure_module result (#9241)

* fix(flows): flag noLogs jobs and lazily resolve them in log panel (#9099)

* fix(flows): flag noLogs jobs and lazily resolve them in log panel

* fix appending to flag

* fix: preserve WM_LOGS_SKIPPED sentinel on SSE/replay completion

pickMoreCompleteLogs resolved both sentinel and undefined to '', so the
SSE completion event (whose job field is fetched .without_logs()) would
clobber the sentinel placed by flagSkippedLogs. The module log panel
then saw '' instead of the sentinel, defeating the lazy-resolve path.

Also wire onLogsResolved on the OutputPickerInner inline LogViewer so a
lazy resolve writes back to flowStateStore.previewLogs, matching
ModulePreviewResultViewer and avoiding repeated fetches on remount.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(main): release 1.705.0 (#9229)

* chore(main): release 1.705.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* chore: add playwright mcp for frontend verification (#9269)

* feat: CLI datatable serve / psql (#9267)

* feat(cli): add datatable list and run commands

* feat(cli): render datatable query results as a table

* feat(cli): serve datatables as a postgres-wire endpoint

* feat(cli): add 'datatable psql' to launch psql against the proxy

* feat(cli): route datatable serve by client-supplied database name

* override database list + password option

* fix: support extended queries in datatable serve

* fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma

* refactor: extract raw_output envelope encoding into pg_raw_output module

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>

* oom_adj nit

* feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting (#9271)

* feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting

Allows operators to point `uv python install` at a private mirror of the
python-build-standalone releases. Configurable via the
`UV_PYTHON_INSTALL_MIRROR` env var or the `uv_python_install_mirror`
instance setting, with the env var as the boot fallback and the instance
setting taking precedence at reload.

Fixes WIN-1966

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: hoist uv_python_install_mirror binding above sandboxing branch

The non-sandboxed uv pip install branch referenced a binding that was
only declared inside the sandboxed branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: neutral placeholder for uv_python_install_mirror

The previous placeholder was the default public URL the setting is meant
to redirect away from. A neutral example mirror URL is clearer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(indexer): tell admins when ingress routes search to wrong pod (#9274)

* [ee] fix(indexer): tell admins when ingress routes search to wrong pod

When the IndexReader is absent on the pod handling a search request but
another pod is actively holding the indexer lock, the EE handler now
returns a tailored error pointing at the ingress/load-balancer
configuration instead of the generic "indexer not running" message.

The indexer status endpoint reads the DB lock so it reports "running"
from any pod, but search endpoints need the in-memory IndexReader that
only exists on the lock holder. In multi-replica deployments this looks
like the indexer is healthy but every search 404s.

Companion: windmill-labs/windmill-ee-private#TBD

Fixes WIN-1968.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817

This commit updates the EE repository reference after PR #586 was merged in windmill-ee-private.

Previous ee-repo-ref: 7dd43d1850813071cc18ba49ba090583e7321f4b

New ee-repo-ref: eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(cli): add `wmill init prompts` and custom override slot (#9266)

* feat(cli): add `wmill init prompts` and custom override slot

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): replace init prompts with refresh prompts + AGENTS.md/AGENTS.cli.md split

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): dedupe claude skills via @-includes and add prompts freshness check

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): drop migration-choice flags from `refresh prompts`

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(cli): add 'Running and previewing local changes' section to AGENTS.cli.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): write full skill content to .claude/, drop @-include wrapper

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): reconcile CLAUDE.md the same way as AGENTS.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review nits — argv parsing, lazy import, comment detection, error propagation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add yolo mode for ai chat tools (#9258)

* feat: add yolo mode for ai chat tools

* nit

* fix: align chat footer controls

* feat: add ai chat autonomy modes

* feat: add autonomy mode dropdown

* fix: highlight yolo autonomy icon

* fix: auto accept flow edits

* fix: hide unsupported autonomy modes

* fix: handle auto-accept flow editor races

* fix(debugger): add non-root user support to Dockerfile (#9277)

Mirrors the main Windmill Dockerfile pattern: creates a windmill user
(UID/GID 1000) and makes cache/work directories world-writable so the
image runs cleanly under Kubernetes securityContext.runAsNonRoot or
runAsUser: 1000 without permission errors on Bun, pip, or windmill
cache writes.

Fixes WIN-1969

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path (#9276)

* fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path

The AI proxy handler accepts an X-Resource-Path header to override the
configured workspace AI provider. When supplied, the handler loaded the
resource value from the resource table using the root DB pool with no
resources:read scope check, so any authenticated workspace user could
point X-Resource-Path at a restricted AI resource (e.g. one in a folder
they cannot read) and the proxy would use that resource's provider
credentials for the outbound AI request.

For user-supplied resource paths, now require resources:read:{path}
scope and fetch the resource through user_db.begin(&authed) so RLS
enforces the same folder/group boundary as the resource API. The RLS-
scoped $var: resolution stays in place as defense in depth. The
admin-configured workspace/instance ai_config path is unchanged.

Fixes WIN-1971

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(ai): regression test for X-Resource-Path RLS enforcement

Cover all four cases:
- non-admin pointing X-Resource-Path at a restricted resource is rejected
- non-admin pointing it at a resource they own still works
- admin can point it at any resource
- workspace-configured proxy flow (no X-Resource-Path) is unchanged

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add userdraft listing primitives (#9268)

* feat: add userdraft listing primitives

* fix: cancel stale userdraft discard writes

* docs: remove global ai userdraft plan

* feat(nsjail): optional disk-backed /tmp via instance setting (#9272)

* feat(nsjail): optional disk-backed /tmp via instance setting

* test(nsjail): unit-test tmp mount resolver and narrow visibility

* refactor(nsjail): switch tmp backing to select + conditional UI

* ui(nsjail): make tmpfs the visible default in /tmp backing select

* fix(nsjail): refuse preexisting jail_tmp to block symlink escape

* fix(nsjail): allow jail_tmp reuse on sequential nsjail calls

Codex flagged that python/ruby/rust executors invoke nsjail twice per
job_dir (install then run). The previous resolver treated any preexisting
jail_tmp as hostile and silently fell back to tmpfs on the second call,
so disk-backed mode never reached the main script run for those langs.

Use symlink_metadata().is_dir() to distinguish a real directory left by
an earlier call in the same job_dir (safe to reuse) from a symlink or
other entity (still refused, as the codebase-tar escape requires).

Also loosen the frontend visibility predicate: only hide nsjail settings
when job_isolation is explicitly 'none' or 'unshare', so deployments
that enable nsjail via DISABLE_NSJAIL=false with no DB setting can
still see the controls.

* chore(main): release 1.706.0 (#9270)

* chore(main): release 1.706.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix(nsjail): gate unix-symlink test behind cfg(unix) for Windows build (#9280)

The disk_backed_refuses_preexisting_symlink_at_jail_tmp test calls
std::os::unix::fs::symlink directly, which doesn't exist on Windows
targets. Without a cfg gate, `cargo check --tests` fails on Windows
with E0433. Other symlink call sites in this crate (php_executor,
bun_executor, rust_executor, etc.) already follow this pattern.

Fixes WIN-1972

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Reduce slim image vulnerability surface (#9279)

* Reduce slim image vulnerability surface

* chore(docker): drop apt-get upgrade -y from slim images

apt-get upgrade hurts build reproducibility (same Dockerfile + same
commit at different times produces divergent images) and trips hadolint
DL3005. The freshness it buys is dominated by simply rebuilding against
the periodically-refreshed debian:bookworm-slim base image.

The --no-install-recommends and apt-list cleanup wins are kept.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* fix(git-sync): bump to hub/28234 with stateless gpg.program wrapper (WIN-1974) (#9282)

* fix(git-sync): revert LATEST_GIT_SYNC_SCRIPT_PATH to hub/28230 to restore GPG-signed deploys (WIN-1974)

hub/28231 (PR #9230) is the "thin" script that hands the actual `git commit`
to the CLI's hidden `sync git-deploy`. The hub script still does the GPG
setup (import key into a fresh GNUPGHOME, dummy `gpg -bsau` to warm the
agent passphrase cache, then `git config user.signingkey` + `commit.gpgsign`
locally), but the commit no longer runs in the same `git_push` flow — it
runs minutes later inside the CLI after workspace API resolution, zip pull,
file extraction, and lockfile autofill. By the time the spawned `git commit`
asks gpg-agent for the cached passphrase, the cache state is no longer
reliable (or the spawned `gpg` ends up talking to a fresh agent), so signing
fails non-interactively with `gpg failed to sign the data`.

hub/28230 is hub/28217's in-script logic rebuilt with windmill-cli@1.703.3:
the GPG setup and the in-script `sh_run("git commit ...")` happen back-to-back
in `git_push`, so the cache is always fresh. It preserves wm_deploy / fork
branch behavior, the EE deployment-callback `main()` signature is unchanged,
and the only min-version check in EE (`is_script_meets_min_version(28103)`)
is comfortably below 28230 — so this revert is safe.

Forward fix (separate PR): publish a new thin script that, alongside the
existing GPG setup, writes a `gpg.program` wrapper using `--pinentry-mode
loopback --passphrase-file` so signing is independent of the agent's cache
state. Re-bump past 28231 then.

Fixes WIN-1974

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(git-sync): check in source-of-truth for the next hub script (gpg.program wrapper)

This is the script that will be published to hub.windmill.dev once verified
on a customer GPG-signed deploy. It replaces hub/28231's agent-cache
pre-warm (`gpg -bsau` with --passphrase) with a stateless gpg.program
wrapper + chmod-600 passphrase file. Every git-invoked gpg call goes
through the wrapper, which always uses --pinentry-mode loopback (and
--passphrase-file when a passphrase exists). Signing no longer depends on
gpg-agent having a cached passphrase by the time the CLI's `git commit`
runs — which closes WIN-1974.

Not wired in yet: LATEST_GIT_SYNC_SCRIPT_PATH stays on hub/28230 until this
script is uploaded and the new hub id is known. This file is checked in so
the diff is reviewable, future bumps have a source of truth, and a CLI
regression test can `cat` it for fixture parity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(frontend): skip format/pattern validation for $var/$res/$jsonvar references in ArgInput

A resource field with a `pattern` constraint (e.g. the gpg_key.private_key
field, whose pattern enforces a `-----BEGIN PGP PRIVATE KEY BLOCK-----`
prefix) rejects values like `$var:u/me/gpg-private-key` with an "invalid
format" error in the resource editor — even though `$var:`/`$res:`/`$jsonvar:`
are placeholders the backend resolves at runtime, not the actual string
that needs to match the regex.

Bail out of all format/pattern checks (email, ipv4, ipv6, uuid, custom
pattern) when the value is one of these references. Required/numeric
bounds/array checks still apply since they're shape-level, not regex.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(git-sync): bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28234 (gpg.program-wrapper fix)

hub/28234 is the forward fix for WIN-1974: replaces hub/28231's agent-cache
pre-warm (which became stale by the time the CLI's `git commit` ran) with
a stateless `gpg.program` wrapper that uses `--pinentry-mode loopback`
(and `--passphrase-file` when a passphrase exists) on every gpg invocation.
Bundled CLI is windmill-cli@1.705.0.

Verified via reproducer at /tmp/git-sync-diff/test-gpg-fix.sh: deliberately
killing gpg-agent between GPG setup and `git commit` reproduces the
customer's `gpg failed to sign the data` error verbatim under the old
flow, and the wrapper signs through it. Holds for passphrase-protected
keys, split-subkey [C]+[S] layouts, and unprotected keys.

Drops the local source-of-truth copy (`hub-scripts/`) — hub is canonical
now that 28234 is published.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(git-sync): drop verbose comment above LATEST_GIT_SYNC_SCRIPT_PATH

The git history (this PR) carries the why; the constant name + value carry
the what.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): wmill sync git-deploy stops committing; caller owns commit+push (#9284)

Single contract for the deployment-callback path: the CLI does branch
checkout + pull, the caller (hub script in production, test in test)
does git add + commit + push. This restores the WIN-1974 invariant —
GPG setup and `git commit` run back-to-back in the same process, so
the agent's pre-warmed passphrase cache is still warm at sign time —
without needing a `--skip-commit` flag for the hub case and a default
"also-commit" for everything else. Same behavior in every call site.

Changes:
  - sync.ts: drop the gitSyncDeployPush call from pull()'s deploy path
    (both the onlyCreateBranch fast-return and the post-pull commit).
    `gitSyncDeployPush` stays exported for any caller that wants the
    same commit/push semantics — just not invoked by the CLI subcommand.
  - gitsync_promotion.test.ts: e2e test now does its own git add +
    commit + push after `wmill sync git-deploy`, mirroring what the
    hub script does in production. Same regression coverage
    (wm_deploy branch created in Case A, main untouched; main updated
    in Case B, no new wm_deploy).

CLI typecheck unchanged (two pre-existing TarAsZip errors at lines
2578/3307, present before this PR). All 743 unit tests still pass.

The accompanying hub script (option-C — CLI for branch+pull, script
for commit+push) lives at /tmp/git-sync-diff/sync-script-to-git-repo-windmill.option-C.ts.
Once published, a follow-up bumps LATEST_GIT_SYNC_SCRIPT_PATH to its id.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* bump git sync to 28236

* fix: fork compare visibility for non-admins and stale-token superadmins (#9283)

* fix: use fork-scoped authed for fork visibility in compare_workspaces

* test: add EE end-to-end repro for fork rename visibility

* chore: restore concurrency_locks sqlx cache lost in cleanup

* test: add regression for stale-superadmin-token fork visibility bug

* chore: update sqlx cache for new test queries

* chore(main): release 1.706.1 (#9281)

* chore(main): release 1.706.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: add wmill job rerun subcommand (#9275)

* feat: add wmill job rerun subcommand

* feat: add wmill job restart subcommand for flow restart-at-step

* chore(system_prompts): point plugin skills sync at plugins/windmill/ (#9287)

* chore(system_prompts): point plugin skills sync at plugins/windmill/

The plugin checkout's plugin folder is being renamed from
`plugins/windmill-code-plugin/` to `plugins/windmill/` to shorten the
slash-command namespace and align with the matching Cursor plugin
layout.

Paired with windmill-labs/windmill-claude-plugin#8. That PR must merge
first so the next sync run finds the new folder.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(system_prompts): update plugin-dir example to plugins/windmill

Co-authored-by: centdix <centdix@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix(cli): wmill sync pull updates wmill-lock.yaml for raw apps (#9289)

* fix: flow recording teardown crash + rename package to @windmill-labs/components (#9288)

* fix: guard against null recording during FlowRecordingReplay teardown

Navigating away from a flow recording inside a workspace file-tree view
threw `TypeError: Cannot read properties of null (reading 'flow')` from
FlowGraphViewer once during the teardown tick.

Svelte 5 compiles child component props as live getters that close over
`$$props.recording.flow`. When `recording` flips to null on the parent's
navigation, an outer `{#if !recording?.flow}` doesn't stop those getters
from firing one more time as derived effects re-evaluate before the
unmount lands — so the getter dereferences null and throws.

Fix at the two layers where the deref actually happens:

- FlowRecordingReplay: use `recording?.flow` at the binding sites
  (FlowViewer + graph-snippet FlowGraphViewer) so the compiler emits an
  optional-chained getter, and guard the snippet branch with
  `{:else if recording?.flow}` so it doesn't mount when there's nothing
  to show.
- FlowGraphViewer: finish the optional chaining the rest of the file
  already used everywhere else (`flow?.value?.skip_expr`,
  `flow?.value?.cache_ttl`, `flow?.schema`). When the upstream
  binding returns undefined during teardown, the graph degrades to an
  empty frame instead of crashing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rename package to @windmill-labs/components

- frontend/package.json: rename `windmill-components` → `@windmill-labs/components`
- frontend/publish.sh: drop the in-place sed rename dance; the checked-in name now matches what's published, so `npm run package && npm publish` is enough
- frontend/package-lock.json, system_prompts/auto-generated/prompts.d.ts: regenerated by `npm run package` under the new name

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* default script name

* save logic

* Keyboard nav

* finish keynav

* nits

* CI fix

* nit stop propagation

* Merge branch 'main' into feat/asset-graph-view

* commit

* update

* fix: cropped save button on small screens

* progress

* managed scheduled removed

* all

* progress

* feat: add data upload pipeline trigger with auto S3 picker

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: avoid pane editor remount flicker when deploying a pipeline draft

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: show only the edited script's I/O in the asset graph, not the saved version's

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: derive script asset rows server-side at deploy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: shared fixture corpus keeps annotation parsers in parity

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: dev-run draft pipeline chains, live badges, deploy drift warning

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: ungate cascade producers, squash pipeline migrations

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: drop committed cli-sync fixtures and stray screenshots

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: show skip-asset-dispatch flag as badge instead of args row

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: pipeline view mode default with activity feed, drafts overlay chip

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: treat DROP TABLE as table-level write in sql asset parser

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: wmill datatable create + actionable sql extension error

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: ephemeral data-pipelines demo sync repo zip for handoff

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: wmill pipeline list/show renders the asset DAG in the terminal

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* nits

* nits

* nits

* nits

* fix: defer draft persist-back past the batch so discard sticks first click

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: band-reserving tidy-tree asset graph layout with join breakpoints

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: route skip-layer and long graph edges around occupied columns

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: seed s3 template outputs with canonical leading-slash paths

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* all

* feat: bundle data-pipeline drafts into the DB-backed user draft system

Pipeline drafts were browser-only (localStorage `pipeline-<folder>`), so they
didn't sync across devices, weren't server-visible, and never showed in the
drafts list. Store them instead as one per-user `draft` row of a new
`data_pipeline` kind, keyed at the folder (`f/<folder>/data_pipeline`), holding
the same `{ drafts, activeDraftPath }` bundle.

Stage 1 — backend kind: add `data_pipeline` to DRAFT_KIND (migration) and
`UserDraftItemKind` (deployed_table=None, private). The list/update handlers
and folder-path access check already cover a backing-table-less kind.

Stage 2 — sync: add `GET /drafts/get_own/{kind}/{path}` so an editor with no
deployed-overlay GET can load its own draft. The pipeline page now hydrates
from the DB on mount (one-time localStorage import for in-flight drafts) and
persists via UserDraftDbSyncer (debounce + optimistic-concurrency), keeping a
localStorage crash mirror.

Stage 3 — surface: the drafts review page renders the bundle as a "pipeline"
row that opens `/pipeline/<folder>` (open-only; excluded from bulk deploy).

Verified end-to-end in-browser: DB-seeded draft hydrates to "Edit (1)", edits
persist back, and the row shows with Open pipeline / Discard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: pipeline Activity panel grouping, run↔graph highlight, deploy-conflict handling

Activity panel (view mode):
- Group cascade runs by the connected component of the asset-dispatch graph
  (new GET /jobs/asset_dispatch_edges over the dispatch_event table, incl.
  join_pending inputs), headed by the earliest originating run + its trigger,
  with a "+N" chip for joins fed by multiple triggers.
- Success/failure count histogram with drag-to-filter brushing, an always-on
  time axis + per-bar tooltips, a Reset, and Last hour/24h/48h/7/30/90d ranges.
- Node run-count/status badges now derive from the same merged historic+live
  events the panel shows (previously session-only).

Run ↔ graph highlight:
- Hovering a run row (or a group header → the whole cascade) rings the
  node(s), animates their incident edges, and borders the adjacent assets in
  the edge hue (blue write / gray read); expanding a run pins a soft-blue ring.
- Switching edit→view re-surfaces the Activity feed.

Deploy:
- Live-content autosave for the open pipeline draft + an autosave indicator.
- Re-saving a script now chains off the hash just created instead of a stale
  parent_hash (fixes the "lineage must be linear" error on a second save), and
  a genuine concurrent deploy opens a keep-mine / view-latest conflict modal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: pipeline editor badge requires asset-parse, not just main-function parse

A pipeline script's asset lineage is load-bearing — a deploy that can't parse
assets silently records no edges. The editor "parsable" dot only reflected
inferArgs (the main function), so a body the asset parser rejects (e.g. a
trailing `/////` in DuckDB) still showed green and deployed with empty lineage.

ScriptEditor gains `requireValidAssets` (set by the pipeline pane); when on, the
EditorBar badge is green only if BOTH the main function and inferAssets parse,
with the tooltip distinguishing "Main function not parsable" / "Assets not
parsable" / "Parsable".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: route asset-graph edges around nodes that sit in their path

Edges could draw straight through an unrelated node (a join fan-out or long
cross-component edge), making it ambiguous whether that node shared the input.
AssetGraphEdge only saw its own endpoints, so it could only detour the
near-vertical same-column skip case.

The canvas now (once per layout, O(edges × nodes) — no per-frame cost) samples
each edge's straight run against every non-incident node center and, on a
crossing, passes a clear gutter lane to the edge via `data.detourX`;
AssetGraphEdge routes the rounded-orthogonal detour through it. Verified: 0
edge↔node box crossings on the orders pipeline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: deploy pipeline drafts with freshly-inferred assets, not a stale snapshot

"Save all" spread `...draft.script` into createScript, which carries a `assets`
snapshot that isn't refreshed when the body is edited. So a renamed/removed
output (e.g. an old `CREATE TABLE exciting_en32z9` later changed to
`exciting_880909`) was re-deployed as a phantom write edge and lingered as an
orphan asset on the graph — shown with no producer, and shifting position on
click as the graph re-derived.

saveDraft now re-runs inferAssets on the current body and passes the result as
`assets`, overriding the snapshot — mirroring the per-pane save. The backend
clears+reinserts from the sent set, so a re-deploy drops the stale rows.
Verified: deploying with the fresh asset set removes the orphan from the graph.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: collect upstream reads from CTAS and CREATE VIEW in SQL asset parser

`CREATE TABLE x AS SELECT … FROM y` (and `CREATE VIEW`) recorded only the
write to x — the source read of y was silently dropped. Table-level reads are
gathered in the `Statement::Query` arm via handle_table_with_joins; the generic
table-factor visitor only picks up read-functions and string literals, not
plain `FROM <table>` references. The AS-query of a CTAS isn't a
`Statement::Query`, so its FROM tables were never walked. On the pipeline
canvas this meant a `datatable://…` upstream consumed by a CTAS step showed no
read node/edge — the step looked like it produced its output from nothing.

Factor the Query arm's read collection into handle_query_reads and call it from
the CreateTable (when it has an AS-query) and CreateView arms, balancing the
cte_name_stack push in post_visit_statement. Updated the drop_then_create test
(which had pinned the old drop-the-read behavior) and added CTAS + CREATE VIEW
read coverage. Verified against the rebuilt asset wasm: the live editor now
infers the read.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* update

* updates

* refactor: dedup asset-graph code, squash migrations, drop artifacts

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf: gate asset dispatch on a cached per-workspace producer set

Cache the producer-path→writes map per workspace and invalidate it from the asset-clear paths via the notify_event polling system, so a top-level script/preview completion that isn't an asset producer costs an in-memory lookup instead of a per-completion query.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove dead unquote fn that failed backend check under -D warnings

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: green the frontend check (pin published wasm-asset, fix type errors)

Pin windmill-parser-wasm-asset to the published 1.728.1 (was a file: link to a gitignored, CI-unbuilt pkg-asset). Exclude test files from svelte-check (the parity test reads a backend fixture via node:fs, which the browser app tsconfig has no @types/node for; vitest still runs them). Fix pre-existing branch type errors: drop the unsupported 2nd getScriptByPath arg, cast script.schema to Schema for inferArgs, coerce has_preprocessor to a definite boolean, and wrap the cancelJob handler so it isn't possibly-undefined.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: move pipeline partition resolution to ee-private (free-CE)

Partition resolution becomes a private module (partition_ee in windmill-ee-private, hidden from the public repo) with an OSS no-op fallback (partition_oss); call sites resolve via the aliased windmill_common::partition. Not enterprise-gated — free to run in CE. Bumps ee-repo-ref to the ee branch carrying partition_ee. Verified building in default, private, and private,enterprise (offline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: move asset-cascade join/debounce/retry to ee-private (free-CE)

Join barrier, debounce, and retry become the private windmill_queue::cascade module (cascade_ee in windmill-ee-private); OSS gets cascade_oss no-op fallbacks (plain OR fan-out). Core cascade stays public. Bumps ee-repo-ref. Verified default/private/private,enterprise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: skeleton enterprise pipeline freshness + backfill (TODO, ee-private)

Gated windmill_common::pipeline_advanced (private; pipeline_advanced_ee) with OSS fallback; entry points return a clear not-implemented error. Deploy surfaces a TODO when a script declares // freshness. Bumps ee-repo-ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: repair asset_trigger_dispatch test after cascade carve-out + cache its queries

Stage-2 moved reap_stale_join_slots to windmill_queue::cascade; update the integration test's import. Also commit the test's sqlx query cache (was never prepared with --tests, so SQLX_OFFLINE cargo test failed pre-existing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: invalidate producer-cache in asset dispatch tests (mirror deploy)

The tests seed asset rows directly and run no notify poller, so the per-workspace producer cache went stale across tests → 0 dispatched. Clear it at the seed point, as a deploy would via notify_event. All 8 asset_trigger_dispatch tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ba677ea142011462ad4dfe77e8375a6dd274cdef

This commit updates the EE repository reference after PR #619 was merged in windmill-ee-private.

Previous ee-repo-ref: 925c350cff55d3ea738d9e2e4098d9ce4bdda418

New ee-repo-ref: ba677ea142011462ad4dfe77e8375a6dd274cdef

Automated by sync-ee-ref workflow.

* test: disable producer cache in asset dispatch tests (isolated-DB safe)

The .remove(WS) approach still raced: #[sqlx::test] gives each test its own DB but they share one workspace id, so the WS-keyed process-global cache clobbered across DBs under concurrent threads. Add an ASSET_PRODUCER_CACHE_DISABLED test hook and set it in the tests so every dispatch reads its own DB. 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: replace asset-cascade depth cap with cycle detection

The hardcoded MAX_CHAIN_DEPTH=5 truncated legitimate deep pipelines (silently — the check returned before event logging). Replace it with per-edge cycle detection: carry the producer lineage in trigger.chain and skip only a subscriber already in the chain, recording a visible cycle_detected dispatch_event. Acyclic pipelines of any depth now cascade fully; a high MAX_CHAIN_LEN backstop guards against runaway. Tests + UI label updated; 8/8 pass at --test-threads=10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: update dispatch_event reason examples (depth_cap → cycle_detected)

Comment-only; the migration is idempotent and already in the potentially_stale self-heal list, so the checksum change re-applies cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: park cascade retry (P1 dead-end) + clear stale script_triggers on rename

Two deploy-path fixes:
- Retry is parked: a retried subscriber is wrapped in a SingleStepFlow, whose run is a flow step and ineligible for asset dispatch, so it would silently dead-end the cascade (P1). Stop persisting retry to script_trigger and warn at deploy; TODO(pipeline-retry) to re-enable once dispatch handles flow-wrapped producers. (Dispatch plumbing kept + still tested via direct seeding.)
- Rename leaves stale script_trigger rows: clear was keyed on ns.path only, so old-path '// on' edges lingered and could trigger a script later recreated at that path. Also clear the old path on rename (assets already handled via the parent-hash clear).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Arnaud <31803803+Araden14@users.noreply.github.com>
Co-authored-by: Diego Imbert <diego@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Aldrin Jenson <aldrinjenson@gmail.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2026-06-18 18:09:02 +02:00
Ruben Fiszel 33bf01b627 fix(python): preserve strings containing Infinity/NaN in result JSON (#9149)
* fix(python): preserve strings containing Infinity/NaN in result JSON

* test(python): add sanity checks for Infinity/NaN in results

* refactor(python): use string-aware regex callback for single-pass cleanup

* refactor(python): compact regex + handle backslash-escape parity

* perf(python): short-circuit cleanup when no Infinity/NaN/NUL in result
2026-05-13 15:49:36 +00:00
Ruben Fiszel 45d959a49e nit tests 2026-05-01 20:52:47 +00:00
Ruben Fiszel 0f26418f4a fix cargo checks 2026-05-01 18:24:45 +00:00
Ruben Fiszel 0c22f52b46 feat: support assigning a worker tag to app inline scripts (#9002)
* feat: support assigning a worker tag to app/raw-app inline scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: omit empty tag field from inline script raw_code payload

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: shrink tag popover width

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-05-01 17:21:17 +00:00
Ruben Fiszel abcd920964 test: isolate WAC v2 python test from stack overflow (#8979)
* test: isolate WAC v2 python test from test-thread stack overflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: bump RUST_MIN_STACK to 4MB for backend tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-29 19:50:17 +00:00
Ruben Fiszel 581658d881 fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor (#8951)
* fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor

Three workflow-as-code bug fixes:

- #8945: Python WAC template with `@workflow async def main(...)` was not
  detected as `auto_kind = "wac"`. The detection only ran when no `main`
  function was found. Hoist the heuristic so it runs whether or not `main`
  is the entrypoint.

- #8946: `scripts/list?kinds=script` filtered out WAC scripts because they
  set `auto_kind = 'wac'` and the SQL hid everything that wasn't NULL.
  Allow both NULL and 'wac' (still excluding 'lib' library scripts).

- #8947: Preprocessor functions defined alongside a WAC workflow were
  ignored. Inject the preprocessor invocation into the Python WAC wrapper
  so it runs before the workflow on the first iteration, then plumb the
  preprocessed args through `handle_wac_v2_output` so inline child
  re-runs see the post-preprocessor args via `checkpoint.input_args`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(wac): integration tests for #8946 (scripts/list) and #8947 (preprocessor)

- test_scripts_list_includes_wac: hit GET /scripts/list?kinds=script and
  assert WAC scripts are in the response (would have failed pre-#8946 fix
  because of the auto_kind IS NULL filter).
- test_python_wac_v2_with_preprocessor: deploy a Python WAC script with a
  preprocessor, run with raw event args, assert the workflow saw the
  preprocessed shape and v2_job.args/preprocessed were updated.
- New wac_preprocessor.sql fixture with auto_kind = 'wac' set explicitly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(wac): address PR review feedback

Five review fixes:

- python_executor.rs: WAC preprocessor now runs inside the wrapper's
  `try:` block so failures route through the same `result.json` error
  serializer as workflow failures. Switched async-coroutine handling
  from deprecated `asyncio.get_event_loop().run_until_complete(...)` to
  `asyncio.run(...)` (the recommended primitive on 3.10+).

- bun_executor.rs: when copying preprocessed args into
  `checkpoint.input_args`, surface JSON parse failures via `?` instead
  of silently coercing to `Value::Null` (which would persist a corrupted
  arg into every child re-run). Also collapsed the redundant double
  iteration into a single pass.

- windmill-api-scripts/scripts.rs: switched the runnable-script filter
  from an allow-list (`auto_kind IS NULL OR = 'wac'`) to a deny-list
  (`<> 'lib'`), so future `auto_kind` values aren't silently filtered
  from triggers/dropdowns.

- windmill-parser-py: aligned the parser's WAC heuristic with the
  runtime detector `is_wac_v2_py` — `@task` is now optional, matching
  the runtime which says workflows that only use inline `step()` are
  still WAC. Added a regression test `test_parse_python_wac_step_only`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-27 13:38:49 +00:00
Ruben Fiszel b3ef4bc26c perf: add inline-persist fast path for WAC v2 step() (#8807) 2026-04-13 16:49:53 +00:00
hugocasa bffa61e33f fix: dedicated worker dispatch, cross-workspace deps, UI improvements (#8689)
* feat: restore bun as default runtime for dedicated workers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add context comment for bun dedicated worker nodejs migration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker dispatch for flows + add E2E tests

- Add workspace_id prefix to dedicated worker map lookup keys
- Update ee-repo-ref for dedicated worker path handling fix
- Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers
- Add 6 E2E tests for dedicated workers:
  - test_dedicated_flow_rawscript (regression for "Script not found" bug)
  - test_dedicated_flow_workspace_script
  - test_dedicated_flow_multiple_steps
  - test_dedicated_standalone_script
  - test_dedicated_runner_group
  - test_dedicated_flow_runners
- Add dedicated_flows.sql fixture with scripts, flows, and worker config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always run dependency job for dedicated worker scripts

When a script with dedicated_worker=true is deployed with a pre-computed
lock (e.g. via wmill sync push), no dependency job was created, so the
dedicated worker never detected the update and kept running the old version.

Now dedicated worker scripts always generate a dependency job regardless
of whether a lock is provided. The dependency job runs on the dedicated
worker and triggers a restart so it picks up the new script version.

Fixes #8638

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races

Dedicated worker tests need non-default worker tags in the global
WORKER_CONFIG. When run in parallel (CI uses --test-threads=10),
multiple tests clobber each other's config. Use #[serial] to ensure
dedicated worker tests run sequentially.

Also load worker config from DB via load_worker_config() instead of
manually setting WORKER_CONFIG fields, ensuring consistency with the
monitor's reload path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests

Fix script_path shadowing in bun_executor nodejs branch where the wrapper
file path was passed to handle_dedicated_process instead of the logical
path, causing "Script not found" for all //nodejs dedicated workers.

Add E2E tests for dedicated flows in all supported languages:
- test_dedicated_flow_deno
- test_dedicated_flow_python
- test_dedicated_flow_bunnative (V8 PrewarmedIsolate path)
- test_dedicated_flow_bun_nodejs (//nodejs annotation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify dedicated worker dispatch + add serialization and E2E tests

- Unified lookup: always use {workspace}:{runnable_path} for dedicated
  worker dispatch, replacing the flow_step_id iteration approach
- Added serialization_semaphore parameter to executor start_worker fns
- Added E2E tests: cross-workspace isolation, conflicting flow step IDs,
  preprocessor on dedicated worker
- Added workspace field to RunJob for cross-workspace test support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: cross-workspace workspace dependencies on workers page

Add two new instance-level endpoints to the configs router:
- GET /configs/list_all_workspace_dependencies
- GET /configs/list_all_dedicated_with_deps

Both require devops role and return data across all workspaces,
enabling the workers page to show a consistent view of which
workspace dependencies exist regardless of which workspace the
user is browsing.

Update DedicatedWorkersSelector to use the new cross-workspace
endpoints with fallback to per-workspace calls for non-devops users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to include dedicated worker lookup simplification

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update ee-repo-ref.txt with new reference

* sqlx

* fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings

- Remove serialization_semaphore from executor start_worker signatures
- Remove serialization test and fixtures
- Fix DedicatedWorkersSelector to preserve tags from other workspaces
  when toggling in the picker
- Track workspace deps per-workspace for conflict detection
- Show warning when dep exists in another workspace but not the script's
- Group runner groups per-workspace to prevent cross-workspace merging
- Add workspace to dep badge link URL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify exec protocol — execd: for single-script, exec: for runner groups

Add execd:/execd_preprocess: commands to bun/deno/python wrappers for
single-script dedicated workers (no path needed). Runner groups keep
exec:/exec_preprocess: with path for multi-script disambiguation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add unit tests for execd:/exec: wrapper protocol

Verify generate_multi_script_wrapper produces both execd: (single-script)
and exec: (runner group) protocol handlers, including preprocessor variants.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* fix: remove beta badge from squash loop, keep tooltip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update protocol tests to use execd: for single-script wrappers

Deno and bun single-script protocol tests now send execd:{args} instead
of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script
(runner group) tests continue to use exec:{path}:{args}.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused TEST_SCRIPT_PATH in deno protocol tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback — down migration, push_as workspace, UI improvements

- Use regexp_replace in down migration for positional accuracy
- Fix push_as() to use self.workspace_id instead of hardcoded value
- Remove per-workspace API fallbacks, use cross-workspace endpoints only
- Skip devops-only API calls when user is not devops (disabled prop)
- Fix duplicate key error for cross-workspace runner groups
- Add workspace to RunnerGroup for unique keying
- Reuse tagRow snippet for standalone items with expand/collapse
- Fix picker alignment: remove empty column for non-expandable items

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: comprehensive dedicated worker test coverage, fix Python execd_preprocess

- Add Python execd_preprocess: handler (was missing for single-script dedicated workers)
- Add 10 E2E tests: flow+standalone conflict, mixed lang fallback, unsupported lang
  flow runners, python runner group, bun/python/deno/bunnative preprocessors,
  runner group preprocessors, branchone flow
- Add 4 Python unit tests for execd:/execd_preprocess: protocol
- Update EE ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback — migration escaping, deno try/catch, loadRunnables guard

- Down migration: use E'...' so \n matches actual newlines
- Up migration: anchor regex with ^ to avoid mid-content matches
- Deno execd_preprocess: move JSON.parse inside try/catch
- DedicatedWorkersSelector: skip devops-only API calls when disabled

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add dedicated worker relative import tests for bun and python

Verifies that build_loader's CURRENT_PATH correctly resolves workspace-
relative imports when running on a dedicated worker subprocess.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker dispatch for nested flow structures (branches/loops)

- Add extract_flow_root() to strip nesting segments from runnable_path
- Dispatch uses flow_root/flow_step_id for nested paths, runnable_path
  for flat paths — deterministic, O(1)
- Fix assert_ran_on_dedicated_worker to BFS all descendants
- Fix python mode labels (python vs python3 for runner groups)
- Add tests: simple forloop, multi-step forloop, whileloop, branchall,
  nested branch-in-loop, mixed lang fallback, unsupported lang runners

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: fix ee-repo-ref SHA

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide picker and skip API calls for read-only users, hide empty runner badge

- Hide "Add more scripts/flows" section when disabled (read-only)
- Skip per-runnable API calls (getScriptByPath, getFlowByPath) for
  disabled users — just show path info
- Hide "0 runners" badge on flows with no eligible steps

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 9422b189762ae27edfc346541ae668a4ad728325

This commit updates the EE repository reference after PR #503 was merged in windmill-ee-private.

Previous ee-repo-ref: 4c6ba214bfc23fff05d1dc3200ac59e650af3f4f

New ee-repo-ref: 9422b189762ae27edfc346541ae668a4ad728325

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-03 17:50:07 +00:00
hugocasa c28314f424 feat: runner groups for shared-process multi-script dedicated workers (#8434)
* feat: add runner groups for shared-process multi-script dedicated workers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: unify dedicated worker and runner group wrappers into single multi-script wrapper

Replace per-language single-script wrappers with the unified load/exec/exec_preprocess/end
protocol. Each start_worker() now writes scripts to scripts/<safe_name>/ and uses
generate_multi_script_wrapper(). handle_dedicated_process() sends load: on start and
exec: per job instead of raw JSON args.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: merge runner groups into dedicated workers with inline arg metadata

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to match EE branch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate EE-only functions behind cfg(feature = "private") to fix OSS dead_code errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: auto-detect runner groups from workspace dependency annotations

- New endpoint GET /scripts/list_dedicated_with_deps: returns dedicated
  scripts with parsed workspace dependency names from content annotations
- Frontend: show dep badges in DedicatedWorkersSelector with links to
  workspace settings, warn when referenced dep doesn't exist, group
  scripts sharing deps into "Shared runner" sections
- Remove manual "Runner groups" tab and RunnerGroupSelector component
- Remove runner_groups from WorkerConfigOpt/WorkerConfig (auto-detected)
- Fix Node.js single dedicated workers: transpile main.ts -> main.js via
  Bun.build so the multi-script wrapper's dynamic import() works under Node
- Add package.json with type:module in scripts dir to silence Node warning

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: unify dedicated worker wrappers with baked-in codegen and routing

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add e2e tests for multi-script dedicated worker routing (bun, deno, python)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dead generate_dedicated_worker_wrapper function

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add dependency installation to runner groups + make dep functions pub(crate)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent bun loader from intercepting absolute paths within cwd

When a plugin's onResolve returns an absolute path, Bun re-invokes
the resolver with that path. The loader was then routing it through
the remote URL resolver, breaking runner group script imports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use _wm_ prefix for runner group scripts to avoid bun loader interception

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract DENO_UNSTABLE_ARGS constant to avoid repeating flags

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: gate private-only exports behind cfg(feature = "private") for OSS build

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move format strings before handle_dedicated_process to fix lifetime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate sqlx offline cache

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix sqlx

* fix: skip empty lines in deno e2e tests (double newline from console.log + '\n')

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use dict() instead of {{}} in python wrapper to avoid set literal

{{{{}}}} in format!() produces {{}} which Python interprets as an
empty set, not a dict. Use dict() which is unambiguous.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: remove deno from runner groups and associated tests

Deno resolves dependencies at runtime via URLs/import maps, so there's
no shared node_modules/pip install to benefit from runner groups.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: revert deno wrapper to inline old-style with exec: protocol

Since deno doesn't support runner groups, the unified multi-script
wrapper is unnecessary. Reverted to the old inline wrapper from main
but adapted to use the exec:<path>:<args> protocol.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: extract deno wrapper into reusable function and add e2e tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use codebase presence (not nodejs annotation) to determine wrapper import extension

On main, codebase scripts import ./main.js (pre-bundled JS).
The wrapper_ext was incorrectly based on annotation.nodejs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: improve dedicated workers UI - combine lists, better badges, tooltips

- Merge shared runners section with selected tags into one unified list
- Move language tag to right side of selector for alignment
- Change dep badge color from dark-gray to indigo
- Add tooltip on yellow warning badge explaining missing workspace dep

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: group shared runners visually in dedicated workers list

- Runner groups shown with a header (Shared runner · language · dep badge)
- Scripts in the same group nested under the header
- Standalone scripts/flows shown after groups
- Used Svelte snippet for reusable tag row rendering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve visual separation between shared runner groups and standalone items

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: give standalone runners same header style as shared runners

- Each standalone script/flow gets its own header row with bg-surface-secondary
- Header shows "Dedicated runner" / "Flow runner" label, dep link, language badge
- Shared runner header: swapped language and dep badge positions
- Dep shown as inline link instead of badge in headers for cleaner look

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: inline standalone runner path in header, language badge on right edge, no max height

- Standalone items: path shown directly in header row (no sub-row)
- Language badge placed after flex-1 spacer (right-aligned)
- Removed max-h-64 overflow constraint from the list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: consistent badges across runner list - dep+language on right, depBadge snippet

- Shared runner scripts: show (workspace) and language badge on right
- Standalone items: dep badges and language badge on right (after flex-1)
- Shared runner header: dep badge and language badge on right
- Extract depBadge snippet to deduplicate dep badge rendering
- Picker selector also uses depBadge snippet

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show language badge on standalone items, hide from shared runner sub-items

- Fetch script language from API when not available from workspace deps
- Hide dep+language badges from tagRow when script is inside a runner group
  (already shown in the group header)
- Standalone items now always show language badge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: differentiate badge colors - gray for language, indigo for workspace deps

Matches codebase convention: gray for metadata (like script hashes),
indigo for linkable features/entities.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use transparent (bordered) badge for language - visible on all backgrounds

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use gray badge for language everywhere

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert skills.ts and AI files, add _wm_ exclusion to Windows loader

- Revert cli/src/guidance/skills.ts to main (not our change)
- Revert AI provider formatting changes (not our change)
- Add _wm_ prefix exclusion to loader.bun.windows.js filterResolve

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update ee-repo-ref and regenerate system prompts after merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: use DISTINCT ON in list_dedicated_with_deps to dedup at DB level

Avoids fetching all script versions and deduplicating in Rust.
Addresses PR review feedback.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use sqlx query! macro for list_dedicated_with_deps and regenerate cache

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker review fixes and test coverage

- Fix Python relative imports in dedicated workers (write loader.py, add
  import loader to wrapper when needed)
- Move Python colon parsing inside try/except to prevent crashes on
  malformed stdin
- Add indexOf guard in Bun/Deno wrappers for malformed protocol messages
- Add stderr logging for unrecognized stdin commands in all wrappers
- Remove asyncio handling from Python wrapper (consistent with normal path)
- Add exec_preprocess protocol tests for Bun, Deno, and Python
- Add argument transformation tests (dates, bytes, kwargs, sentinel)
- Add relative import detection test for Python wrapper
- Add PreprocessedArgs variant to DedicatedWorkerResult test helper

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove symlink from git and gate has_relative_imports behind private feature

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update ee-repo-ref for dedicated_worker_ee.rs changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add mixed exec+preprocess test to use ProtocolCmd::Exec variant

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove hanging deno missing-preprocessor test

The Deno wrapper only generates the exec_preprocess handler when the
script has a preprocessor function. Without one, the message is
unrecognized and the test hangs reading stdout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 182943e5ad9bf2a905ccdf07d4e346437fb329a9

This commit updates the EE repository reference after PR #466 was merged in windmill-ee-private.

Previous ee-repo-ref: 995f701fe3754be6260fc6b679e5de8fc636e68a

New ee-repo-ref: 182943e5ad9bf2a905ccdf07d4e346437fb329a9

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-25 15:13:04 +00:00
Ruben Fiszel 31d6660d56 feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing modules field to RawCode in bun_executor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* feat: enrich WAC templates with checkpoint and replay semantics

Add prominent comments explaining that all computation must happen
inside task/step/taskScript or it will be replayed on resume/retry.
Clarify that waitForApproval does not hold a worker and that
approve/reject URLs are available in the timeline step details.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): script module sync idempotency, per-module hash tracking, and preview support

- Fix pull→push idempotency: use `??` instead of `||` for module lock
  field so empty strings are preserved (matches API's `lock: ""`)
- Add per-module hash tracking in wmill-lock.yaml following the flow
  inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes)
- Selective module lock regeneration: only regenerate locks for modules
  whose content actually changed, not all modules
- Use unfiltered rawWorkspaceDependencies for module hashes to match
  what updateModuleLocks passes to fetchScriptLock
- Show changed module names in stale script output for clarity
- Add module support to `script preview` command: read modules from
  __mod/ folder and pass them in the preview API request
- Add preview tests for taskScript pattern (flat and folder layout)
- Update test assertion for module stale detection output

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation

- Reorder WAC template buttons: TypeScript before Python in
  ScriptBuilder, CreateActionsScript, and CreateActionsFlow
- Remove dropdown items from +Script button (simplify to direct link)
- Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer
- Add module tab rename: pencil icon on hover opens popover with
  validation, fixed-width icon container prevents layout shift

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: remaining module-mode changes from working branch

- Backend parser updates for WAC detection
- CLI sync/types updates for raw app path and module support
- Frontend UI polish (Dev.svelte, ScriptRow, script hash page)
- Test fixture updates

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(cli): add test for module modification detection in generate-metadata

Verifies that modifying a single module file re-triggers stale
detection and only the changed module is listed, not all modules.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): critical fixes from PR review

- Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use
  "windmill-client" import instead of absolute filesystem path
- Fix missed no_main_func → auto_kind rename in parser TS test
- Add modules column to clone_script SQL (windmill-common and
  windmill-api-workspaces) so cloned scripts retain their modules
- Add modules: None to RawCode structs in worker tests
- Restore complete sqlx cache (merge main's cache + our new queries)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix clone warning treated as error in CI

Change `.clone()` on double reference to `*k` dereference in
scripts.rs hash implementation. Update sqlx cache with new query
hashes from modified clone_script SQL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use published parser wasm versions for CI build

The local file:// paths for windmill-parser-wasm-py and
windmill-parser-wasm-ts don't exist in the Cloudflare Pages build
environment. Revert to published npm versions (1.655.0).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): update parser wasm packages to 1.657.2

Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py
v1.657.2 which include auto_kind/WAC detection changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): regenerate package-lock.json for npm ci compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(frontend): use main's lockfile as base, update only parser wasm packages

Regenerating package-lock.json from scratch pulled different dependency
versions causing svelte-check type errors. Instead, start from main's
lockfile and only update the two changed packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): add modules column to fetch_script_for_update query

The Script<SR> struct has a modules field (FromRow), but
fetch_script_for_update didn't SELECT modules, causing a runtime
error "no column found for name: modules" when the worker processed
dependency jobs. This was the root cause of the relock_skip test
timeout.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix script module execution for Python and Bun

- Fix modules not passed through job queue: inject _MODULES into
  PushArgs.extra when pushing Code jobs so worker can extract them
- Fix Python module imports: use relative imports (from .helper)
  and add sys.path.insert for module directory in wrapper
- Fix Python tests: use relative imports and empty lock to prevent
  pip from resolving module names as packages
- Add local file check in Bun loader for module resolution
- Ignore Bun module test (bundle mode loader integration tracked
  separately)
- Add missing modules column to fetch_script_for_update query

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): remove unnecessary empty lock in Python module tests

Relative imports (from .helper) are not parsed as pip packages,
so the empty lock workaround is not needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(backend): fix module execution for Python and Bun — all tests pass

Python modules:
- Use relative imports (from .helper import greet) since scripts run
  as packages
- Add sys.path.insert for module directory in wrapper to ensure local
  modules take precedence over pip packages with same name

Bun modules:
- Use bundled output (./out/main.js) as wrapper import when modules
  are present — the bundled output has module content inlined by
  Bun.build, avoiding runtime loader resolution issues
- Add local file check in loader.bun.js onResolve to short-circuit
  API URL resolution for module files on disk

Job queue:
- Inject _MODULES into PushArgs.extra when pushing Code jobs so
  the worker can extract them at execution time

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — simplify, fix correctness, remove dead code

Critical fixes:
- Replace all CLI `no_main_func` references with `auto_kind` (string)
  to match the backend migration and API changes
- Remove duplicated `compute_python_module_dir` in worker.rs, use
  the canonical version from python_executor.rs

High priority:
- Auto-create `__init__.py` in intermediate directories for nested
  Python modules so imports like `from .utils.math import add` work
  without users manually creating __init__.py files
- Remove redundant `sys_path_insert` — relative imports use Python's
  package system, not sys.path

Medium:
- Fix lock file base name extraction: use regex to strip only the
  final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")`
  which breaks for files like `helper.test.ts`

Simplification:
- Remove dead `{#if false}` Popover block in ScriptEditor.svelte
- Guard loader.bun.js local file check to only run for relative paths
  (matching the Windows loader pattern)
- Add clarifying comment on Bun dual mechanism (build + run phases)
- Add maintenance comment on manual Hash impl for NewScript

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: final review fixes — stale cleanup, baseName, auto_kind export

- Fix sync.ts baseName extraction using indexOf(".") → regex
  (same fix as script.ts/metadata.ts, missed this instance)
- Add stale module file cleanup in writeModulesToDisk: removes files
  from __mod/ that are no longer in the modules map before writing,
  fixing the pull→push cycle that couldn't delete modules
- Log warning when _MODULES serialization fails in job push instead
  of silently dropping modules
- Use strict equality (===) for auto_kind comparison
- Exclude auto_kind from workspace export — it is auto-detected by
  the parser at deploy time from script content

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): remove auto_kind from push, comparison, and metadata

auto_kind is auto-detected by the parser at deploy time, so the CLI
should not send it, compare it, or write it to script.yaml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove erroneously added backend/backend/.sqlx directory

Duplicate .sqlx cache was committed at the wrong nested path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback + fix CI dead_code warning

Frontend (ScriptEditor.svelte):
- Fix switchToMain() missing lastSyncedCode update — prevents stale
  code sync on external changes while editing a module tab
- Fix formatAction saving module code to main script's localStorage
  draft — now saves main code when on a module tab
- Fix non-null assertion on inferModuleLang in renameModule — fall
  back to original language instead of force unwrap
- Remove redundant activeModuleTab truthy check in runTest

CLI (script.ts):
- Clean up empty directories after removing stale module files in
  writeModulesToDisk

Backend:
- Add path traversal guard in write_module_files — reject module
  paths containing ".."
- Fix dead_code warning on auto_kind field in workspace export struct

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(frontend): improve auto_kind UX + address review findings

- Rename "Include without main function" toggle to "Include library
  scripts" in script list (ItemsList.svelte)
- Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip
- Filter module file extensions by main script language — Python
  scripts only allow .py modules, TypeScript only .ts, etc.
- Split flushModuleState into flushModuleContent (no UI side-effect)
  and flushModuleState (flush + reset tab), reducing duplication
- Dynamic placeholder and hint text in add module popover based on
  main script language

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-17 01:20:09 +00:00
Ruben Fiszel 424ca59dfe feat: make WINDMILL_DIR configurable via environment variable (#8215)
* fix: auto-heal corrupted python runtime cache on remote workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: auto-heal corrupted python runtime cache on remote workers"

This reverts commit 0ea013a554.

* feat: make WINDMILL_DIR configurable via environment variable

Allow users to configure the base directory for Windmill's tmp/cache files
via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python
runtime cache corruption on RHEL systems where systemd-tmpfiles-clean
removes files from /tmp.

Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory
constants from compile-time const &str (concatcp!) to runtime lazy_static
String values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref to branch name for CI compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref lazy_static constants in all executor files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: panic if WINDMILL_DIR has trailing slash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: also reject trailing backslash in WINDMILL_DIR for Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref GO_BIN_CACHE_DIR in test utils

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore Windows /tmp path translation in go and bun executors

The Windows path translation replaces /tmp with the Windows temp dir
(e.g. C:\tmp) before normalizing slashes. Without this, the default
WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on
Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private.

Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f

New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-04 08:53:25 +00:00
Ruben Fiszel ea52a8b8ce refactor: move integration tests to subcrates to reduce recompilation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 14:04:54 +00:00
Pyra fe56191422 feat(internal): runnable settings (#7298)
* rework everything again

Signed-off-by: pyranota <pyra@duck.com>

* updcate sqlx

Signed-off-by: pyranota <pyra@duck.com>

* update ref

Signed-off-by: pyranota <pyra@duck.com>

* fix things

Signed-off-by: pyranota <pyra@duck.com>

* fix function

Signed-off-by: pyranota <pyra@duck.com>

* final fixes

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* fix script creation

Signed-off-by: pyranota <pyra@duck.com>

* address todo

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* fixups

Signed-off-by: pyranota <pyra@duck.com>

* fix cargo.toml

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

* ref

Signed-off-by: pyranota <pyra@duck.com>

* fix

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg

Signed-off-by: pyranota <pyra@duck.com>

* sqlx

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395

This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.

Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c

New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395

Automated by sync-ee-ref workflow.

* ci: force runnable settings

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Diego Imbert 7b1a8b0dce fix: improve S3 etag caching (#7301) 2025-12-05 18:01:27 +00:00
Pyra 1dfb0f0078 refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs

Signed-off-by: pyranota <pyra@duck.com>

* cleanup queue::jobs::pull

Signed-off-by: pyranota <pyra@duck.com>

* done

Signed-off-by: pyranota <pyra@duck.com>

* migrated the rest

Signed-off-by: pyranota <pyra@duck.com>

* implement todo!

* impmenet todo! v2

* more cleanup

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* remove unrelated code

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* do not serialize rawcode

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
Pyra d38c96db36 feat: workspace dependencies (#7124)
* commit raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* implement `parse_annotation`

Signed-off-by: pyranota <pyra@duck.com>

* more progress on wdeps

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* cli improvements + raw deps

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix python versions

Signed-off-by: pyranota <pyra@duck.com>

* progress

Signed-off-by: pyranota <pyra@duck.com>

* update :)

Signed-off-by: pyranota <pyra@duck.com>

* add MaybeLock

Signed-off-by: pyranota <pyra@duck.com>

* go WIP

Signed-off-by: pyranota <pyra@duck.com>

* fix python ignoring py version from requirements

Signed-off-by: pyranota <pyra@duck.com>

* optimize php

Signed-off-by: pyranota <pyra@duck.com>

* require admin to alter

Signed-off-by: pyranota <pyra@duck.com>

* fix(cli): flow generateLocks raw deps

Signed-off-by: pyranota <pyra@duck.com>

* progress in checklist

Signed-off-by: pyranota <pyra@duck.com>

* fix agent workers

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nit: remove default features

Signed-off-by: pyranota <pyra@duck.com>

* oh-wow

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* add indexes

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* add debug flag

Signed-off-by: pyranota <pyra@duck.com>

* cli: remove noise

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* trigger deps correctly

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend again

Signed-off-by: pyranota <pyra@duck.com>

* finally fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* more fixes...

Signed-off-by: pyranota <pyra@duck.com>

* remove test

Signed-off-by: pyranota <pyra@duck.com>

* Update backend-test.yml

* comment out legacy test

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix ci?

Signed-off-by: pyranota <pyra@duck.com>

* comment out thing

Signed-off-by: pyranota <pyra@duck.com>

* ignore test

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* base fixture

Signed-off-by: pyranota <pyra@duck.com>

* fix regression

Signed-off-by: pyranota <pyra@duck.com>

* fix docs links

Signed-off-by: pyranota <pyra@duck.com>

* update min version

Signed-off-by: pyranota <pyra@duck.com>

* simplify

* implement cache for get_latest

Signed-off-by: pyranota <pyra@duck.com>

* move to workspace settings

Signed-off-by: pyranota <pyra@duck.com>

* sqlx + migration

Signed-off-by: pyranota <pyra@duck.com>

* more migrations

Signed-off-by: pyranota <pyra@duck.com>

* use box pin

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-28 16:00:11 +00:00
Pyra edece035f8 feat: job debouncing (#6878)
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
Pyra ed806bf9d0 fix(backend): rework dependency_map handling (#6598)
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-09-24 13:40:39 +00:00
Ruben Fiszel cf70265ee9 rust tests cleanup (#6573)
* test improvements

* improve tests

* improve tests
2025-09-10 14:43:10 +00:00