Commit Graph

1898 Commits

Author SHA1 Message Date
Ruben Fiszel 4e91f83b8f chore(main): release 1.703.3 (#9200)
* chore(main): release 1.703.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-18 09:09:50 +00:00
Ruben Fiszel 20719b4731 chore(main): release 1.703.2 (#9195)
* chore(main): release 1.703.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-17 12:59:58 +00:00
Ruben Fiszel fa090f3081 chore(main): release 1.703.1 (#9182)
* chore(main): release 1.703.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-16 08:42:33 +00:00
Ruben Fiszel e3a3dbb89c chore(main): release 1.703.0 (#9170)
* chore(main): release 1.703.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-15 13:15:28 +00:00
Ruben Fiszel 7f589a8c7d chore(main): release 1.702.1 (#9166)
* chore(main): release 1.702.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-14 10:54:35 +00:00
Ruben Fiszel 90f494975d chore(main): release 1.702.0 (#9160)
* chore(main): release 1.702.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-14 09:18:42 +00:00
Ruben Fiszel 4f3a1e3109 chore(main): release 1.701.0 (#9131)
* chore(main): release 1.701.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-13 16:19:55 +00:00
centdix 17cf538a2d refactor: move ai providers to windmill-ai (#9120) 2026-05-12 14:09:31 +00:00
Ruben Fiszel c6346aabe0 chore(main): release 1.700.2 (#9117)
* chore(main): release 1.700.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-12 12:32:07 +00:00
Ruben Fiszel 411ca47ffd chore(main): release 1.700.1 (#9114)
* chore(main): release 1.700.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-11 23:23:08 +00:00
Ruben Fiszel b972fabab4 chore(main): release 1.700.0 (#9092)
* chore(main): release 1.700.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-11 22:37:58 +00:00
Ruben Fiszel 0a5f8dcd48 deps: pin tokio-postgres to forked branch with query_typed_raw deadlock fix (#9106)
* deps: pin tokio-postgres to MaterializeInc fork to fix query_typed_raw deadlock

`pg_executor`'s `Client::query_typed_raw` (and `Client::prepare` on the
streaming path) deadlock when the result schema contains a column whose
Oid the client doesn't know yet — citext, custom enums, custom domains,
postgis types. Easy to reproduce against any partitioned table with a
citext column: ~100+ rows is enough on localhost, less on slower links.
`psql` works fine for the same query because the simple-query protocol
doesn't trigger the typeinfo lookup path.

## Root cause (unchanged tokio-postgres bug for years)

`query::query_typed` calls `get_type(client, oid).await` synchronously
while still holding the original query's `Responses` stream. The
original query's `DataRow`s back up in the per-request
`mpsc::channel(1)`, `Connection::poll_read` stops draining the wire,
and the typeinfo sub-query response (queued on the same socket behind
those DataRows) never arrives. Classic head-of-line blocking.

## Fix

Pin `tokio-postgres` / `postgres-types` / `postgres-protocol` (via
[patch.crates-io]) and the workspace `rust-postgres` /
`rust-postgres-native-tls` aliases to the
[MaterializeInc rust-postgres fork at `78c1222577`](https://github.com/MaterializeInc/rust-postgres/tree/master).
MI's [PR #33 "bigger-channels"](https://github.com/MaterializeInc/rust-postgres/pull/33)
(merged 2025-12-11) resized the per-request response channel from
`mpsc::channel(1)` → `mpsc::channel(1024)`. That gives the connection
task 1024 batches of headroom while a streaming consumer is paused
mid-stream — orders of magnitude more than realistic typeinfo
deferral needs (≈3 batches).

## Why MaterializeInc and not a windmill-labs fork

`windmill-trigger-postgres` already depended on the imor fork for the
`postgres-replication` crate (logical replication: `CopyBothDuplex`,
`LogicalReplicationStream`, `TupleData` decoding including binary
tuples). That crate has never been on upstream rust-postgres — petrosagg's
[PR #752](https://github.com/rust-postgres/rust-postgres/pull/752) was
closed in 2021 in favour of a smaller split,
[PR #778](https://github.com/rust-postgres/rust-postgres/pull/778) is
still open today after five years. petrosagg keeps the replication work
alive on the MaterializeInc fork.

MaterializeInc is a strict superset of what we previously got from imor:
- imor's binary-tuple commit (sha `20265ef38e`) was merged into MI master.
- petrosagg has added perf + correctness fixes on top (allocation reuse,
  proper decoding fixes).
- The deadlock mitigation (`channel(1024)`) was added three weeks before
  this issue surfaced.

MI tracks upstream rust-postgres with a periodic catch-up merge (12-18 mo
cadence; last on 2025-12-03, ~100 commits picked up). Not an abandoned
fork.

## Why this works now (didn't on earlier attempt)

A previous attempt at this PR (`248ccb5a97`) hit CI failure because the
MI fork's `postgres-types 0.2.11` requires `serde_core ^1.0.221`, but
Windmill's workspace pinned `serde = "=1.0.220"` for swc_common 0.37.5's
`pub use serde::__private as serde;` hack. Bumping serde above 1.0.220
broke the swc_ecma_ast `Deserialize` derive under the
`enterprise,deno_core,…` feature set.

The earlier blocker is now resolved by #9111 which bumped the deno +
swc pin set to a "goldilocks" combination where `swc_common 14.0.4`
drops the `__private` hack, freeing the workspace serde pin to `^1`.
serde now resolves to 1.0.228, which satisfies MI's `serde_core ^1.0.221`
requirement transitively — no extra workspace pin needed.

## Diff shape

Two files only:
- `backend/Cargo.toml` (+34/-1): three new `[patch.crates-io]` entries
  (`tokio-postgres`, `postgres-types`, `postgres-protocol` → MI fork)
  plus comment block, plus the two workspace deps (`rust-postgres` /
  `rust-postgres-native-tls`) repointed from imor's fork to MI's.
- `backend/Cargo.lock` — auto-regenerated. Replaces all `imor/rust-postgres`
  references with `MaterializeInc/rust-postgres`, bumps the affected
  crate versions to MI's set (tokio-postgres 0.7.11 → 0.7.15,
  postgres-types 0.2.7 → 0.2.11, postgres-protocol 0.6.7 → 0.6.9,
  postgres-native-tls 0.5.0 → 0.5.2). No source code changes.

## Verification

- `cargo check --features quickjs` → clean.
- `cargo check -p windmill-worker --features quickjs` → clean (pg_executor
  builds).
- Repro tested earlier in the thread that produced this PR: the
  partitioned-citext-table query on Neon goes from "hangs indefinitely"
  (server idle on `wait_event=ClientRead` while client awaits typeinfo
  behind undrained DataRows) to "completes in ~1.0s, 100 rows" with the
  MI fork's `bounded(1024)` response channel.

## Caveats

- **`bounded(1024)` is a mitigation, not a closure.** Theoretical failure
  mode remains at >~64 MB single-query results with a custom-Oid column
  (typeinfo defers for >1024 batches of ~64 KB each). The strict-correct
  fix is `mpsc::unbounded()` — proposed as a follow-up PR to MI. For
  realistic Windmill workloads, 1024 batches of headroom is well past
  the ~3-batch typeinfo deferral that's actually needed.
- **`postgres-replication` is now upstream-of-fork's only home.** No
  realistic path to upstream rust-postgres merging it. The MI pin is
  intended to stay in place until either upstream changes course
  (unlikely) or MI publishes to crates.io (also unlikely — they don't
  publish releases of the fork).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* chore: update ee-repo-ref to 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c

This commit updates the EE repository reference after PR #568 was merged in windmill-ee-private.

Previous ee-repo-ref: 4d01d171228196f28ddabc1150242bfab623d5cf

New ee-repo-ref: 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 21:43:18 +00:00
Ruben Fiszel 3cd0eac8c1 deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling (#9111)
* deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling

Bumps every deno_* and swc_* workspace dep to a hand-picked "goldilocks"
combination that drops the serde =1.0.220 ceiling without crashing into
the rustls / aws-sdk resolver wall that the obvious deno v2.6.0 target
hits.

## What's the goldilocks set

| crate            | old      | new       | source                                |
|------------------|----------|-----------|---------------------------------------|
| deno_core        | 0.336.0  | 0.352.0   | deno v2.4.0                           |
| deno_fetch       | 0.214.0  | 0.233.0   | deno v2.4.0                           |
| deno_tls         | 0.177.0  | 0.196.0   | deno v2.4.0 (last permissive-rustls)  |
| deno_console     | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_url         | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_webidl      | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_web         | 0.221.0  | 0.240.0   | deno v2.4.0                           |
| deno_io          | 0.100.0  | 0.119.0   | deno v2.4.0                           |
| deno_net         | 0.182.0  | 0.201.0   | deno v2.4.0                           |
| deno_permissions | 0.49.0   | 0.68.0    | deno v2.4.0                           |
| deno_telemetry   | 0.12.0   | 0.31.0    | deno v2.4.0                           |
| deno_error       | =0.5.5   | =0.6.1    | deno v2.4.0                           |
| deno_ast         | =0.44.0  | =0.51.0   | **override** — see "load-bearing" below |
| deno_fs          | (new)    | 0.119.0   | new workspace dep — FetchPermissions exposes deno_fs::CheckedPath / GetPath as public API |
| v8               | =130.0.7 | =137.1.0  | deno_core 0.352 transitive            |
| swc_common       | =0.37.5  | =14.0.4   | **the load-bearing pin**              |
| swc_ecma_ast     | =0.118.2 | =15.0.0   | matched set with swc_common 14.0.4    |
| swc_ecma_parser  | =0.149.1 | =24.0.3   | matched set                           |
| swc_ecma_visit   | =0.104.8 | =15.0.0   | matched set                           |
| serde            | =1.0.220 | ^1        | **freed** (resolves to 1.0.228+)      |

## Why this combination and not v2.6.0

The obvious target was deno v2.6.0 (with deno_ast 0.52 → swc_common 17,
well past the `__private` ceiling). That hits three resolver collisions:

1. libsqlite3-sys: deno_cache → rusqlite 0.37 → libsqlite3-sys 0.35
   vs sqlx → libsqlite3-sys 0.30. **Already killed by PR #9110** —
   we dropped deno_runtime, which is what pulled in deno_cache.
2. fqdn 0.4.6/0.4.7 yanked, required by deno_permissions 0.81.0. Solvable
   by injecting the yanked entry into Cargo.lock manually but ugly.
3. rustls: deno_tls 0.198+ hard-pins `=0.23.28`, but aws-sdk-bedrockruntime
   1.122.0 → aws-smithy-http-client 1.1.5 wants `^0.23.31`. Within-major
   conflict, no resolver path. The unbeatable wall.

Goldilocks-set choice sidesteps (2) and (3) entirely:

- `deno_tls 0.196.0` was the last version before deno tightened
  `rustls ^0.23.11` (range, accepts 0.23.31) to exact `=0.23.28`. With
  ^0.23.11, the resolver picks rustls 0.23.35 (latest 0.23 patch) which
  satisfies both deno_tls's `>=0.23.11` and aws-sdk's `>=0.23.31`. Verified
  empirically: lockfile has rustls 0.23.35 after this bump.
- `deno_permissions 0.68.0` (v2.4.0's pin) doesn't depend on fqdn at all.
  The fqdn dep was added in a later deno_permissions release.

## Why deno_ast =0.51.0 specifically (not 0.48.0 from v2.4.0)

`swc_common 14.0.4` is the first patch that **drops the
`pub use serde::__private as serde;` line** in `src/private/mod.rs`. Older
14.0.x and all 0.37.5–13.x revisions still have it, and that line is
what was capping `serde = "=1.0.220"` (the workspace pin's "stuck because
of swc" comment). Empirically verified by inspecting the tarballs of
14.0.0 / 14.0.1 / 14.0.2 / 14.0.3 / 14.0.4:

    14.0.0: has hack
    14.0.1: has hack
    14.0.2: has hack
    14.0.3: has hack
    14.0.4: NO HACK    ← inflection point

`deno_ast 0.51.0` pins `swc_common =14.0.4` exactly — older deno_ast
versions pin earlier swc_common patches that still have the hack.
Notably, deno v2.4.0 itself pins `deno_ast =0.48.0` (swc_common 9.2.0,
still has hack) — we deliberately deviate from v2.4.0's deno_ast pin
to escape the swc serde wall, while keeping the rest of v2.4.0's pin
set for resolver compatibility with aws-sdk. deno_ast 0.51 was never
shipped in any deno release (v2.4.5 used 0.49, v2.5.0 jumped to 0.50,
v2.6.0 to 0.52), but it's published on crates.io and compatible with
v2.4.0's deno_core 0.352.

## What this unblocks

- PR #9106's `serde = "=1.0.224"` bump variant can rebase onto this
  and resolve cleanly (MaterializeInc/rust-postgres' `postgres-types`
  needs `serde_core ^1.0.221`, which is satisfied now that we're on
  serde 1.0.228).
- Future deno_* / swc_* bumps no longer need to argue about the serde
  ceiling — it's gone.

## What changes in source code

This commit is Cargo.toml + Cargo.lock only. Source changes that the
new deno_core / deno_fetch API requires live in the follow-up commits:

- `parsers/windmill-parser-{ts,ts-asset,wac}`: swc 0.37 → 14
  (`code.into()` ambiguity fix at 5 sites)
- `windmill-runtime-nativets/build.rs` + `src/lib.rs`: deno_core 0.336
  → 0.352 API moves (`init_ops_and_esm()` → `init()`,
  `FetchPermissions` / `NetPermissions` trait signature updates,
  `deno_tls::Proxy` enum shape change)

A companion change in windmill-ee-private adjusts
`otel_tracing_proxy_ee.rs:521` for `deno_telemetry::init`'s second arg
becoming by-value (was `&OtelConfig`).

* fix(parsers): adapt to swc_common 14 BytesStr ambiguity

swc 0.37.5 → 14.0.4 changed `SourceMap::new_source_file`'s `src` argument
from `String` to `impl Into<BytesStr>`. With `BytesStr` available, the
existing call sites' `code.into()` on a `&str` becomes ambiguous between
`Into<Bytes>` (from the bytes crate) and `Into<BytesStr>` (from
bytes_str). Switch to `code.to_string()` to produce an owned `String`
that satisfies `From<String> for BytesStr` unambiguously.

Five call sites across three crates:
- windmill-parser-ts/src/lib.rs (3 sites)
- windmill-parser-ts-asset/src/lib.rs (1 site)
- windmill-parser-wac/src/typescript.rs (1 site)

* fix(nativets): adapt to deno_core 0.352 / deno_fetch 0.233 API changes

The goldilocks deno bump (deno_core 0.336 → 0.352, deno_fetch 0.214 →
0.233, etc.) ripples through nativets' build.rs and src/lib.rs.
Source-level changes required:

## 1. `extension!` macro: `init_ops_and_esm()` and `init_ops()` removed

deno_core 0.352's `extension!` macro now generates a single `init()`
function on the extension struct (full: ops + esm), plus `lazy_init()`
(ops only, with `needs_lazy_init = true` and a contract that the
caller invokes `JsRuntime::lazy_init_extensions` after construction).

- `build.rs` (snapshot creation, wants both ops and esm baked in):
  `X::init_ops_and_esm(...)` → `X::init(...)`.
- `src/lib.rs:create_nativets_runtime` (runtime, was using `init_ops()`
  because the snapshot already provides esm): also → `X::init(...)`.
  deno_core's snapshot path skips esm re-execution when the snapshot
  provides them, so the esm registration is a no-op at runtime. This
  is how deno's own v2.4.0 runtime works.

Avoided `lazy_init` because it requires plumbing
`JsRuntime::lazy_init_extensions(ext_args_vec)` correctly across the
codebase, which is invasive for no behavioural benefit.

## 2. Local `fetch` extension now declared in both build.rs and lib.rs

deno_core 0.352 validates extension order between snapshot and runtime.
Our snapshot's last extension is the local `fetch` ext (which provides
ext:fetch/src/runtime.js). To avoid a runtime panic:

    "Extensions from snapshot loaded in wrong order: expected fetch but got windmill"

…the runtime extension list now ends with `fetch::init()` matching the
snapshot order. The macro requires the same `esm` argument to type-check,
even though the ESM is not re-executed at runtime (it's in the snapshot).

## 3. `FetchPermissions` and `NetPermissions` trait shape

`deno_fetch::FetchPermissions` (deno_fetch 0.233.0) added new methods
and changed signatures:

- `check_read` / `check_write`: now take `path: Cow<'a, Path>` plus
  a new `get_path: &'a dyn deno_fs::GetPath` parameter, and return
  `Result<deno_fs::CheckedPath<'a>, FsError>` instead of `Result<Cow<Path>, FsError>`.
- New `check_write` (didn't exist) and `check_net_vsock` methods.

`deno_net::NetPermissions` (deno_net 0.201.0) gained `check_vsock`
and `check_write_path` now takes `Cow<'_, Path>`.

For `build.rs`'s `PermissionsContainer` (used only during snapshot
creation, where permissions are never actually checked): all methods
`unreachable!("snapshotting")`.

For `src/lib.rs`'s `PermissionsContainer` (used at runtime — the
nativets policy is "allow everything"): `check_read` / `check_write`
return `Ok(CheckedPath::Unresolved(path))`, `check_*_vsock` return
`Ok(())`. Smoke tests confirm fetch/net/url/web/blob/timers/structuredClone
behaviour is intact end-to-end.

## 4. `deno_tls::Proxy` is now an enum

`deno_tls::Proxy` was a struct, is now an enum with `Http`, `Https`,
`Socks5` variants. Our call site uses HTTP proxies — switched the
struct literal `deno_tls::Proxy { url, basic_auth }` to
`deno_tls::Proxy::Http { url, basic_auth }`.

## 5. New `deno_fs` direct workspace dep

`FetchPermissions` exposes `deno_fs::CheckedPath` and `deno_fs::GetPath`
as part of its public API. We can't avoid naming `deno_fs` directly any
more. Pinned to 0.119.0 (v2.4.0's matched version, transitively present
already through deno_fetch). Added to workspace `[dependencies]` plus
nativets's `[dependencies]` and `[build-dependencies]`.

## Validation

`cargo check --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline`
→ clean.

`cargo test -p windmill-runtime-nativets smoke -- --ignored --skip smoke_net_`
→ 8 passed; 0 failed (the full local smoke suite covering fetch,
setTimeout/Promise.all, URL/SearchParams, Blob/btoa/atob, large payload
roundtrip, error propagation, concurrent isolates, TS enum/union
transpile).

Network smoke tests (`smoke_net_fetch_example_com`,
`smoke_net_fetch_json_and_headers`) not run as part of the validation
gate but expected to pass — the change preserves deno_fetch behaviour
through the trait surface.

* chore: update ee-repo-ref to pick up deno_telemetry::init by-value fix

Points at windmill-ee-private branch deps/bump-deno-and-swc-goldilocks
which contains the companion otel_tracing_proxy_ee.rs adjustment for
deno_telemetry 0.12 → 0.31 (second arg of `init` is now by-value).
EE-only file, doesn't affect OSS build.

* chore(nix): bump rusty_v8 in flake.nix to 137.1.0 to match Cargo.toml

Cargo.toml's v8 pin moved from =130.0.7 to =137.1.0 as part of the
deno_core 0.336 → 0.352 bump, but I missed the comment directly above
the version pin:

    # Exact version NOTE: Do not forget to update version and hash in flake.nix

flake.nix provides the prebuilt librusty_v8 binary that the v8 crate
links against. A version mismatch would either fail to fetch (if the
137.1.0 release didn't exist) or cause link-time symbol mismatches.
Nix is used by rust-client-check.yml and rust_on_release.yml in CI,
plus the dev shell — stale flake pin breaks all of those.

Updates x86_64-linux's sha256 to match the actual hash of
librusty_v8_release_x86_64-unknown-linux-gnu.a.gz at the 137.1.0 tag.
Other targets (aarch64-linux, x86_64-darwin, aarch64-darwin) remain
as lib.fakeHash — they were already placeholders in the previous
pin, so we don't regress on them.

Caught by both cubic and Pi reviewers on PR #9111.

* docs(nativets): clarify snapshot-prefix rule in extension-order comment

Claude reviewer caught that the doc comment claimed the runtime
extension list matches the snapshot's order — implying an exact match.
The truth is more permissive: deno_core 0.352 requires the snapshot's
extension list to be a *prefix* of the runtime's, not an exact match.
Runtime is allowed to append extra extensions (which we do — the
windmill `ext` carrying our ops is the last entry at runtime but absent
from the snapshot).

The code is correct as-is; only the comment wording was misleading.

Also fixes the same wording in PR description.
2026-05-11 21:13:24 +00:00
Ruben Fiszel 36b316d9e8 deps(nativets): inline maybe_transpile_source, drop deno_runtime (#9110)
windmill-runtime-nativets was the workspace's only consumer of the
deno_runtime crate, and its only use of it was one call site in
build.rs:

    deno_runtime::transpile::maybe_transpile_source(specifier, source)

That function (`deno_runtime-0.198.0/transpile.rs`, ~80 lines) is a pure
deno_ast + deno_core + deno_error wrapper — it doesn't touch any
deno_runtime state. Inline it verbatim into our build.rs and drop the
entire deno_runtime dep.

Why this matters now: deno_runtime transitively pulls in deno_cache →
rusqlite → libsqlite3-sys. From deno_cache 0.128.0 (Feb-Mar 2025)
onwards, rusqlite was bumped to ^0.34, which means libsqlite3-sys ^0.35.
sqlx 0.8 transitively requires libsqlite3-sys ^0.30. Cargo's `links =
"sqlite3"` rule allows only one libsqlite3-sys in a build graph, so the
two crates collide on any deno release ≥ v2.5. Inlining the transpile
helper sidesteps the collision entirely — sqlx-sqlite stays the sole
libsqlite3-sys consumer at 0.30.1.

All other appearances of "deno_runtime" in the source tree are for a
Windmill-internal function named `setup_deno_runtime`, not the crate.

Build artifacts validated:
- `cargo check --features quickjs` → green.
- `cargo test -p windmill-runtime-nativets smoke -- --ignored --skip smoke_net_`
  → 8 passed (the in-process V8 runtime + deno_fetch + deno_web + swc
  transpilation surface still works end-to-end through the inlined
  function).
- `cargo tree --invert deno_cache` → "did not match any packages"
  (gone from the graph).
- Single `libsqlite3-sys` entry in Cargo.lock at 0.30.1 (sqlx's).
2026-05-11 20:21:46 +00:00
centdix 6f7d31e56b refactor: move ai image handling to windmill-ai (#9098) 2026-05-11 14:59:40 +00:00
centdix 27acbbf3d5 refactor: move ai sse plumbing to windmill-ai (#9059)
* docs: refine windmill ai refactor plan

* refactor: move ai sse plumbing to windmill-ai

* refactor: remove ai re-export shims

* fix: update ee ai memory ref

* chore: update ee-repo-ref to d3bc7fa85195b46b7a38d43c2f806520bf8b5454

This commit updates the EE repository reference after PR #560 was merged in windmill-ee-private.

Previous ee-repo-ref: ff35bf7cc198e13884b33654e1d6dbd8a8b314d3

New ee-repo-ref: d3bc7fa85195b46b7a38d43c2f806520bf8b5454

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 10:01:45 +00:00
Ruben Fiszel 91ddb930c3 chore(main): release 1.699.0 (#9082)
* chore(main): release 1.699.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-08 18:03:06 +00:00
Ruben Fiszel 2510a09750 chore(main): release 1.698.0 (#9076)
* chore(main): release 1.698.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-08 06:16:21 +00:00
Ruben Fiszel e1a7c75e19 perf(flows): cache resolved flow_env per flow execution (#9079)
* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures
2026-05-08 07:43:46 +02:00
Ruben Fiszel 0b0999ef93 chore(main): release 1.697.0 (#9067)
* chore(main): release 1.697.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-07 13:53:30 +00:00
hugocasa 80475f011b feat(kafka-trigger): OAUTHBEARER + SASL_SSL support (#9054)
* [ee] feat(kafka-trigger): enable librdkafka OIDC for OAUTHBEARER

Adds the curl-static feature to the rdkafka dependency so librdkafka is
built with libcurl + OpenSSL, which it requires to perform the
client_credentials token exchange used by sasl.oauthbearer.method=oidc.
Without this feature librdkafka rejects sasl.oauthbearer.token.endpoint.url
at runtime with "OAuth/OIDC depends on libcurl and OpenSSL which were not
available at build time".

Pairs with windmill-ee-private#<TBD> which adds the SASL_SSL_OAUTHBEARER
KafkaResourceSecurity variant.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to c6d55247a7fd951f561e0b2ad2ac13051274aa77

This commit updates the EE repository reference after PR #559 was merged in windmill-ee-private.

Previous ee-repo-ref: ce88502234f3f5886d2094dc25b30a702de9b8b1

New ee-repo-ref: c6d55247a7fd951f561e0b2ad2ac13051274aa77

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-05-07 13:25:39 +00:00
Ruben Fiszel 5ecb644dd7 chore(main): release 1.696.2 (#9066)
* chore(main): release 1.696.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-07 00:10:14 +00:00
Ruben Fiszel bfe80355b0 chore(main): release 1.696.1 (#9050)
* chore(main): release 1.696.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-06 10:27:59 +00:00
Ruben Fiszel f07f19ebe7 chore(main): release 1.696.0 (#9040)
* chore(main): release 1.696.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-05 21:20:40 +00:00
Ruben Fiszel 8247f4ee19 chore(main): release 1.695.0 (#9011)
* chore(main): release 1.695.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-04 21:48:04 +00:00
Ruben Fiszel dfe534b1a6 chore(main): release 1.694.0 (#8998)
* chore(main): release 1.694.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-01 21:02:43 +00:00
Ruben Fiszel 02fe2e7511 chore(main): release 1.693.4 (#8994)
* chore(main): release 1.693.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-01 06:57:17 -04:00
Ruben Fiszel 6922631b03 chore(main): release 1.693.3 (#8989)
* chore(main): release 1.693.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-30 14:21:04 +00:00
Ruben Fiszel 34ba176f52 chore(main): release 1.693.2 (#8987)
* chore(main): release 1.693.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-30 12:24:16 +00:00
Ruben Fiszel 8f68f048d8 chore(main): release 1.693.1 (#8982)
* chore(main): release 1.693.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-29 21:25:34 +00:00
Ruben Fiszel e147546b3d chore(main): release 1.693.0 (#8957)
* chore(main): release 1.693.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-29 19:59:25 +00:00
Ruben Fiszel b2004f357d chore(main): release 1.692.0 (#8950)
* chore(main): release 1.692.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-27 20:22:48 +00:00
Ruben Fiszel 76108ed5b2 chore(main): release 1.691.1 (#8941)
* chore(main): release 1.691.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-27 04:49:19 +00:00
Ruben Fiszel 612a39bcfc chore(main): release 1.691.0 (#8931)
* chore(main): release 1.691.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-24 19:49:44 +00:00
Ruben Fiszel 489337d533 feat: cli diff/deploy no-op handling + promotion debouncing (#8936)
* feat: cli diff & deploy no-op handling + promotion debouncing

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ed842061576c3ac9b9eb89bb87f6db5b67904474

This commit updates the EE repository reference after PR #551 was merged in windmill-ee-private.

Previous ee-repo-ref: 1210d9f63de8eea4c3a210a10c60fe6382df477b

New ee-repo-ref: ed842061576c3ac9b9eb89bb87f6db5b67904474

Automated by sync-ee-ref workflow.

* test(git-sync): e2e tests for promotion-mode debounce keys

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-24 18:12:10 +00:00
Ruben Fiszel 161ec8d722 chore(main): release 1.690.0 (#8921)
* chore(main): release 1.690.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-23 14:33:53 -04:00
hugocasa d6c642b170 feat: add Azure Event Grid triggers (#8888)
* feat: add Azure Event Grid triggers (EE)

Introduces a new enterprise trigger kind `azure` that supports three
modes via a single unified trigger type:
- basic_push: Azure Event Grid basic — custom topics, system topics
  (Storage, Resource Manager, Key Vault, etc.), domains (push only)
- namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push)
- namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token
  ack/reject for dead-lettering)

Auth uses a Service Principal resource (tenant_id, client_id,
client_secret, subscription_id). Subscriptions are created in
CloudEvents 1.0 schema so the push webhook handler and the pull listener
share one payload parser.

Backend
- New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from
  windmill-ee-private)
- Migration `azure_trigger` table with CHECK constraints enforcing
  mode/columns coherence
- `TriggerKind::Azure`, `JobTriggerKind::Azure`,
  `DeployedObject::AzureTrigger` variants
- Push route `/api/azure/w/{workspace}/*path` handles classic
  Event Grid SubscriptionValidation handshake and CloudEvents 1.0
  abuse-protection OPTIONS handshake
- Optional inbound JWT validation (audience check only for v1)
- Feature flag `azure_trigger` propagated through windmill-api,
  windmill-store (resource helper), and added to ee_core

Frontend
- `triggers/azure/` editor with mode toggle (basic/namespace-push/
  namespace-pull) and per-mode config (topic ARM id / namespace +
  topic name / subscription / filters / push auth / pull options)
- Registered in icon map, display names, save functions, badge,
  wrapper, editor, add-trigger menu

OpenAPI
- `AzureTrigger`, `AzureTriggerData`, `AzureMode`,
  `AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection`
  schemas; `/azure_triggers/*` endpoints; client regenerated

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8

This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private.

Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9

New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8

Automated by sync-ee-ref workflow.

* feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity

Frontend:
- Split mode selector into Namespace/Basic + Pull/Push
- ARM resource dropdowns (namespaces, Basic topics, namespace topics)
  populated from the service principal; cascade with stale-selection
  reset on SP / edition change
- Remove stale authenticate toggle + audience input (server-managed
  push_auth_config has replaced them)
- Azure listing page: "Create from template" button; "Also delete Azure
  subscription" toggle in the delete modal; simplified trigger label
  falling back to path
- AzureCapture.svelte: "Test subscription name" with -wm-capture suffix
- CompareWorkspaces.svelte: wire Azure for fork/compare
- Drop Trigger-deployed/event-loss warning (capture subscription is
  isolated with -wm-capture)

Backend:
- Shared-secret push auth (see EE crate for detail)
- JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)]
  so clients/CLI/exports never see it
- Drop redundant enabled column; mode supersedes
- Azure capture infra: AzureTriggerConfig + set_azure_trigger_config +
  azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on
  capture subscriptions so they bound storage after tab close
- Granular ACLs, users offboarding, trash, git-sync deployed-object:
  all include azure_trigger

CLI:
- Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath,
  trigger commands (get/update/create/list/template), sync delete
  switch + regex; e2e test for `trigger new --kind azure`
- system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger;
  auto-generated/* regenerated

Skill:
- .claude/skills/adding-a-trigger/ checklist covering every file that
  needs editing when wiring a new trigger type (learned from this PR)

ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts

- frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger'
  to the Kind type so the listing page's "Permissions" action compiles
  (ts2345 — caught by npm_check on CI, missed by fast-check locally).
- system_prompts/auto-generated/: regenerate to drop the stale
  delivery_config / AzureDeliveryConfig fields from the Azure schema
  (check-freshness on CI).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(azure-trigger): use workspace constant_time_eq crate

Drop hand-rolled constant-time compare in favour of the workspace
constant_time_eq crate (same one used by http_trigger_auth).

ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): pass placeholder + disabled via inputProps

`TextInput`'s `placeholder` and `disabled` go through its `inputProps`
prop — CI's `npm run check` caught the stale top-level passing that
`npm run check:fast` missed. Align with the DefaultEmailConfigSection
pattern.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213

The hub deploy of the azure-aware sync-script is version 28213, not
28214. Backend was pinning a non-existent hub script, which broke the
git_sync_e2e suite (every deploy's sync step 404'd).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(azure-trigger): add azure_triggers to token scope selector + skill

- windmill-api/src/token.rs: `build_trigger_scope_domains` was missing
  `("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope
  selector didn't surface azure_triggers:read/write. Backend already had
  `ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it.
- .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related
  files under the hardcoded-arrays section so future triggers don't miss
  the UI surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(adding-a-trigger-skill): clarify token.rs scope effect

Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS
just means the scope works via API/CLI but has no UI checkbox.

* docs(adding-a-trigger-skill): trim token.rs bullet

* fix(azure-trigger): regen openapi-deref + swap textarea for TextInput

- Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the
  12 azure_triggers paths + schemas. These files are served by the
  runtime (include_str! in windmill-api/src/lib.rs) to external SDK
  consumers; without this regen the new endpoints wouldn't be advertised.
- Replace the raw <textarea> for event type filters with the
  design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw
  HTML elements).

Addresses cubic + claude PR review items.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-23 16:30:18 +00:00
Ruben Fiszel 6abe33109a chore(main): release 1.689.0 (#8894)
* chore(main): release 1.689.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-22 20:06:31 +00:00
Ruben Fiszel e73770c247 fix: derive debug signing key deterministically from JWT_SECRET (#8917)
Previously each API replica generated a random Ed25519 signing key at
startup (unless DEBUG_SIGNING_KEY_SEED was set). In multi-replica
deployments this caused "Invalid JWT signature" rejections in the
multiplayer server: the browser could sign a token on pod A while
`windmill-extra` had cached the JWKS public key from pod B.

Derive the seed deterministically from the DB-backed JWT_SECRET using
SHA-256 with a domain-separation tag so all pods agree without
coordination. Re-derive on JWT_SECRET rotation. The
DEBUG_SIGNING_KEY_SEED env var is still honored as an override.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 20:05:08 +00:00
centdix 26a6d1e4ce refactor: create windmill-ai crate (part 1 — types, traits, base modules) (#8530)
* refactor: create windmill-ai crate and move base AI types from windmill-common

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move worker AI types to windmill-ai crate

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move QueryBuilder trait and StreamEventSink abstraction to windmill-ai

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add base64 dependency to windmill-ai for bedrock PDF support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add windmill-ai refactor plan

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — remove dead bedrock feature, add boxed_sink helper, move plan to docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 11:20:27 +00:00
Ruben Fiszel 18b1bf8f58 chore(main): release 1.688.0 (#8881)
* chore(main): release 1.688.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-20 20:37:37 +00:00
Ruben Fiszel e063db68c9 chore(main): release 1.687.0 (#8871)
* chore(main): release 1.687.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-17 18:16:51 +00:00
Ruben Fiszel f683c70ef4 chore(main): release 1.686.0 (#8860)
* chore(main): release 1.686.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-17 14:30:19 +00:00
Ruben Fiszel 24cb414fed chore(main): release 1.685.0 (#8838)
* chore(main): release 1.685.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-16 17:54:26 +00:00
Ruben Fiszel a98f5b9dfd chore(main): release 1.684.1 (#8834)
* chore(main): release 1.684.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 21:48:03 +00:00
Ruben Fiszel 75e204dad1 pin tree-sitter 2026-04-14 21:36:00 +00:00
Ruben Fiszel 8ee14644f4 chore(main): release 1.684.0 (#8831)
* chore(main): release 1.684.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 20:58:06 +00:00
Ruben Fiszel 6bb80ff28b chore(main): release 1.683.2 (#8820)
* chore(main): release 1.683.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-14 00:23:20 +00:00
Ruben Fiszel 5b3913052e refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert atomic primitive globals to AtomicBool/AtomicI64

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to arcswap-refactor EE branch commit

* refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit

* chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d

This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private.

Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4

New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d

Automated by sync-ee-ref workflow.

* fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c

This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private.

Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7

New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c

Automated by sync-ee-ref workflow.

* fix: secret_backend_integration test — BASE_URL.write().await → .store()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to e587df8 (post-#535 merge)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-14 00:04:10 +00:00
Ruben Fiszel eb85da932a chore(main): release 1.683.1 (#8817)
* chore(main): release 1.683.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-04-13 22:49:27 +00:00