Commit Graph
1931 Commits
Author SHA1 Message Date
Ruben Fiszelandrubenfiszel 66c0334e70 chore(main): release 1.721.0 (#9480)
* chore(main): release 1.721.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-09 08:14:02 +00:00
Ruben Fiszelandrubenfiszel a3740d571a chore(main): release 1.720.0 (#9464)
* chore(main): release 1.720.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-08 18:02:10 +00:00
Ruben Fiszelandrubenfiszel 004339032e chore(main): release 1.719.0 (#9459)
* chore(main): release 1.719.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-06 08:16:43 +00:00
Ruben FiszelandClaude Opus 4.8 fbdf81ba5f fix: authenticate slack callback payload with per-workspace hmac (#9461)
* fix: authenticate slack callback payload with per-workspace hmac

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: regression tests for unauthenticated slack callback decryption

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: verify slack submission signature before resume + close workspace oracle

Addresses review: verify private_metadata HMAC before handle_resume_action so a
tampered/unsigned submission is rejected up front, and map get_workspace_key
failure to the generic 401 so the status code is not a workspace-existence oracle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: domain-separate slack payload hmac from resume-secret hmac

Both MAC families key Hmac<Sha256> on the same per-workspace key; resume secrets
are distributed to approvers in resume URLs, so add a fixed domain tag
(slack_payload_v1) to the slack payload MAC to make the two non-interchangeable
by construction rather than by byte-layout coincidence.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 08:02:20 +00:00
Ruben Fiszelandrubenfiszel 3887bf67dc chore(main): release 1.718.0 (#9450)
* chore(main): release 1.718.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-05 14:55:10 +00:00
Ruben Fiszelandrubenfiszel 24fa61d3c0 chore(main): release 1.717.1 (#9444)
* chore(main): release 1.717.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-04 10:20:28 +00:00
Ruben FiszelandClaude Opus 4.8 f595787409 fix: invalidate relative-import cache when imported script changes (#9443)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-04 10:09:06 +00:00
Ruben Fiszelandrubenfiszel 6b6c16e6bc chore(main): release 1.717.0 (#9439)
* chore(main): release 1.717.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-04 07:37:15 +00:00
Ruben Fiszelandrubenfiszel d537c82c4f chore(main): release 1.716.0 (#9430)
* chore(main): release 1.716.0

* Apply automatic changes

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-03 14:01:02 +00:00
Ruben FiszelandClaude Opus 4.8 0ba128afe7 fix(security): scope variable and resource value caches by caller identity (#9427)
The variable and resource value caches (backing
`GET /api/w/{w}/variables/get_value/{path}?allow_cache=true` and
`.../resources/get_value_interpolated/{path}?allow_cache=true`) are consulted
before the per-folder RLS query and store the already-decrypted value. The
resource cache was keyed only by `workspace:path` with no caller identity, so a
cache entry warmed by a privileged peer using `allow_cache=true` could be
returned to a caller with no access to the resource's folder on a cache hit
within the 30s TTL — leaking another folder's decrypted secrets.

Scope both caches to the caller's full authorization identity. The key is now
`auth_identity(authed):workspace:path`, where `auth_identity` is a SHA-256 of the
caller's effective authorization context (email, username, is_admin, is_operator,
sorted groups, sorted folders, sorted scopes) — mirroring
`job_read_access_cache_key`. Email alone is insufficient: the same email can
resolve to different effective permissions via job/owner-scoped tokens, so a
lower-privilege context must not reuse a higher-privilege context's entry.

Job-context resource interpolation is handled correctly: only `$WM_*` contextual
variables are resolved (and only when a `job_id` is present). The interpolation
reports whether the value contains a `$WM_*` placeholder
(`transform_json_value_tracked` + an `AtomicBool`). A value containing one is
job-dependent — even on a no-job read where it's left unresolved — and is never
cached (so a later job read never gets a stale placeholder or another job's
context). Any value without a `$WM_*` placeholder is job-independent and cached
under the identity key, shared across job contexts, so reads carrying a `job_id`
still hit the cache.

BEHAVIOR CHANGE: custom workspace environment variables are no longer interpolated
into resource values via `$NAME` (this was undocumented and prevented caching of
any `$`-prefixed value). Custom envs remain available to scripts/workers as before.
Built-in `$WM_*` contextual variables in resource values are unchanged.

The variable cache previously wrote with an identity-scoped key but read with the
unscoped key, so it never hit (a latent functional bug that happened to be safe).
Aligning the read path enables the cache and makes it identity-scoped by
construction. Secret variables are cached too, but the entry carries the
`is_secret` flag so a cache hit re-runs the per-read side effects a secret read
performs — the EE `variables.decrypt_secret` audit and running-job secret
registration (factored into `audit_decrypt_secret`, shared by both paths).

The unused `invalidate_{variable,resource}_cache` helpers can no longer target
identity-scoped entries; documented the constraint and refreshed the stale
key-format docs on the cache statics.

Tests:
- integration regression for both caches: a folder-scoped user warms the cache via
  allow_cache=true, then a user without folder access is denied (401) and never
  receives the cached value.
- integration regression that variables (secret included) are served from cache.
- integration regression for job context: plain and non-`$WM_` `$`-string resources
  stay cached and are served under a job_id, while a `$WM_*` resource (warmed without
  a job_id) is not cached.
- unit tests for `auth_identity`.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-03 10:31:45 +00:00
Ruben Fiszelandrubenfiszel 47c96204de chore(main): release 1.715.0 (#9421)
* chore(main): release 1.715.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-03 10:08:16 +00:00
Ruben Fiszelandrubenfiszel 00cd89fff3 chore(main): release 1.714.1 (#9408)
* chore(main): release 1.714.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-02 12:58:41 +00:00
Ruben Fiszelandrubenfiszel 2ac198396e chore(main): release 1.714.0 (#9390)
* chore(main): release 1.714.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-02 08:22:19 +00:00
Ruben Fiszelandrubenfiszel 32b4771f19 chore(main): release 1.713.1 (#9389)
* chore(main): release 1.713.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-06-01 09:10:53 +02:00
Ruben Fiszelandrubenfiszel b16828d480 chore(main): release 1.713.0 (#9369)
* chore(main): release 1.713.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-31 08:43:37 +00:00
Ruben Fiszelandrubenfiszel 889101b7f0 chore(main): release 1.712.0 (#9340)
* chore(main): release 1.712.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-28 19:52:36 +02:00
Ruben Fiszelandrubenfiszel 8d72a7a4a4 chore(main): release 1.711.0 (#9337)
* chore(main): release 1.711.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-26 16:47:53 +00:00
Ruben Fiszelandrubenfiszel edea1b3631 chore(main): release 1.710.1 (#9327)
* chore(main): release 1.710.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-26 09:38:13 +00:00
Ruben Fiszelandrubenfiszel 80f6a5a6e8 chore(main): release 1.710.0 (#9323)
* chore(main): release 1.710.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-26 06:34:55 +00:00
Ruben Fiszel 6f363163df fix(websocket-trigger): honor HTTPS_PROXY/HTTP_PROXY/NO_PROXY (#9324)
* feat(websocket-trigger): honor HTTPS_PROXY/HTTP_PROXY/NO_PROXY (WIN-1988)

`tokio_tungstenite::connect_async` opens a raw TCP socket and ignores
the standard outbound-proxy env vars, so deployments behind a forward
HTTP proxy can't reach the WebSocket endpoint and Test Connection
times out after 30s.

Add a small `proxy` module that resolves the right proxy URL for the
target host (HTTPS_PROXY for wss://, HTTP_PROXY for ws://, NO_PROXY
exclusions, ALL_PROXY fallback, lowercase variants), opens an HTTP
CONNECT tunnel when one applies, and hands the resulting TcpStream to
`client_async_tls_with_config` for the TLS + WS handshake. Direct
connect remains the default when no proxy env is set.

Unit tests cover NO_PROXY matching, proxy URL parsing (including IPv6
literals and basic-auth userinfo), and the CONNECT handshake itself
against an in-process fake proxy (success, basic-auth header, 407
rejection).

Fixes WIN-1988

* refactor(websocket-trigger): reduce blast radius and reuse existing logic

Follow-up to the proxy support change. Three things:

1. Skip the new code path entirely when no proxy is configured.
   `connect_async_with_proxy` now checks the env-var snapshots up front
   and delegates straight to `tokio_tungstenite::connect_async` if
   neither `HTTP_PROXY` nor `HTTPS_PROXY` is set. Same fall-through
   applies when proxy env is set but `NO_PROXY` excludes the host or
   the proxy URL doesn't parse. Non-proxied deployments now exercise
   exactly the previous code path.

2. Move the `NO_PROXY` / `HTTP_PROXY` / `HTTPS_PROXY` env-var snapshots
   from `windmill-worker::worker` into `windmill-common`. The worker's
   `PROXY_ENVS` static now reads from there, and the websocket trigger
   reads from the same source — one place reads the env, one source
   of truth for both call sites.

3. Replace the hand-rolled proxy-URL parser with `url::Url::parse`
   (already a workspace dep, used across the codebase). Half the LoC
   and handles edge cases (userinfo percent-encoding, IPv6 literals,
   path/query stripping) via the well-tested crate instead of by hand.

All 13 proxy unit tests still pass. `cargo check` is clean.

* fix(websocket-trigger): unbreak EE build + trim proxy tests

- Re-export `NO_PROXY` / `HTTP_PROXY` / `HTTPS_PROXY` from
  `windmill-worker::worker` (via `pub use windmill_common::...`) so the
  EE `otel_tracing_proxy_ee` module's `use crate::{HTTPS_PROXY, ...}`
  resolves like it did before. Fixes the `check_ee_full` / `cargo_test`
  CI failures from the previous commit.

- Trim the proxy tests to one un-ignored canary
  (`http_connect_tunnel_sends_well_formed_request_and_unwraps_stream`)
  that exercises the actual on-wire CONNECT handshake plus byte-perfect
  tunnel passthrough. The NO_PROXY-matching, URL-parsing, and edge-case
  tunnel tests are kept under `#[ignore]` for manual debugging
  (`cargo test -- --ignored`) since they're either delegated to
  `url::Url::parse` or trivial string matching — low ROI on every CI run.
2026-05-26 06:17:07 +00:00
Ruben Fiszel 8bf7fd2c92 feat(queue): stochastic admission + EE availability of workspace fairness algorithm (#9321) 2026-05-26 04:51:53 +00:00
Ruben Fiszelandrubenfiszel f9c7fa2e43 chore(main): release 1.709.0 (#9312)
* chore(main): release 1.709.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-25 18:19:53 +00:00
centdix c96be0e903 refactor: move bedrock proxy handling to windmill-ai (#9309)
* refactor: move bedrock proxy handling to windmill-ai

* docs: track ai refactor follow-ups
2026-05-25 14:22:10 +00:00
Ruben Fiszelandrubenfiszel ff685eb2d3 chore(main): release 1.708.0 (#9304)
* chore(main): release 1.708.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-24 23:51:19 +00:00
Ruben Fiszelandrubenfiszel 7b11ebe5f5 chore(main): release 1.707.0 (#9285)
* chore(main): release 1.707.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-22 23:35:01 +02:00
Ruben Fiszelandrubenfiszel 9b218dc405 chore(main): release 1.706.1 (#9281)
* chore(main): release 1.706.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-22 08:26:25 +00:00
Ruben Fiszelandrubenfiszel e6f80dad1c chore(main): release 1.706.0 (#9270)
* chore(main): release 1.706.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-21 16:33:02 +00:00
Ruben Fiszelandrubenfiszel 9cb34397eb chore(main): release 1.705.0 (#9229)
* chore(main): release 1.705.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-20 16:28:29 +00:00
centdix 289549a048 refactor: move google ai proxy handling to windmill-ai (#9260)
* refactor: add ai proxy execution mode

* refactor: move google ai proxy handling

* refactor: share google ai request building
2026-05-20 15:24:07 +00:00
Ruben FiszelandClaude Opus 4.7 aa12c66c25 feat(snowflake): derive public key from private key when omitted (WIN-1959) (#9251)
* feat(snowflake): derive public key from private key when omitted (WIN-1959)

Snowflake key-pair auth needs a SHA256 fingerprint of the public key for
the JWT iss claim, but the public key is mathematically derivable from
the RSA private key. Other tools (e.g. Power BI) only require the
private key, so requiring users to supply both is redundant. When
public_key is missing, fall back to deriving it from private_key (PKCS#8
or PKCS#1 PEM) instead of erroring out.

Fixes WIN-1959

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(snowflake): treat empty public_key/private_key as missing

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 06:48:10 +00:00
Ruben Fiszelandrubenfiszel c8ab030aa4 chore(main): release 1.704.1 (#9226)
* chore(main): release 1.704.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-19 05:46:13 +00:00
Ruben Fiszelandrubenfiszel 11c03ca14e chore(main): release 1.704.0 (#9210)
* chore(main): release 1.704.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-19 00:04:41 +00:00
centdix bd32c5f951 refactor: move openai-compatible proxy building (#9133)
* refactor: introduce ai proxy request types

* refactor: move openai-compatible proxy building
2026-05-18 10:24:16 +00:00
Ruben Fiszelandrubenfiszel 4e91f83b8f chore(main): release 1.703.3 (#9200)
* chore(main): release 1.703.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-18 09:09:50 +00:00
Ruben Fiszelandrubenfiszel 20719b4731 chore(main): release 1.703.2 (#9195)
* chore(main): release 1.703.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-17 12:59:58 +00:00
Ruben Fiszelandrubenfiszel fa090f3081 chore(main): release 1.703.1 (#9182)
* chore(main): release 1.703.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-16 08:42:33 +00:00
Ruben Fiszelandrubenfiszel e3a3dbb89c chore(main): release 1.703.0 (#9170)
* chore(main): release 1.703.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-15 13:15:28 +00:00
Ruben Fiszelandrubenfiszel 7f589a8c7d chore(main): release 1.702.1 (#9166)
* chore(main): release 1.702.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-14 10:54:35 +00:00
Ruben Fiszelandrubenfiszel 90f494975d chore(main): release 1.702.0 (#9160)
* chore(main): release 1.702.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-14 09:18:42 +00:00
Ruben Fiszelandrubenfiszel 4f3a1e3109 chore(main): release 1.701.0 (#9131)
* chore(main): release 1.701.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-13 16:19:55 +00:00
centdix 17cf538a2d refactor: move ai providers to windmill-ai (#9120) 2026-05-12 14:09:31 +00:00
Ruben Fiszelandrubenfiszel c6346aabe0 chore(main): release 1.700.2 (#9117)
* chore(main): release 1.700.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-12 12:32:07 +00:00
Ruben Fiszelandrubenfiszel 411ca47ffd chore(main): release 1.700.1 (#9114)
* chore(main): release 1.700.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-11 23:23:08 +00:00
Ruben Fiszelandrubenfiszel b972fabab4 chore(main): release 1.700.0 (#9092)
* chore(main): release 1.700.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-11 22:37:58 +00:00
Ruben Fiszelandwindmill-internal-app[bot] 0a5f8dcd48 deps: pin tokio-postgres to forked branch with query_typed_raw deadlock fix (#9106)
* deps: pin tokio-postgres to MaterializeInc fork to fix query_typed_raw deadlock

`pg_executor`'s `Client::query_typed_raw` (and `Client::prepare` on the
streaming path) deadlock when the result schema contains a column whose
Oid the client doesn't know yet — citext, custom enums, custom domains,
postgis types. Easy to reproduce against any partitioned table with a
citext column: ~100+ rows is enough on localhost, less on slower links.
`psql` works fine for the same query because the simple-query protocol
doesn't trigger the typeinfo lookup path.

## Root cause (unchanged tokio-postgres bug for years)

`query::query_typed` calls `get_type(client, oid).await` synchronously
while still holding the original query's `Responses` stream. The
original query's `DataRow`s back up in the per-request
`mpsc::channel(1)`, `Connection::poll_read` stops draining the wire,
and the typeinfo sub-query response (queued on the same socket behind
those DataRows) never arrives. Classic head-of-line blocking.

## Fix

Pin `tokio-postgres` / `postgres-types` / `postgres-protocol` (via
[patch.crates-io]) and the workspace `rust-postgres` /
`rust-postgres-native-tls` aliases to the
[MaterializeInc rust-postgres fork at `78c1222577`](https://github.com/MaterializeInc/rust-postgres/tree/master).
MI's [PR #33 "bigger-channels"](https://github.com/MaterializeInc/rust-postgres/pull/33)
(merged 2025-12-11) resized the per-request response channel from
`mpsc::channel(1)` → `mpsc::channel(1024)`. That gives the connection
task 1024 batches of headroom while a streaming consumer is paused
mid-stream — orders of magnitude more than realistic typeinfo
deferral needs (≈3 batches).

## Why MaterializeInc and not a windmill-labs fork

`windmill-trigger-postgres` already depended on the imor fork for the
`postgres-replication` crate (logical replication: `CopyBothDuplex`,
`LogicalReplicationStream`, `TupleData` decoding including binary
tuples). That crate has never been on upstream rust-postgres — petrosagg's
[PR #752](https://github.com/rust-postgres/rust-postgres/pull/752) was
closed in 2021 in favour of a smaller split,
[PR #778](https://github.com/rust-postgres/rust-postgres/pull/778) is
still open today after five years. petrosagg keeps the replication work
alive on the MaterializeInc fork.

MaterializeInc is a strict superset of what we previously got from imor:
- imor's binary-tuple commit (sha `20265ef38e`) was merged into MI master.
- petrosagg has added perf + correctness fixes on top (allocation reuse,
  proper decoding fixes).
- The deadlock mitigation (`channel(1024)`) was added three weeks before
  this issue surfaced.

MI tracks upstream rust-postgres with a periodic catch-up merge (12-18 mo
cadence; last on 2025-12-03, ~100 commits picked up). Not an abandoned
fork.

## Why this works now (didn't on earlier attempt)

A previous attempt at this PR (`248ccb5a97`) hit CI failure because the
MI fork's `postgres-types 0.2.11` requires `serde_core ^1.0.221`, but
Windmill's workspace pinned `serde = "=1.0.220"` for swc_common 0.37.5's
`pub use serde::__private as serde;` hack. Bumping serde above 1.0.220
broke the swc_ecma_ast `Deserialize` derive under the
`enterprise,deno_core,…` feature set.

The earlier blocker is now resolved by #9111 which bumped the deno +
swc pin set to a "goldilocks" combination where `swc_common 14.0.4`
drops the `__private` hack, freeing the workspace serde pin to `^1`.
serde now resolves to 1.0.228, which satisfies MI's `serde_core ^1.0.221`
requirement transitively — no extra workspace pin needed.

## Diff shape

Two files only:
- `backend/Cargo.toml` (+34/-1): three new `[patch.crates-io]` entries
  (`tokio-postgres`, `postgres-types`, `postgres-protocol` → MI fork)
  plus comment block, plus the two workspace deps (`rust-postgres` /
  `rust-postgres-native-tls`) repointed from imor's fork to MI's.
- `backend/Cargo.lock` — auto-regenerated. Replaces all `imor/rust-postgres`
  references with `MaterializeInc/rust-postgres`, bumps the affected
  crate versions to MI's set (tokio-postgres 0.7.11 → 0.7.15,
  postgres-types 0.2.7 → 0.2.11, postgres-protocol 0.6.7 → 0.6.9,
  postgres-native-tls 0.5.0 → 0.5.2). No source code changes.

## Verification

- `cargo check --features quickjs` → clean.
- `cargo check -p windmill-worker --features quickjs` → clean (pg_executor
  builds).
- Repro tested earlier in the thread that produced this PR: the
  partitioned-citext-table query on Neon goes from "hangs indefinitely"
  (server idle on `wait_event=ClientRead` while client awaits typeinfo
  behind undrained DataRows) to "completes in ~1.0s, 100 rows" with the
  MI fork's `bounded(1024)` response channel.

## Caveats

- **`bounded(1024)` is a mitigation, not a closure.** Theoretical failure
  mode remains at >~64 MB single-query results with a custom-Oid column
  (typeinfo defers for >1024 batches of ~64 KB each). The strict-correct
  fix is `mpsc::unbounded()` — proposed as a follow-up PR to MI. For
  realistic Windmill workloads, 1024 batches of headroom is well past
  the ~3-batch typeinfo deferral that's actually needed.
- **`postgres-replication` is now upstream-of-fork's only home.** No
  realistic path to upstream rust-postgres merging it. The MI pin is
  intended to stay in place until either upstream changes course
  (unlikely) or MI publishes to crates.io (also unlikely — they don't
  publish releases of the fork).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

* chore: update ee-repo-ref to 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c

This commit updates the EE repository reference after PR #568 was merged in windmill-ee-private.

Previous ee-repo-ref: 4d01d171228196f28ddabc1150242bfab623d5cf

New ee-repo-ref: 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 21:43:18 +00:00
Ruben Fiszel 3cd0eac8c1 deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling (#9111)
* deps: bump deno_core / deno_ast / swc to the goldilocks pin set; drop serde ceiling

Bumps every deno_* and swc_* workspace dep to a hand-picked "goldilocks"
combination that drops the serde =1.0.220 ceiling without crashing into
the rustls / aws-sdk resolver wall that the obvious deno v2.6.0 target
hits.

## What's the goldilocks set

| crate            | old      | new       | source                                |
|------------------|----------|-----------|---------------------------------------|
| deno_core        | 0.336.0  | 0.352.0   | deno v2.4.0                           |
| deno_fetch       | 0.214.0  | 0.233.0   | deno v2.4.0                           |
| deno_tls         | 0.177.0  | 0.196.0   | deno v2.4.0 (last permissive-rustls)  |
| deno_console     | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_url         | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_webidl      | 0.190.0  | 0.209.0   | deno v2.4.0                           |
| deno_web         | 0.221.0  | 0.240.0   | deno v2.4.0                           |
| deno_io          | 0.100.0  | 0.119.0   | deno v2.4.0                           |
| deno_net         | 0.182.0  | 0.201.0   | deno v2.4.0                           |
| deno_permissions | 0.49.0   | 0.68.0    | deno v2.4.0                           |
| deno_telemetry   | 0.12.0   | 0.31.0    | deno v2.4.0                           |
| deno_error       | =0.5.5   | =0.6.1    | deno v2.4.0                           |
| deno_ast         | =0.44.0  | =0.51.0   | **override** — see "load-bearing" below |
| deno_fs          | (new)    | 0.119.0   | new workspace dep — FetchPermissions exposes deno_fs::CheckedPath / GetPath as public API |
| v8               | =130.0.7 | =137.1.0  | deno_core 0.352 transitive            |
| swc_common       | =0.37.5  | =14.0.4   | **the load-bearing pin**              |
| swc_ecma_ast     | =0.118.2 | =15.0.0   | matched set with swc_common 14.0.4    |
| swc_ecma_parser  | =0.149.1 | =24.0.3   | matched set                           |
| swc_ecma_visit   | =0.104.8 | =15.0.0   | matched set                           |
| serde            | =1.0.220 | ^1        | **freed** (resolves to 1.0.228+)      |

## Why this combination and not v2.6.0

The obvious target was deno v2.6.0 (with deno_ast 0.52 → swc_common 17,
well past the `__private` ceiling). That hits three resolver collisions:

1. libsqlite3-sys: deno_cache → rusqlite 0.37 → libsqlite3-sys 0.35
   vs sqlx → libsqlite3-sys 0.30. **Already killed by PR #9110** —
   we dropped deno_runtime, which is what pulled in deno_cache.
2. fqdn 0.4.6/0.4.7 yanked, required by deno_permissions 0.81.0. Solvable
   by injecting the yanked entry into Cargo.lock manually but ugly.
3. rustls: deno_tls 0.198+ hard-pins `=0.23.28`, but aws-sdk-bedrockruntime
   1.122.0 → aws-smithy-http-client 1.1.5 wants `^0.23.31`. Within-major
   conflict, no resolver path. The unbeatable wall.

Goldilocks-set choice sidesteps (2) and (3) entirely:

- `deno_tls 0.196.0` was the last version before deno tightened
  `rustls ^0.23.11` (range, accepts 0.23.31) to exact `=0.23.28`. With
  ^0.23.11, the resolver picks rustls 0.23.35 (latest 0.23 patch) which
  satisfies both deno_tls's `>=0.23.11` and aws-sdk's `>=0.23.31`. Verified
  empirically: lockfile has rustls 0.23.35 after this bump.
- `deno_permissions 0.68.0` (v2.4.0's pin) doesn't depend on fqdn at all.
  The fqdn dep was added in a later deno_permissions release.

## Why deno_ast =0.51.0 specifically (not 0.48.0 from v2.4.0)

`swc_common 14.0.4` is the first patch that **drops the
`pub use serde::__private as serde;` line** in `src/private/mod.rs`. Older
14.0.x and all 0.37.5–13.x revisions still have it, and that line is
what was capping `serde = "=1.0.220"` (the workspace pin's "stuck because
of swc" comment). Empirically verified by inspecting the tarballs of
14.0.0 / 14.0.1 / 14.0.2 / 14.0.3 / 14.0.4:

    14.0.0: has hack
    14.0.1: has hack
    14.0.2: has hack
    14.0.3: has hack
    14.0.4: NO HACK    ← inflection point

`deno_ast 0.51.0` pins `swc_common =14.0.4` exactly — older deno_ast
versions pin earlier swc_common patches that still have the hack.
Notably, deno v2.4.0 itself pins `deno_ast =0.48.0` (swc_common 9.2.0,
still has hack) — we deliberately deviate from v2.4.0's deno_ast pin
to escape the swc serde wall, while keeping the rest of v2.4.0's pin
set for resolver compatibility with aws-sdk. deno_ast 0.51 was never
shipped in any deno release (v2.4.5 used 0.49, v2.5.0 jumped to 0.50,
v2.6.0 to 0.52), but it's published on crates.io and compatible with
v2.4.0's deno_core 0.352.

## What this unblocks

- PR #9106's `serde = "=1.0.224"` bump variant can rebase onto this
  and resolve cleanly (MaterializeInc/rust-postgres' `postgres-types`
  needs `serde_core ^1.0.221`, which is satisfied now that we're on
  serde 1.0.228).
- Future deno_* / swc_* bumps no longer need to argue about the serde
  ceiling — it's gone.

## What changes in source code

This commit is Cargo.toml + Cargo.lock only. Source changes that the
new deno_core / deno_fetch API requires live in the follow-up commits:

- `parsers/windmill-parser-{ts,ts-asset,wac}`: swc 0.37 → 14
  (`code.into()` ambiguity fix at 5 sites)
- `windmill-runtime-nativets/build.rs` + `src/lib.rs`: deno_core 0.336
  → 0.352 API moves (`init_ops_and_esm()` → `init()`,
  `FetchPermissions` / `NetPermissions` trait signature updates,
  `deno_tls::Proxy` enum shape change)

A companion change in windmill-ee-private adjusts
`otel_tracing_proxy_ee.rs:521` for `deno_telemetry::init`'s second arg
becoming by-value (was `&OtelConfig`).

* fix(parsers): adapt to swc_common 14 BytesStr ambiguity

swc 0.37.5 → 14.0.4 changed `SourceMap::new_source_file`'s `src` argument
from `String` to `impl Into<BytesStr>`. With `BytesStr` available, the
existing call sites' `code.into()` on a `&str` becomes ambiguous between
`Into<Bytes>` (from the bytes crate) and `Into<BytesStr>` (from
bytes_str). Switch to `code.to_string()` to produce an owned `String`
that satisfies `From<String> for BytesStr` unambiguously.

Five call sites across three crates:
- windmill-parser-ts/src/lib.rs (3 sites)
- windmill-parser-ts-asset/src/lib.rs (1 site)
- windmill-parser-wac/src/typescript.rs (1 site)

* fix(nativets): adapt to deno_core 0.352 / deno_fetch 0.233 API changes

The goldilocks deno bump (deno_core 0.336 → 0.352, deno_fetch 0.214 →
0.233, etc.) ripples through nativets' build.rs and src/lib.rs.
Source-level changes required:

## 1. `extension!` macro: `init_ops_and_esm()` and `init_ops()` removed

deno_core 0.352's `extension!` macro now generates a single `init()`
function on the extension struct (full: ops + esm), plus `lazy_init()`
(ops only, with `needs_lazy_init = true` and a contract that the
caller invokes `JsRuntime::lazy_init_extensions` after construction).

- `build.rs` (snapshot creation, wants both ops and esm baked in):
  `X::init_ops_and_esm(...)` → `X::init(...)`.
- `src/lib.rs:create_nativets_runtime` (runtime, was using `init_ops()`
  because the snapshot already provides esm): also → `X::init(...)`.
  deno_core's snapshot path skips esm re-execution when the snapshot
  provides them, so the esm registration is a no-op at runtime. This
  is how deno's own v2.4.0 runtime works.

Avoided `lazy_init` because it requires plumbing
`JsRuntime::lazy_init_extensions(ext_args_vec)` correctly across the
codebase, which is invasive for no behavioural benefit.

## 2. Local `fetch` extension now declared in both build.rs and lib.rs

deno_core 0.352 validates extension order between snapshot and runtime.
Our snapshot's last extension is the local `fetch` ext (which provides
ext:fetch/src/runtime.js). To avoid a runtime panic:

    "Extensions from snapshot loaded in wrong order: expected fetch but got windmill"

…the runtime extension list now ends with `fetch::init()` matching the
snapshot order. The macro requires the same `esm` argument to type-check,
even though the ESM is not re-executed at runtime (it's in the snapshot).

## 3. `FetchPermissions` and `NetPermissions` trait shape

`deno_fetch::FetchPermissions` (deno_fetch 0.233.0) added new methods
and changed signatures:

- `check_read` / `check_write`: now take `path: Cow<'a, Path>` plus
  a new `get_path: &'a dyn deno_fs::GetPath` parameter, and return
  `Result<deno_fs::CheckedPath<'a>, FsError>` instead of `Result<Cow<Path>, FsError>`.
- New `check_write` (didn't exist) and `check_net_vsock` methods.

`deno_net::NetPermissions` (deno_net 0.201.0) gained `check_vsock`
and `check_write_path` now takes `Cow<'_, Path>`.

For `build.rs`'s `PermissionsContainer` (used only during snapshot
creation, where permissions are never actually checked): all methods
`unreachable!("snapshotting")`.

For `src/lib.rs`'s `PermissionsContainer` (used at runtime — the
nativets policy is "allow everything"): `check_read` / `check_write`
return `Ok(CheckedPath::Unresolved(path))`, `check_*_vsock` return
`Ok(())`. Smoke tests confirm fetch/net/url/web/blob/timers/structuredClone
behaviour is intact end-to-end.

## 4. `deno_tls::Proxy` is now an enum

`deno_tls::Proxy` was a struct, is now an enum with `Http`, `Https`,
`Socks5` variants. Our call site uses HTTP proxies — switched the
struct literal `deno_tls::Proxy { url, basic_auth }` to
`deno_tls::Proxy::Http { url, basic_auth }`.

## 5. New `deno_fs` direct workspace dep

`FetchPermissions` exposes `deno_fs::CheckedPath` and `deno_fs::GetPath`
as part of its public API. We can't avoid naming `deno_fs` directly any
more. Pinned to 0.119.0 (v2.4.0's matched version, transitively present
already through deno_fetch). Added to workspace `[dependencies]` plus
nativets's `[dependencies]` and `[build-dependencies]`.

## Validation

`cargo check --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline`
→ clean.

`cargo test -p windmill-runtime-nativets smoke -- --ignored --skip smoke_net_`
→ 8 passed; 0 failed (the full local smoke suite covering fetch,
setTimeout/Promise.all, URL/SearchParams, Blob/btoa/atob, large payload
roundtrip, error propagation, concurrent isolates, TS enum/union
transpile).

Network smoke tests (`smoke_net_fetch_example_com`,
`smoke_net_fetch_json_and_headers`) not run as part of the validation
gate but expected to pass — the change preserves deno_fetch behaviour
through the trait surface.

* chore: update ee-repo-ref to pick up deno_telemetry::init by-value fix

Points at windmill-ee-private branch deps/bump-deno-and-swc-goldilocks
which contains the companion otel_tracing_proxy_ee.rs adjustment for
deno_telemetry 0.12 → 0.31 (second arg of `init` is now by-value).
EE-only file, doesn't affect OSS build.

* chore(nix): bump rusty_v8 in flake.nix to 137.1.0 to match Cargo.toml

Cargo.toml's v8 pin moved from =130.0.7 to =137.1.0 as part of the
deno_core 0.336 → 0.352 bump, but I missed the comment directly above
the version pin:

    # Exact version NOTE: Do not forget to update version and hash in flake.nix

flake.nix provides the prebuilt librusty_v8 binary that the v8 crate
links against. A version mismatch would either fail to fetch (if the
137.1.0 release didn't exist) or cause link-time symbol mismatches.
Nix is used by rust-client-check.yml and rust_on_release.yml in CI,
plus the dev shell — stale flake pin breaks all of those.

Updates x86_64-linux's sha256 to match the actual hash of
librusty_v8_release_x86_64-unknown-linux-gnu.a.gz at the 137.1.0 tag.
Other targets (aarch64-linux, x86_64-darwin, aarch64-darwin) remain
as lib.fakeHash — they were already placeholders in the previous
pin, so we don't regress on them.

Caught by both cubic and Pi reviewers on PR #9111.

* docs(nativets): clarify snapshot-prefix rule in extension-order comment

Claude reviewer caught that the doc comment claimed the runtime
extension list matches the snapshot's order — implying an exact match.
The truth is more permissive: deno_core 0.352 requires the snapshot's
extension list to be a *prefix* of the runtime's, not an exact match.
Runtime is allowed to append extra extensions (which we do — the
windmill `ext` carrying our ops is the last entry at runtime but absent
from the snapshot).

The code is correct as-is; only the comment wording was misleading.

Also fixes the same wording in PR description.
2026-05-11 21:13:24 +00:00
Ruben Fiszel 36b316d9e8 deps(nativets): inline maybe_transpile_source, drop deno_runtime (#9110)
windmill-runtime-nativets was the workspace's only consumer of the
deno_runtime crate, and its only use of it was one call site in
build.rs:

    deno_runtime::transpile::maybe_transpile_source(specifier, source)

That function (`deno_runtime-0.198.0/transpile.rs`, ~80 lines) is a pure
deno_ast + deno_core + deno_error wrapper — it doesn't touch any
deno_runtime state. Inline it verbatim into our build.rs and drop the
entire deno_runtime dep.

Why this matters now: deno_runtime transitively pulls in deno_cache →
rusqlite → libsqlite3-sys. From deno_cache 0.128.0 (Feb-Mar 2025)
onwards, rusqlite was bumped to ^0.34, which means libsqlite3-sys ^0.35.
sqlx 0.8 transitively requires libsqlite3-sys ^0.30. Cargo's `links =
"sqlite3"` rule allows only one libsqlite3-sys in a build graph, so the
two crates collide on any deno release ≥ v2.5. Inlining the transpile
helper sidesteps the collision entirely — sqlx-sqlite stays the sole
libsqlite3-sys consumer at 0.30.1.

All other appearances of "deno_runtime" in the source tree are for a
Windmill-internal function named `setup_deno_runtime`, not the crate.

Build artifacts validated:
- `cargo check --features quickjs` → green.
- `cargo test -p windmill-runtime-nativets smoke -- --ignored --skip smoke_net_`
  → 8 passed (the in-process V8 runtime + deno_fetch + deno_web + swc
  transpilation surface still works end-to-end through the inlined
  function).
- `cargo tree --invert deno_cache` → "did not match any packages"
  (gone from the graph).
- Single `libsqlite3-sys` entry in Cargo.lock at 0.30.1 (sqlx's).
2026-05-11 20:21:46 +00:00
centdix 6f7d31e56b refactor: move ai image handling to windmill-ai (#9098) 2026-05-11 14:59:40 +00:00
centdixandwindmill-internal-app[bot] 27acbbf3d5 refactor: move ai sse plumbing to windmill-ai (#9059)
* docs: refine windmill ai refactor plan

* refactor: move ai sse plumbing to windmill-ai

* refactor: remove ai re-export shims

* fix: update ee ai memory ref

* chore: update ee-repo-ref to d3bc7fa85195b46b7a38d43c2f806520bf8b5454

This commit updates the EE repository reference after PR #560 was merged in windmill-ee-private.

Previous ee-repo-ref: ff35bf7cc198e13884b33654e1d6dbd8a8b314d3

New ee-repo-ref: d3bc7fa85195b46b7a38d43c2f806520bf8b5454

Automated by sync-ee-ref workflow.

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-05-11 10:01:45 +00:00
Ruben Fiszelandrubenfiszel 91ddb930c3 chore(main): release 1.699.0 (#9082)
* chore(main): release 1.699.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-05-08 18:03:06 +00:00