mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-21 08:03:33 +00:00
Ruben Fiszelandwindmill-internal-app[bot]
0a5f8dcd48
* deps: pin tokio-postgres to MaterializeInc fork to fix query_typed_raw deadlock `pg_executor`'s `Client::query_typed_raw` (and `Client::prepare` on the streaming path) deadlock when the result schema contains a column whose Oid the client doesn't know yet — citext, custom enums, custom domains, postgis types. Easy to reproduce against any partitioned table with a citext column: ~100+ rows is enough on localhost, less on slower links. `psql` works fine for the same query because the simple-query protocol doesn't trigger the typeinfo lookup path. ## Root cause (unchanged tokio-postgres bug for years) `query::query_typed` calls `get_type(client, oid).await` synchronously while still holding the original query's `Responses` stream. The original query's `DataRow`s back up in the per-request `mpsc::channel(1)`, `Connection::poll_read` stops draining the wire, and the typeinfo sub-query response (queued on the same socket behind those DataRows) never arrives. Classic head-of-line blocking. ## Fix Pin `tokio-postgres` / `postgres-types` / `postgres-protocol` (via [patch.crates-io]) and the workspace `rust-postgres` / `rust-postgres-native-tls` aliases to the [MaterializeInc rust-postgres fork at `78c1222577`](https://github.com/MaterializeInc/rust-postgres/tree/master). MI's [PR #33 "bigger-channels"](https://github.com/MaterializeInc/rust-postgres/pull/33) (merged 2025-12-11) resized the per-request response channel from `mpsc::channel(1)` → `mpsc::channel(1024)`. That gives the connection task 1024 batches of headroom while a streaming consumer is paused mid-stream — orders of magnitude more than realistic typeinfo deferral needs (≈3 batches). ## Why MaterializeInc and not a windmill-labs fork `windmill-trigger-postgres` already depended on the imor fork for the `postgres-replication` crate (logical replication: `CopyBothDuplex`, `LogicalReplicationStream`, `TupleData` decoding including binary tuples). That crate has never been on upstream rust-postgres — petrosagg's [PR #752](https://github.com/rust-postgres/rust-postgres/pull/752) was closed in 2021 in favour of a smaller split, [PR #778](https://github.com/rust-postgres/rust-postgres/pull/778) is still open today after five years. petrosagg keeps the replication work alive on the MaterializeInc fork. MaterializeInc is a strict superset of what we previously got from imor: - imor's binary-tuple commit (sha `20265ef38e`) was merged into MI master. - petrosagg has added perf + correctness fixes on top (allocation reuse, proper decoding fixes). - The deadlock mitigation (`channel(1024)`) was added three weeks before this issue surfaced. MI tracks upstream rust-postgres with a periodic catch-up merge (12-18 mo cadence; last on 2025-12-03, ~100 commits picked up). Not an abandoned fork. ## Why this works now (didn't on earlier attempt) A previous attempt at this PR (`248ccb5a97`) hit CI failure because the MI fork's `postgres-types 0.2.11` requires `serde_core ^1.0.221`, but Windmill's workspace pinned `serde = "=1.0.220"` for swc_common 0.37.5's `pub use serde::__private as serde;` hack. Bumping serde above 1.0.220 broke the swc_ecma_ast `Deserialize` derive under the `enterprise,deno_core,…` feature set. The earlier blocker is now resolved by #9111 which bumped the deno + swc pin set to a "goldilocks" combination where `swc_common 14.0.4` drops the `__private` hack, freeing the workspace serde pin to `^1`. serde now resolves to 1.0.228, which satisfies MI's `serde_core ^1.0.221` requirement transitively — no extra workspace pin needed. ## Diff shape Two files only: - `backend/Cargo.toml` (+34/-1): three new `[patch.crates-io]` entries (`tokio-postgres`, `postgres-types`, `postgres-protocol` → MI fork) plus comment block, plus the two workspace deps (`rust-postgres` / `rust-postgres-native-tls`) repointed from imor's fork to MI's. - `backend/Cargo.lock` — auto-regenerated. Replaces all `imor/rust-postgres` references with `MaterializeInc/rust-postgres`, bumps the affected crate versions to MI's set (tokio-postgres 0.7.11 → 0.7.15, postgres-types 0.2.7 → 0.2.11, postgres-protocol 0.6.7 → 0.6.9, postgres-native-tls 0.5.0 → 0.5.2). No source code changes. ## Verification - `cargo check --features quickjs` → clean. - `cargo check -p windmill-worker --features quickjs` → clean (pg_executor builds). - Repro tested earlier in the thread that produced this PR: the partitioned-citext-table query on Neon goes from "hangs indefinitely" (server idle on `wait_event=ClientRead` while client awaits typeinfo behind undrained DataRows) to "completes in ~1.0s, 100 rows" with the MI fork's `bounded(1024)` response channel. ## Caveats - **`bounded(1024)` is a mitigation, not a closure.** Theoretical failure mode remains at >~64 MB single-query results with a custom-Oid column (typeinfo defers for >1024 batches of ~64 KB each). The strict-correct fix is `mpsc::unbounded()` — proposed as a follow-up PR to MI. For realistic Windmill workloads, 1024 batches of headroom is well past the ~3-batch typeinfo deferral that's actually needed. - **`postgres-replication` is now upstream-of-fork's only home.** No realistic path to upstream rust-postgres merging it. The MI pin is intended to stay in place until either upstream changes course (unlikely) or MI publishes to crates.io (also unlikely — they don't publish releases of the fork). 🤖 Generated with [Claude Code](https://claude.com/claude-code) * chore: update ee-repo-ref to 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c This commit updates the EE repository reference after PR #568 was merged in windmill-ee-private. Previous ee-repo-ref: 4d01d171228196f28ddabc1150242bfab623d5cf New ee-repo-ref: 8fe0d290fb0b71c24184eb5ad99bbdc7c813697c Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Windmill Backend
This folder holds all backend components, the src/ folder only contains files used to build the "root" binary.
Components
| name | description |
|---|---|
| windmill-api | The API server, exposing functionality to other components and the frontend |
| windmill-audit | Contains audit functionality, allowing different components to record important actions |
| windmill-common | Common code shared by all crates |
| windmill-queue | Contains job & flow queuing functionality, commonly written to by the API server and read from by workers |
| windmill-worker | The worker. Used to process and execute flows & jobs. |
| parsers | Contains code to parse signatures in different langauges. |
Compile sqlx for offline ci
cargo sqlx prepare --workspace -- --bin windmill --features enterprise