Files
windmill/backend/windmill-queue
Ruben Fiszel 0dbd9c1231 perf: eliminate dual-connection DB pool contention across worker, queue, and api (#9798)
* perf: eliminate dual-connection DB pool contention across worker, queue, and api

Reuse the held transaction (or move pool reads before begin()) instead of
checking out a second pool connection while a tx is open, extending the
fix from #9789/#7861. Targets the per-worker pool (max 5) hot paths plus
several server-pool API handlers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: pass owned pool to get_email_from_permissioned_as in http trigger handler

The generified signature takes impl PgExecutor; the http trigger handler
passed &db where db is already &DB, yielding &&Pool which does not impl
PgExecutor (only surfaced under the full feature set in CI).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: keep RLS-exposed reads on the non-RLS pool and isolate flow-eval reads in a savepoint

Addresses review of the dual-connection sweep:

- worker_flow: wrap the stop_after_all_iters_if reads in a SAVEPOINT. The
  caller swallows the error and keeps using tx, so a DB read failure must
  not leave the outer transaction aborted (it would fail the later commit).
  Matches the previous pool-read semantics.

- Revert reads that were moved onto an RLS (user_db) transaction back to the
  non-RLS pool, since RLS row-visibility/role context can change results:
  push_scheduled_job (email/tag/settings lookups; reachable with a user_db
  tx from api-schedule/api-flows), push_inner native-retry dedicated_worker
  routing (RLS isolation variants), resources.rs app-namespace folder
  auto-create (non-admins must not be blocked), and the script archive/delete
  UPDATEs. Non-RLS db.begin() reuse and move-before-begin are kept.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: failpoint proving the stop_after_all_iters_if savepoint isolates an aborted read

Adds a worker-crate failpoints feature and a data-driven hook: when the
stop_after_all_iters_if expr is the magic sentinel, the in-evaluation read runs
SELECT 1/0 to abort its (savepoint) transaction. The test asserts the flow still
completes (iteration marked failed) — which only holds if the savepoint keeps the
outer status-update transaction committable. Without the savepoint the abort would
poison the outer tx and the job would never complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 21:33:17 +00:00
..