mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-19 16:02:14 +00:00
Ruben Fiszel
75bafabeee
* perf(monitor): hash active-root exclusion in retention delete The expired-job retention delete (delete_expired_jobs_batch) excluded jobs belonging to still-active root flows with `COALESCE(j.root_job, j.flow_innermost_root_job, jc.id) != ALL($3)`. That ScalarArrayOp is evaluated per candidate row as a linear scan of $3, so cost grows with the number of active root jobs. Express the exclusion as `NOT IN (SELECT u FROM unnest($3) u WHERE u IS NOT NULL)` instead. The subquery form lets Postgres build a one-time hashed SubPlan and apply it as a filter on the ordered index scan, giving O(1) membership per candidate while preserving the `ORDER BY completed_at ASC LIMIT` early termination. The `u IS NOT NULL` guard sidesteps NOT IN's null-trap semantics ($3 holds non-null PK ids). Measured on a 2M-row synthetic v2_job_completed (batch LIMIT 20000, 5-run min): active roots | != ALL (before) | NOT IN hashed (after) -------------|-----------------|---------------------- 100 | 108 ms | 104 ms 1000 | 168 ms | 105 ms 10000 | 719 ms | 131 ms Both forms return identical row sets (verified via EXCEPT, 0 diff). Neutral at small active-root counts, ~5.5x faster when many flows are active. Relates to WIN-2088 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf(monitor): apply hashed active-root exclusion to log_cleanup mirror windmill-api-settings/log_cleanup.rs::delete_expired_jobs_batch carries a byte-identical copy of the retention delete and shared its prepared-query cache. Updating only monitor.rs removed that shared cache entry and broke the SQLX_OFFLINE build of the mirror. Apply the same NOT IN (hashed SubPlan) rewrite so both copies converge on one cached query and the mirror gets the same speedup. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>