From 499d0707d25f653175a51902de75513af3f7c659 Mon Sep 17 00:00:00 2001 From: Konstantin Knizhnik Date: Fri, 8 Sep 2023 18:05:08 +0300 Subject: [PATCH] Perform throttling for concurrent build index which is done outside transaction (#5048) See https://neondb.slack.com/archives/C03H1K0PGKH/p1692550646191429 ## Problem Build index concurrently is writing WAL outside transaction. `backpressure_throttling_impl` doesn't perform throttling for read-only transactions (not assigned XID). It cause huge write lag which can cause large delay of accessing the table. ## Summary of changes Looks at `PROC_IN_SAFE_IC` in process state set during concurrent index build. ## Checklist before requesting a review - [ ] I have performed a self-review of my code. - [ ] If it is a core feature, I have added thorough tests. - [ ] Do we need to implement analytics? if so did you add the relevant metrics to the dashboard? - [ ] If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section. ## Checklist before merging - [ ] Do not forget to reformat commit message to not include the above checklist --------- Co-authored-by: Konstantin Knizhnik Co-authored-by: Heikki Linnakangas --- pgxn/neon/walproposer.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/pgxn/neon/walproposer.c b/pgxn/neon/walproposer.c index d9999ef2b1..7ca335945d 100644 --- a/pgxn/neon/walproposer.c +++ b/pgxn/neon/walproposer.c @@ -2539,8 +2539,15 @@ backpressure_throttling_impl(void) ? PrevProcessInterruptsCallback() : false; - /* Don't throttle read only transactions and wal sender. */ - if (am_walsender || !TransactionIdIsValid(GetCurrentTransactionIdIfAny())) + /* + * Don't throttle read only transactions or wal sender. + * Do throttle CREATE INDEX CONCURRENTLY, however. It performs some + * stages outside a transaction, even though it writes a lot of WAL. + * Check PROC_IN_SAFE_IC flag to cover that case. + */ + if (am_walsender + || (!(MyProc->statusFlags & PROC_IN_SAFE_IC) + && !TransactionIdIsValid(GetCurrentTransactionIdIfAny()))) return retry; /* Calculate replicas lag */