pageserver: add separate semaphore for L0 compaction (#10780)

## Problem

L0 compaction frequently gets starved out by other background tasks and
image/GC compaction. L0 compaction must be responsive to keep read
amplification under control.

Touches #10694.
Resolves #10689.

## Summary of changes

Use a separate semaphore for the L0-only compaction pass.

* Add a `CONCURRENT_L0_COMPACTION_TASKS` semaphore and
`BackgroundLoopKind::L0Compaction`.
* Add a setting `compaction_l0_semaphore` (default off via
`compaction_l0_first`).
* Use the L0 semaphore when doing an `OnlyL0Compaction` pass.
* Use the background semaphore when doing a regular compaction pass
(which includes an initial L0 pass).
* While waiting for the background semaphore, yield for L0 compaction if
triggered.
* Add `CompactFlags::NoYield` to disable L0 yielding, and set it for the
HTTP API route.
* Remove the old `use_compaction_semaphore` setting and
compaction-scoped semaphore.
* Remove the warning when waiting for a semaphore; it's noisy and we
have metrics.
This commit is contained in:
Erik Grinaker
2025-02-12 17:12:21 +01:00
committed by GitHub
parent ec354884ea
commit f62047ae97
12 changed files with 110 additions and 67 deletions

View File

@@ -141,6 +141,7 @@ def test_fully_custom_config(positive_env: NeonEnv):
"compaction_threshold": 13,
"compaction_upper_limit": 100,
"compaction_l0_first": False,
"compaction_l0_semaphore": False,
"l0_flush_delay_threshold": 25,
"l0_flush_stall_threshold": 42,
"l0_flush_wait_upload": False,