pageserver: do all L0 compaction before image compaction (#10744)

## Problem

Image compaction can starve out L0 compaction if a tenant has several
timelines with L0 debt.

Touches #10694.
Requires #10740.

## Summary of changes

* Add an initial L0 compaction pass, in order of L0 count.
* Add a tenant option `compaction_l0_first` to control the L0 pass
(disabled by default).
* Add `CompactFlags::OnlyL0Compaction` to run an L0-only compaction
pass.
* Clean up the compaction iteration logic.

A later PR will use separate semaphores for the L0 and image compaction
passes to avoid cross-tenant L0 starvation. That PR will also make image
compaction yield if _any_ of the tenant's timelines have pending L0
compaction to further avoid starvation.
This commit is contained in:
Erik Grinaker
2025-02-11 22:08:46 +00:00
committed by GitHub
parent 635b67508b
commit 6c83ac3fd2
8 changed files with 209 additions and 126 deletions
@@ -140,9 +140,10 @@ def test_fully_custom_config(positive_env: NeonEnv):
"compaction_period": "1h",
"compaction_threshold": 13,
"compaction_upper_limit": 100,
"compaction_l0_first": False,
"l0_flush_delay_threshold": 25,
"l0_flush_stall_threshold": 42,
"l0_flush_wait_upload": True,
"l0_flush_wait_upload": False,
"compaction_target_size": 1048576,
"checkpoint_distance": 10000,
"checkpoint_timeout": "13m",