Konstantin Knizhnik 412e0aa985 Skip largest N holes during compaction (#3597) ...

## Describe your changes

This is yet another attempt to address problem with storage size
ballooning #2948
Previous PR #3348 tries to address this problem by maintaining list of
holes for each layer.
The problem with this approach is that we have to load all layer on
pageserver start.
Lazy loading of layers is not possible any more.

This PR tries to collect information of N largest holes on compaction
time and exclude this holes from produced layers.
It can cause generation of larger number of layers (up to 2 times) and
producing small layers.
But it requires minimal changes in code and doesn't affect storage
format.

For graphical explanation please see thread:
https://github.com/neondatabase/neon/pull/3597#discussion_r1112704451

## Issue ticket number and link

#2948
#3348

## Checklist before requesting a review
- [ ] I have performed a self-review of my code.
- [ ] If it is a core feature, I have added thorough tests.
- [ ] Do we need to implement analytics? if so did you add the relevant
metrics to the dashboard?
- [ ] If this PR requires public announcement, mark it with
/release-notes label and add several sentences in this section.

2023-02-22 18:28:01 +02:00

..

benches

feat: LayerMap::replace (#3513)

2023-02-03 15:33:46 +02:00

fixtures

walredo: simple tests and bench updates (#3045)

2023-01-16 18:24:45 +02:00

src

Skip largest N holes during compaction (#3597)

2023-02-22 18:28:01 +02:00

Cargo.toml

automatic layer eviction

2023-02-09 13:33:55 +01:00