Alex Chi Z 7701ca45dd feat(pageserver): generate image layers for sparse keyspace (#7567) ...

Part of https://github.com/neondatabase/neon/issues/7462

Sparse keyspace does not generate image layers for now. This pull
request adds support for generating image layers for sparse keyspace.


## Summary of changes

* Use the scan interface to generate compaction data for sparse
keyspace.
* Track num of delta layers reads during scan.
* Read-trigger compaction: when a scan on the keyspace touches too many
delta files, generate an image layer. There are one hard-coded threshold
for now: max delta layers we want to touch for a scan.
* L0 compaction does not need to compute holes for metadata keyspace.

Know issue: the scan interface currently reads past the image layer,
which causes `delta_layer_accessed` keeps increasing even if image
layers are generated. The pull request to fix that will be separate, and
orthogonal to this one.

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>

2024-05-20 16:08:45 +00:00

..

benches

chore!: always use async walredo, warn if sync is configured (#7754)

2024-05-15 15:04:52 +02:00

client

feat(pagebench): add aux file bench (#7746)

2024-05-17 20:04:02 +00:00

compaction

Tiered compaction: improvements to the windows (#7787)

2024-05-16 22:25:19 +02:00

ctl

feat(pageserver): persist aux file policy in index part (#7668)

2024-05-17 19:22:49 +00:00

pagebench

feat(pagebench): add aux file bench (#7746)

2024-05-17 20:04:02 +00:00

src

feat(pageserver): generate image layers for sparse keyspace (#7567)

2024-05-20 16:08:45 +00:00

test_data

Add walingest test (#5892)

2023-11-28 12:50:53 -05:00

Cargo.toml

feat(pageserver): add metadata key range and aux key encoding (#7401)

2024-04-23 15:16:04 +00:00