Alex Chi Z 9f4511c554 feat(pageserver): add k-merge layer iterator with lazy loading (#8053) ...

Part of https://github.com/neondatabase/neon/issues/8002. This pull
request adds a k-merge iterator for bottom-most compaction.

## Summary of changes

* Added back lsn_range / key_range in delta layer inner. This was
removed due to https://github.com/neondatabase/neon/pull/8050, but added
back because iterators need that information to process lazy loading.
* Added lazy-loading k-merge iterator.
* Added iterator wrapper as a unified iterator type for image+delta
iterator.

The current status and test should cover the use case for L0 compaction
so that the L0 compaction process can bypass page cache and have a fixed
amount of memory usage. The next step is to integrate this with the new
bottom-most compaction.

---------

Signed-off-by: Alex Chi Z <chi@neon.tech>
Co-authored-by: Christian Schwarz <christian@neon.tech>

2024-07-10 14:11:27 -04:00

..

benches

fix: shutdown does not kill walredo processes (#8150)

2024-06-27 15:58:28 +02:00

client

pageserver: move page_service's import basebackup / import wal to mgmt API (#8292)

2024-07-09 23:17:42 +02:00

compaction

refactor(rtc): remove the duplicate IndexLayerMetadata (#7860)

2024-05-23 23:24:31 +03:00

ctl

remote_storage config: move handling of empty inline table {} to callers (#8193)

2024-07-02 12:53:08 +02:00

pagebench

Move keyspace utils to inherent impls (#7929)

2024-06-03 16:18:07 +02:00

src

feat(pageserver): add k-merge layer iterator with lazy loading (#8053)

2024-07-10 14:11:27 -04:00

test_data

Add walingest test (#5892)

2023-11-28 12:50:53 -05:00

Cargo.toml

pageserver: switch to jemalloc (#8307)

2024-07-08 14:10:42 +01:00