Alex Chi Z 11945e64ec chore(pageserver): improve in-memory layer vectored get (#7467) ...

previously in https://github.com/neondatabase/neon/pull/7375, we
observed that for in-memory layers, we will need to iterate every key in
the key space in order to get the result. The operation can be more
efficient if we use BTreeMap as the in-memory layer representation, even
if we are doing vectored get in a dense keyspace. Imagine a case that
the in-memory layer covers a very little part of the keyspace, and most
of the keys need to be found in lower layers. Using a BTreeMap can
significantly reduce probes for nonexistent keys.

## Summary of changes

* Use BTreeMap as in-memory layer representation.
* Optimize the vectored get flow to utilize the range scan functionality
of BTreeMap.

Signed-off-by: Alex Chi Z <chi@neon.tech>

2024-04-29 17:16:42 +00:00

..

bin

pageserver: coalesce read paths (#7477)

2024-04-25 13:29:17 +01:00

consumption_metrics

pageserver: improve readability of shard.rs (#7330)

2024-04-15 11:50:26 +01:00

context

feat(per-tenant throttling): exclude throttled time from page_service metrics + regression test (#6953)

2024-03-05 13:44:00 +00:00

deletion_queue

move timeouts and cancellation handling to remote_storage (#6697)

2024-02-14 23:24:07 +00:00

http

neon_local: add "tenant import" (#7399)

2024-04-29 08:52:18 +01:00

tenant

chore(pageserver): improve in-memory layer vectored get (#7467)

2024-04-29 17:16:42 +00:00

virtual_file

fix(virtual_file): compile warnings on macos (#7525)

2024-04-26 17:09:51 -04:00

walredo

add async walredo mode (disabled-by-default, opt-in via config) (#6548)

2024-04-15 22:14:42 +02:00

auth.rs

attachment_service: JWT auth enforcement (#6897)

2024-02-26 18:17:06 +00:00

aux_file.rs

chore(pageserver): shrink aux keyspace to 0x60-0x7F (#7502)

2024-04-26 13:35:01 -04:00

basebackup.rs

Remove the now-unused 'latest' arguments (#7377)

2024-04-25 20:45:37 +03:00

config.rs

pageserver: coalesce read paths (#7477)

2024-04-25 13:29:17 +01:00

consumption_metrics.rs

pageserver: improve readability of shard.rs (#7330)

2024-04-15 11:50:26 +01:00

context.rs

feat(per-tenant throttling): exclude throttled time from page_service metrics + regression test (#6953)

2024-03-05 13:44:00 +00:00

control_plane_client.rs

tests: improve stability of test_deletion_queue_recovery (#7325)

2024-04-05 18:01:31 +01:00

deletion_queue.rs

pageserver: extend /re-attach response to include tenant mode (#6941)

2024-03-21 13:39:23 +00:00

disk_usage_eviction_task.rs

pageserver: remove bare mgr::get_tenant, mgr::list_tenants (#7237)

2024-03-26 18:29:08 +00:00

import_datadir.rs

pageserver: apply shard filtering to blocks ingested during initdb (#7319)

2024-04-05 18:07:35 +01:00

lib.rs

feat(pageserver): add metadata key range and aux key encoding (#7401)

2024-04-23 15:16:04 +00:00

metrics.rs

metrics: record time to update gc info as a per timeline metric (#7473)

2024-04-29 07:14:53 +03:00

page_cache.rs

Fix warnings and compile errors on nightly (#6886)

2024-03-01 17:14:19 +01:00

page_service.rs

Server support for new pagestream protocol version (#7377)

2024-04-25 20:45:37 +03:00

pgdatadir_mapping.rs

Return Past instead of Present or Future when commit_lsn < min_lsn (#7520)

2024-04-26 16:23:25 +02:00

repository.rs

feat: copy delta layer prefix or "truncate" (#7228)

2024-04-18 10:43:04 +03:00

span.rs

debug_assert presence of shard_id tracing field (#6572)

2024-02-06 14:43:33 +00:00

statvfs.rs

Also allow unnecessary_fallible_conversions lint (#6294)

2024-01-09 04:22:36 +00:00

task_mgr.rs

refactor(ephemeral_file): reuse owned_buffers_io::BufferedWriter (#7484)

2024-04-26 13:01:26 +02:00

tenant.rs

neon_local: add "tenant import" (#7399)

2024-04-29 08:52:18 +01:00

trace.rs

fix: replace all std::PathBufs with camino::Utf8PathBuf (#5352)

2023-10-04 17:52:23 +03:00

utilization.rs

implement Serialize/Deserialize for SystemTime with RFC3339 format (#7203)

2024-04-08 15:53:07 +01:00

virtual_file.rs

refactor(ephemeral_file): reuse owned_buffers_io::BufferedWriter (#7484)

2024-04-26 13:01:26 +02:00

walingest.rs

Remove the now-unused 'latest' arguments (#7377)

2024-04-25 20:45:37 +03:00

walrecord.rs

feat: copy delta layer prefix or "truncate" (#7228)

2024-04-18 10:43:04 +03:00

walredo.rs

add async walredo mode (disabled-by-default, opt-in via config) (#6548)

2024-04-15 22:14:42 +02:00