Erik Grinaker f18aa04b90 safekeeper: use set_len() to zero out segments (#9665) ...

## Problem

When we create a new segment, we zero it out in order to avoid changing
the length and fsyncing metadata on every write. However, we zeroed it
out by writing 8 KB zero-pages, and Tokio file writes have non-trivial
overhead.

## Summary of changes

Zero out the segment using
[`File::set_len()`](https://docs.rs/tokio/latest/i686-unknown-linux-gnu/tokio/fs/struct.File.html#method.set_len)
instead. This will typically (depending on the filesystem) just write a
sparse file and omit the 16 MB of data entirely. This improves WAL
append throughput for large messages by over 400% with fsync disabled,
and 100% with fsync enabled.

2024-11-07 15:09:57 +00:00

..

bin

safekeeper: retry eviction. (#9485)

2024-10-25 17:48:29 +03:00

http

safekeeper: don't un-evict timelines during snapshot API handler (#9428)

2024-10-28 08:47:12 +00:00

auth.rs

storage controller: use 'infra' JWT scope for node registration (#9343)

2024-10-10 12:26:43 +01:00

broker.rs

safekeeper: more consistent task naming.

2024-09-03 17:21:36 +03:00

control_file_upgrade.rs

Add eviction_state to control file (#8125)

2024-06-21 13:15:02 +01:00

control_file.rs

safekeeper: don't pass conf into storage constructors (#9523)

2024-10-25 18:19:52 +01:00

copy_timeline.rs

safekeeper: don't allocate send buffers on stack (#9644)

2024-11-05 17:05:30 +00:00

debug_dump.rs

safekeeper: don't allocate send buffers on stack (#9644)

2024-11-05 17:05:30 +00:00

handler.rs

Remove async_trait from the Handler trait (#8958)

2024-09-10 02:40:00 +02:00

json_ctrl.rs

postgres_ffi: add WAL generator for tests/benchmarks (#9503)

2024-10-30 14:46:39 +03:00

lib.rs

safekeeper: add basic WAL ingestion benchmarks (#9531)

2024-11-07 13:24:03 +00:00

metrics.rs

safekeeper: add walreceiver metrics (#9450)

2024-11-04 15:22:46 +00:00

patch_control_file.rs

Patch safekeeper control file on HTTP request (#6455)

2024-01-29 18:20:57 +00:00

pull_timeline.rs

safekeeper: don't un-evict timelines during snapshot API handler (#9428)

2024-10-28 08:47:12 +00:00

rate_limit.rs

Improve safekeepers eviction rate limiting (#8456)

2024-08-02 15:26:46 +01:00

receive_wal.rs

safekeeper: add walreceiver metrics (#9450)

2024-11-04 15:22:46 +00:00

recovery.rs

safekeeper: more consistent task naming.

2024-09-03 17:21:36 +03:00

remove_wal.rs

safekeeper: remove local WAL files ignoring peer_horizon_lsn. (#8900)

2024-10-04 19:07:39 +03:00

safekeeper.rs

safekeeper: add walreceiver metrics (#9450)

2024-11-04 15:22:46 +00:00

send_wal.rs

safekeeper: don't allocate send buffers on stack (#9644)

2024-11-05 17:05:30 +00:00

state.rs

safekeeper: add basic WAL ingestion benchmarks (#9531)

2024-11-07 13:24:03 +00:00

timeline_eviction.rs

safekeeper: don't un-evict timelines during snapshot API handler (#9428)

2024-10-28 08:47:12 +00:00

timeline_guard.rs

Fix 1.82 clippy lint too_long_first_doc_paragraph (#8941)

2024-09-06 14:33:52 +02:00

timeline_manager.rs

safekeeper: don't un-evict timelines during snapshot API handler (#9428)

2024-10-28 08:47:12 +00:00

timeline.rs

safekeeper: add basic WAL ingestion benchmarks (#9531)

2024-11-07 13:24:03 +00:00

timelines_global_map.rs

safekeeper: don't pass conf into storage constructors (#9523)

2024-10-25 18:19:52 +01:00

timelines_set.rs

Fix 1.82 clippy lint too_long_first_doc_paragraph (#8941)

2024-09-06 14:33:52 +02:00

wal_backup_partial.rs

Fix 1.82 clippy lint too_long_first_doc_paragraph (#8941)

2024-09-06 14:33:52 +02:00

wal_backup.rs

remote_storage: add DownloadOpts::byte_(start|end) (#9293)

2024-10-09 10:29:06 +01:00

wal_service.rs

Fix 1.82 clippy lint too_long_first_doc_paragraph (#8941)

2024-09-06 14:33:52 +02:00

wal_storage.rs

safekeeper: use set_len() to zero out segments (#9665)

2024-11-07 15:09:57 +00:00