mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-04 12:02:55 +00:00
9c6ff3aa2b
Main change: - `BufferedWriter` owns the `W`; no more `Arc<W>` - We introduce auto-delete-on-drop wrappers for `VirtualFile`. - `TempVirtualFile` for write-only users - `TempVirtualFileCoOwnedByEphemeralFileAndBufferedWriter` for EphemeralFile which requires read access to the immutable prefix of the file (see doc comments for details) - Users of `BufferedWriter` hand it such a wrapped `VirtualFile`. - The wrapped `VirtualFile` moves to the background flush task. - On `BufferedWriter` shutdown, ownership moves back. - Callers remove the wrapper (`disarm_into_inner()`) after doing final touches, e.g., flushing index blocks and summary for delta/image layer writers. If the BufferedWriter isn't shut down properly via `BufferedWriter::shutdown`, or if there is an error during final touches, the wrapper type ensures that the file gets unlinked. We store a GateGuard inside the wrapper to ensure that the Timeline is still alive when unlinking on drop. Rust doesn't have async drop yet, so, the unlinking happens using a synchronous syscall. NB we don't fsync the surrounding directory. This is how it's been before this PR; I believe it is correct because all of these files are temporary paths that get cleaned up on timeline load. Again, timeline load does not need to fsync because the next timeline load will unlink again if the file reappears. The auto-delete-on-drop can happen after a higher-level mechanism retries. Therefore, we switch all users to monotonically increasing, never-reused temp file disambiguators. The aspects pointed out in the last two paragraphs will receive further cleanup in follow-up task - https://github.com/neondatabase/neon/issues/11692 Drive-by changes: - It turns out we can remove the two-pronged code in the layer file download code. No need to make this a separate PR because all of production already uses `tokio-epoll-uring` with the buffered writer for many weeks. Refs - epic https://github.com/neondatabase/neon/issues/9868 - alternative to https://github.com/neondatabase/neon/pull/11544