mirror of
https://github.com/neondatabase/neon.git
synced 2026-05-23 08:00:37 +00:00
556422f9bb
Change the way the page versions are stored in an in-memory layer. Instead of storing the original PageVersion structs, serialize them into an expandable buffer. That has a couple of advantages: first, the serialized form is more compact, which greatly reduces the memory usage. Secondly, we can track the memory used more accurately. This moves the serialization work from the checkpointer to the WAL receiver. It also means that if a page in an in-memory layer is accessed, it needs to be deserialized on the GetPage call. That's adds a bit of latency, but it's not significant compared to the overhead of replaying WAL records. The simpler memory accounting is worth it. Per Konstantin's idea a long time ago.
80 lines
2.8 KiB
Rust
80 lines
2.8 KiB
Rust
//! zenith_utils is intended to be a place to put code that is shared
|
|
//! between other crates in this repository.
|
|
|
|
#![allow(clippy::manual_range_contains)]
|
|
|
|
/// `Lsn` type implements common tasks on Log Sequence Numbers
|
|
pub mod lsn;
|
|
/// SeqWait allows waiting for a future sequence number to arrive
|
|
pub mod seqwait;
|
|
|
|
/// append only ordered map implemented with a Vec
|
|
pub mod vec_map;
|
|
|
|
/// expandable byte buffer consisting of fixed-size chunks
|
|
pub mod chunked_buffer;
|
|
|
|
// Async version of SeqWait. Currently unused.
|
|
// pub mod seqwait_async;
|
|
|
|
pub mod bin_ser;
|
|
pub mod postgres_backend;
|
|
pub mod pq_proto;
|
|
|
|
// dealing with connstring parsing and handy access to it's parts
|
|
pub mod connstring;
|
|
|
|
// helper functions for creating and fsyncing directories/trees
|
|
pub mod crashsafe_dir;
|
|
|
|
// common authentication routines
|
|
pub mod auth;
|
|
|
|
// utility functions and helper traits for unified unique id generation/serialization etc.
|
|
pub mod zid;
|
|
// http endpoint utils
|
|
pub mod http;
|
|
|
|
// socket splitting utils
|
|
pub mod sock_split;
|
|
|
|
// common log initialisation routine
|
|
pub mod logging;
|
|
|
|
// Misc
|
|
pub mod accum;
|
|
|
|
// Utility for binding TcpListeners with proper socket options.
|
|
pub mod tcp_listener;
|
|
|
|
// Utility for putting a raw file descriptor into non-blocking mode
|
|
pub mod nonblock;
|
|
|
|
// This is a shortcut to embed git sha into binaries and avoid copying the same build script to all packages
|
|
//
|
|
// we have several cases:
|
|
// * building locally from git repo
|
|
// * building in CI from git repo
|
|
// * building in docker (either in CI or locally)
|
|
//
|
|
// One thing to note is that .git is not available in docker (and it is bad to include it there).
|
|
// So everything becides docker build is covered by git_version crate.
|
|
// For docker use environment variable to pass git version, which is then retrieved by buildscript (build.rs).
|
|
// It takes variable from build process env and puts it to the rustc env. And then we can retrieve it here by using env! macro.
|
|
// Git version received from environment variable used as a fallback in git_version invokation.
|
|
// And to avoid running buildscript every recompilation, we use rerun-if-env-changed option.
|
|
// So the build script will be run only when GIT_VERSION envvar has changed.
|
|
//
|
|
// Why not to use buildscript to get git commit sha directly without procmacro from different crate?
|
|
// Caching and workspaces complicates that. In case zenith_utils is not
|
|
// recompiled due to caching then version may become outdated.
|
|
// git_version crate handles that case by introducing a dependency on .git internals via include_bytes! macro,
|
|
// so if we changed the index state git_version will pick that up and rerun the macro.
|
|
//
|
|
// Note that with git_version prefix is `git:` and in case of git version from env its `git-env:`.
|
|
use git_version::git_version;
|
|
pub const GIT_VERSION: &str = git_version!(
|
|
prefix = "git:",
|
|
fallback = concat!("git-env:", env!("GIT_VERSION"))
|
|
);
|