mirror of
https://github.com/neondatabase/neon.git
synced 2026-01-15 17:32:56 +00:00
ddbdcdddd7
Refactor the tenant_size_model code. Segment now contains just the
minimum amount of information needed to calculate the size. Other
information that is useful for building up the segment tree, and for
display purposes, is now kept elsewhere. The code in 'main.rs' has a new
ScenarioBuilder struct for that.
Calculating which Segments are "needed" is now the responsibility of the
caller of tenant_size_mode, not part of the calculation itself. So it's
up to the caller to make all the decisions with retention periods for
each branch.
The output of the sizing calculation is now a Vec of SizeResults, rather
than a tree. It uses a tree representation internally, when doing the
calculation, but it's not exposed to the caller anymore.
Refactor the way the recursive calculation is performed.
Rewrite the code in size.rs that builds the Segment model. Get rid of
the intermediate representation with Update structs. Build the Segments
directly, with some local HashMaps and Vecs to track branch points to
help with that.
retention_period is now an input to gather_inputs(), rather than an
output.
Update pageserver http API: rename /size endpoint to /synthetic_size
with following parameters:
- /synthetic_size?inputs_only to get debug info;
- /synthetic_size?retention_period=0 to override cutoff that is used to
calculate the size;
pass header -H "Accept: text/html" to get HTML output, otherwise JSON is
returned
Update python tests and openapi spec.
---------
Co-authored-by: Anastasia Lubennikova <anastasia@neon.tech>
Co-authored-by: Joonas Koivunen <joonas@neon.tech>
71 lines
2.7 KiB
Rust
71 lines
2.7 KiB
Rust
//! Synthetic size calculation
|
|
|
|
mod calculation;
|
|
pub mod svg;
|
|
|
|
/// StorageModel is the input to the synthetic size calculation. It represents
|
|
/// a tree of timelines, with just the information that's needed for the
|
|
/// calculation. This doesn't track timeline names or where each timeline
|
|
/// begins and ends, for example. Instead, it consists of "points of interest"
|
|
/// on the timelines. A point of interest could be the timeline start or end point,
|
|
/// the oldest point on a timeline that needs to be retained because of PITR
|
|
/// cutoff, or snapshot points named by the user. For each such point, and the
|
|
/// edge connecting the points (implicit in Segment), we store information about
|
|
/// whether we need to be able to recover to the point, and if known, the logical
|
|
/// size at the point.
|
|
///
|
|
/// The segments must form a well-formed tree, with no loops.
|
|
#[derive(serde::Serialize)]
|
|
pub struct StorageModel {
|
|
pub segments: Vec<Segment>,
|
|
}
|
|
|
|
/// Segment represents one point in the tree of branches, *and* the edge that leads
|
|
/// to it (if any). We don't need separate structs for points and edges, because each
|
|
/// point can have only one parent.
|
|
///
|
|
/// When 'needed' is true, it means that we need to be able to reconstruct
|
|
/// any version between 'parent.lsn' and 'lsn'. If you want to represent that only
|
|
/// a single point is needed, create two Segments with the same lsn, and mark only
|
|
/// the child as needed.
|
|
///
|
|
#[derive(Clone, Debug, Eq, PartialEq, serde::Serialize, serde::Deserialize)]
|
|
pub struct Segment {
|
|
/// Previous segment index into ['Storage::segments`], if any.
|
|
pub parent: Option<usize>,
|
|
|
|
/// LSN at this point
|
|
pub lsn: u64,
|
|
|
|
/// Logical size at this node, if known.
|
|
pub size: Option<u64>,
|
|
|
|
/// If true, the segment from parent to this node is needed by `retention_period`
|
|
pub needed: bool,
|
|
}
|
|
|
|
/// Result of synthetic size calculation. Returned by StorageModel::calculate()
|
|
pub struct SizeResult {
|
|
pub total_size: u64,
|
|
|
|
// This has same length as the StorageModel::segments vector in the input.
|
|
// Each entry in this array corresponds to the entry with same index in
|
|
// StorageModel::segments.
|
|
pub segments: Vec<SegmentSizeResult>,
|
|
}
|
|
|
|
#[derive(Clone, Debug, Eq, PartialEq, serde::Serialize, serde::Deserialize)]
|
|
pub struct SegmentSizeResult {
|
|
pub method: SegmentMethod,
|
|
// calculated size of this subtree, using this method
|
|
pub accum_size: u64,
|
|
}
|
|
|
|
/// Different methods to retain history from a particular state
|
|
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Serialize, serde::Deserialize)]
|
|
pub enum SegmentMethod {
|
|
SnapshotHere, // A logical snapshot is needed after this segment
|
|
Wal, // Keep WAL leading up to this node
|
|
Skipped,
|
|
}
|