Use timestamp->LSN mapping instead of file modification time.
Fix 'latest_gc_cutoff_lsn' - set it to the minimum of pitr_cutoff and gc_cutoff.
Add new test: test_pitr_gc
Try to follow Prometheus style-guide https://prometheus.io/docs/practices/naming/ for metrics names. More specifically:
- Use `pageserver_` prefix for all pagserver metrics
- Specify `_seconds` unit in time metrics
- Use unit as a suffix in other cases, such as `_hits`, `_bytes`, `_records`
- Use `_total` suffix for accumulating counters (note that Histograms append that suffix internally)
* Do not apply records with LSN smaller than LSN of cached image in delta layer
* Do not apply records with LSN smaller than LSN of cached image in delta layer
Move the responsibility for training the dictionary into
DeltaLayerWriter, so that the callers don't need to know about it.
Add comments.
If the compressed version of a blob would be larger than the original,
store it uncompressed.
* Do not set LSN for new FPI page
refer #1656
* Add page_is_new, page_get_lsn, page_set_lsn functions
* Fix page_is_new implementation
* Add comment from XLogReadBufferForRedoExtended
This relies on the zstd 'experimental' feature, because the
zstd::bulk::Decompressor::upper_bound() function, which uses the
function ZSTD_decompressBound() function, is still experimental in the
zstd library. I'm OK with that, it's unlikely that the function would
change, and if it does, I'm sure there will be a replacement. There's
also the zstd_safe::get_decompressed_size() function that we could
use, but we only need an upper-bound, not the exact size, so
upper_bound() seems more appropriate.
Now princeple is following: acceptor threads (libpq and http) error will
bring the pageserver down, but all per-tenant thread failures will be treated
as an error.
A new `get_lsn_by_timestamp` command is added to the libpq page service
API.
An extra timestamp field is now stored in an extra field after each
Clog page. It is the timestamp of the latest commit, among all the
transactions on the Clog page. To find the overall latest commit, we
need to scan all Clog pages, but this isn't a very frequent operation
so that's not too bad.
To find the LSN that corresponds to a timestamp, we perform a binary
search. The binary search starts with min = last LSN when GC ran, and
max = latest LSN on the timeline. On each iteration of the search we
check if there are any commits with a higher-than-requested timestamp
at that LSN.
Implements github issue 1361.
* Traverse frozen layer in get_reconstruct_data in reverse order
* Fix comments on frozen layers.
Note explicitly the order that the layers are in the queue.
* Add fail point to reproduce failpoint iteration error
Co-authored-by: Heikki Linnakangas <heikki@neon.tech>
Konstantin Knizhnik
Anastasia Lubennikova
Kirill Bulatov
Arthur Petukhovsky
Heikki Linnakangas
Dhammika Pathirana
Thang Pham
Dmitry Rodionov
bojanserafimov
Stas Kelvich
Anastasia Lubennikova
Dhammika Pathirana
Dmitry Rodionov