Heikki Linnakangas
d383ed4e68
Add missing fsyncs
2022-03-18 11:57:19 +02:00
Heikki Linnakangas
13ec0ce7b2
fix formatting
2022-03-17 19:40:08 +02:00
Heikki Linnakangas
80fc133833
Add sequential scan tests
2022-03-17 17:12:30 +02:00
Heikki Linnakangas
3da14d56f2
Fix materialized page caching.
2022-03-17 17:12:30 +02:00
Heikki Linnakangas
b0b2093d00
Improve comments and tidy up the code in pgdatadir_mapping.rs.
2022-03-17 13:14:33 +02:00
Heikki Linnakangas
7560854370
Rename things in KeyPartition, per Bojan's suggestions.
2022-03-16 19:29:07 +02:00
Heikki Linnakangas
adbbb0a4c8
Merge remote-tracking branch 'origin/main' into heikki-kvstore
2022-03-16 19:08:45 +02:00
Heikki Linnakangas
9c1a9a1d9f
Update Cargo.lock for new dependencies ( #1354 )
...
Commit b2ad8342d2
2022-03-14 21:06:25 +03:00
Heikki Linnakangas
6a264aaca3
Stopgap "fix" for test_parallel_copy failure in debug mode.
2022-03-14 19:54:38 +02:00
Heikki Linnakangas
60ed6b3710
Shave some CPU cycles from reading blobs from files.
...
This shows up in 'perf' profile when running in debug mode. Not so
significant in release mode, but still.
2022-03-14 19:53:00 +02:00
Arseny Sher
d5a96d3d50
Fix finding end of WAL on safekeepers after f86cf93435.
...
That commit dropped wal_start_lsn, now we're looking since commit_lsn, which is
the real end of WAL if no records follow it.
ref #1351
2022-03-14 18:54:59 +03:00
Heikki Linnakangas
89690d7349
Prevent compaction from running at same time as GC.
...
For same reasons as we prohibited concurrent checkpointing and GC
previosly.
2022-03-14 14:22:04 +02:00
Heikki Linnakangas
09f2dff537
Refactor the checkpoint and compaction functions.
...
The concept of a "checkpoint" had become quite muddled. This tries to
clarify it again.
2022-03-14 13:22:46 +02:00
Heikki Linnakangas
2d8587f67d
Separate flushing in-memory layer to disk from checkpoints.
...
When 'checkpoint_distance' is reached, freeze the current in-memory
layer directly in the WAL receiver thread. And to flush the frozen
layer to disk, launch a separate "layer flushing thread". This leaves
only the compaction duty to the checkpoint thread.
2022-03-14 11:37:22 +02:00
Heikki Linnakangas
c559c72ede
Merge remote-tracking branch 'origin/main' into HEAD
2022-03-14 10:26:05 +02:00
Heikki Linnakangas
f06707badc
Bugfix: a few constant keys were missing from collect_keyspace
...
As a result, you got "could not find data for key" errors.
2022-03-13 01:15:32 +02:00
Heikki Linnakangas
64cdd6064d
Don't ClearVisibilityMapFlags records for non-existent blocks.
...
We create a ClearVisibilityMapFlags record for the VM page, when a heap
WAL record indicates that the VM bit needs to be cleared. However,
sometimes the VM block would not exist. It seems that PostgreSQL
sometimes sets the clear-VM bit on WAL records, even though the
corresponding VM page hasn't been initialized yet. There's no point in
trying to clear a bit on a non-existent bit, so just skip emitting the
record if the VM page doesn't exist.
I'm not entirely sure why we're only seeing this bug with this PR, I
think it existed before. Maybe we were more sloppy and returned
an all-zeros page?
2022-03-13 01:14:58 +02:00
Heikki Linnakangas
4bd557ca61
Move RFC doc here.
...
From https://github.com/zenithdb/rfcs/pull/17
2022-03-12 11:35:28 +02:00
Heikki Linnakangas
d93fc371f3
Import all existing RFCs documents from the separate 'rfcs' repository.
2022-03-11 18:49:36 +02:00
Heikki Linnakangas
ee40297758
Refactor keyspace code
...
Have separate classes for the KeySpace, a partitioning of the KeySpace
(KeyPartitioning), and a builder object used to construct the KeySpace.
Previously, KeyPartitioning did all those things, and it was a bit
confusing.
2022-03-11 16:24:13 +02:00
Heikki Linnakangas
d5b8380dae
Improve comments on image layer.
...
Make it more explicit that if a key doesn't exist in an image layer, it
doesn't exist.
2022-03-11 09:47:09 +02:00
Heikki Linnakangas
bce2da4e55
Another 'tablespace' test fix.
2022-03-11 00:53:46 +02:00
Dhammika Pathirana
5d7bd8643a
Fix page reconstruct time histo
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-03-10 14:42:28 -08:00
Dhammika Pathirana
a8a7dc9ca6
Fix zid encoding
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-03-10 14:42:28 -08:00
Dhammika Pathirana
b2ad8342d2
Add zid stringify bench test
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-03-10 14:42:28 -08:00
Dhammika Pathirana
27dadba52c
Fix retain references to layer histograms
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-03-10 14:42:28 -08:00
Dhammika Pathirana
f67d010d1b
Add ps smgr/storage metrics tenant tags
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
Add tenant_id,timeline_id in smgr/storage metrics (#1234 )
2022-03-10 14:42:28 -08:00
Heikki Linnakangas
3948956e87
Fix pg_table_size() on a view
2022-03-10 23:35:24 +02:00
Heikki Linnakangas
a726b555fb
Handle tablespaces gracefully.
...
We don't really support tablespaces. But this makes the 'tablespace'
Postgres regression test pass, like it did previously.
2022-03-10 22:56:37 +02:00
Kirill Bulatov
093ad8ab59
Send 409 HTTP responses on timeline and tenant creation for existing entity
2022-03-10 19:38:58 +02:00
Kirill Bulatov
c51d545fd9
Serialize Lsn as strings in http api
2022-03-10 19:38:58 +02:00
Kirill Bulatov
fe6fccfdae
Allow already existing repo when creating a tenant
2022-03-10 19:38:58 +02:00
Kirill Bulatov
dd74c66ef0
Do not create timeline along with tenant
2022-03-10 19:38:58 +02:00
Kirill Bulatov
a5e10c4f64
Tidy up pageserver's endpoints
2022-03-10 19:38:58 +02:00
Kirill Bulatov
7b5482bac0
Properly store the branch name mappings
2022-03-10 19:38:58 +02:00
Kirill Bulatov
c7569dce47
Allow passing initial timeline id into zenith CLI commands
2022-03-10 19:38:58 +02:00
Kirill Bulatov
4d0f7fd1e4
Update Zenith CLI config between runs
2022-03-10 19:38:58 +02:00
Kirill Bulatov
f49990ed43
Allow creating timelines by branching off ancestors
2022-03-10 19:38:58 +02:00
Kirill Bulatov
0c91091c63
Avoid point in time concept on pageserver level
2022-03-10 19:38:58 +02:00
Kirill Bulatov
10f811e886
Use timeline instead of branch in pageserver's API
2022-03-10 19:38:58 +02:00
Heikki Linnakangas
0e3512aad0
Crank down logging again
2022-03-10 18:50:12 +02:00
Heikki Linnakangas
6fb566b46f
Bump vendor/postgres to fix a bug with smgrnblocks() on newly created rel
2022-03-10 16:05:21 +02:00
Heikki Linnakangas
dd56eeefbf
Crank up logging
2022-03-10 15:45:50 +02:00
Anastasia Lubennikova
2883a25761
Bump vendor/postgres to use local relation cache for smgr_exists
2022-03-10 17:36:09 +04:00
anastasia
87f306c516
Tune backpressure in python tests to make them more stable
2022-03-10 17:36:09 +04:00
anastasia
5b34afe893
Bump vendor/postgres to use local relation cache for smgr_exists
2022-03-10 17:36:09 +04:00
Heikki Linnakangas
d19a293e7e
Add a test for branching
2022-03-10 14:56:13 +02:00
Heikki Linnakangas
be4aebd7e9
silence clippy
2022-03-10 13:36:28 +02:00
Heikki Linnakangas
dac73328ba
Fix bug where reldir was not written to image layer.
2022-03-10 13:20:08 +02:00
Heikki Linnakangas
fb79c7f1f0
Make compaction more concurrent
2022-03-10 13:20:08 +02:00
