rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-05-18 05:30:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,388 Commits 1,487 Branches 282 Tags
35d8167f6852d8ee9176f6b2f598d5320e1438cd
Commit Graph

1388 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Bojan Serafimov
35d8167f68 Measure that materializing all versions bloats by 0.3 2022-03-28 15:42:16 -04:00
Bojan Serafimov
2f7d9d2dd5 Count modified bits per wal (WIP) 2022-03-27 18:41:14 -04:00
Bojan Serafimov
21f9774ea4 Merge branch 'heikki-kvstore' into bojan-psbench-over-kvstore 2022-03-18 16:27:18 -04:00
Bojan Serafimov
098d7046f8 Improve test 2022-03-18 14:37:01 -04:00
Konstantin Knizhnik
a39de2997f Optimize reading versions for delta_layer 
Store blob size in layer metadata for all layers types

Heikki: This is a squashed version of PR #1369
 2022-03-18 14:35:13 +02:00
Heikki Linnakangas
d756921220 RFC fixes, per comments in the PR 2022-03-18 14:18:25 +02:00
Heikki Linnakangas
2bc9ed164f Merge remote-tracking branch 'origin/main' into heikki-kvstore 2022-03-18 12:08:17 +02:00
Heikki Linnakangas
8c4d270cde Fix InMemoryLayer::dump 2022-03-18 11:57:19 +02:00
Heikki Linnakangas
35584f7242 Bump magic IDs, to distinguish old file format from new 2022-03-18 11:57:19 +02:00
Heikki Linnakangas
12141523f6 Improve comments 2022-03-18 11:57:19 +02:00
Heikki Linnakangas
d383ed4e68 Add missing fsyncs 2022-03-18 11:57:19 +02:00
Heikki Linnakangas
13ec0ce7b2 fix formatting 2022-03-17 19:40:08 +02:00
Heikki Linnakangas
80fc133833 Add sequential scan tests 2022-03-17 17:12:30 +02:00
Heikki Linnakangas
3da14d56f2 Fix materialized page caching. 2022-03-17 17:12:30 +02:00
Dmitry Ivanov
a7544eead5 Remove the last non-borrowed string from BeMessage (#1376) 2022-03-17 16:46:58 +03:00
Heikki Linnakangas
b0b2093d00 Improve comments and tidy up the code in pgdatadir_mapping.rs. 2022-03-17 13:14:33 +02:00
Bojan Serafimov
02aa7c023a Add todos 2022-03-16 22:51:39 -04:00
Bojan Serafimov
180631da1f Remove todos 2022-03-16 22:40:36 -04:00
Bojan Serafimov
811d46f070 Fix 2022-03-16 22:39:55 -04:00
Bojan Serafimov
728f299641 Add hot page workload 2022-03-16 19:18:28 -04:00
Bojan Serafimov
fb49418e7f Trim dead code 2022-03-16 19:05:40 -04:00
Bojan Serafimov
887dc8f112 Print some stats 2022-03-16 19:01:54 -04:00
Bojan Serafimov
aa7b32d892 Simplify 2022-03-16 18:42:58 -04:00
Bojan Serafimov
d7ed9d8e01 cleanup 2022-03-16 17:40:19 -04:00
Andrey Taranik
ab124c161b Merge branch 'release' into main 2022-03-17 00:05:51 +03:00
Andrey Taranik
1fddb0556f deploy playbook fix - interaction with console (#1374) 2022-03-17 00:01:17 +03:00
Andrey Taranik
15a2a2bf04 release 2202-03-16 (#1373) 
production deploy
 2022-03-16 23:00:01 +03:00
Bojan Serafimov
96c2b3a80a WIP working pageserver get_page client 2022-03-16 14:31:42 -04:00
Dmitry Ivanov
705f51db27 [proxy] Propagate some errors to user (#1329) 
* [proxy] Propagate most errors to user

This change enables propagation of most errors to the user
(e.g. auth and connectivity errors). Some of them will be
stripped of sensitive information.

As a side effect, most occurrences of `anyhow::Error` were
replaced with concrete error types.

* [proxy] Box weighty errors
 2022-03-16 21:20:04 +03:00
Heikki Linnakangas
7560854370 Rename things in KeyPartition, per Bojan's suggestions. 2022-03-16 19:29:07 +02:00
Heikki Linnakangas
adbbb0a4c8 Merge remote-tracking branch 'origin/main' into heikki-kvstore 2022-03-16 19:08:45 +02:00
Heikki Linnakangas
9c1a9a1d9f Update Cargo.lock for new dependencies (#1354) 
Commit b2ad8342d2 added dependency on 'criterion', which pulled along
some other crates.
 2022-03-14 21:06:25 +03:00
Heikki Linnakangas
6a264aaca3 Stopgap "fix" for test_parallel_copy failure in debug mode. 2022-03-14 19:54:38 +02:00
Heikki Linnakangas
60ed6b3710 Shave some CPU cycles from reading blobs from files. 
This shows up in 'perf' profile when running in debug mode. Not so
significant in release mode, but still.
 2022-03-14 19:53:00 +02:00
Arseny Sher
d5a96d3d50 Fix finding end of WAL on safekeepers after f86cf93435. 
That commit dropped wal_start_lsn, now we're looking since commit_lsn, which is
the real end of WAL if no records follow it.

ref #1351
 2022-03-14 18:54:59 +03:00
Heikki Linnakangas
89690d7349 Prevent compaction from running at same time as GC. 
For same reasons as we prohibited concurrent checkpointing and GC
previosly.
 2022-03-14 14:22:04 +02:00
Heikki Linnakangas
09f2dff537 Refactor the checkpoint and compaction functions. 
The concept of a "checkpoint" had become quite muddled. This tries to
clarify it again.
 2022-03-14 13:22:46 +02:00
Heikki Linnakangas
2d8587f67d Separate flushing in-memory layer to disk from checkpoints. 
When 'checkpoint_distance' is reached, freeze the current in-memory
layer directly in the WAL receiver thread. And to flush the frozen
layer to disk, launch a separate "layer flushing thread". This leaves
only the compaction duty to the checkpoint thread.
 2022-03-14 11:37:22 +02:00
Heikki Linnakangas
c559c72ede Merge remote-tracking branch 'origin/main' into HEAD 2022-03-14 10:26:05 +02:00
Heikki Linnakangas
f06707badc Bugfix: a few constant keys were missing from collect_keyspace 
As a result, you got "could not find data for key" errors.
 2022-03-13 01:15:32 +02:00
Heikki Linnakangas
64cdd6064d Don't ClearVisibilityMapFlags records for non-existent blocks. 
We create a ClearVisibilityMapFlags record for the VM page, when a heap
WAL record indicates that the VM bit needs to be cleared. However,
sometimes the VM block would not exist. It seems that PostgreSQL
sometimes sets the clear-VM bit on WAL records, even though the
corresponding VM page hasn't been initialized yet. There's no point in
trying to clear a bit on a non-existent bit, so just skip emitting the
record if the VM page doesn't exist.

I'm not entirely sure why we're only seeing this bug with this PR, I
think it existed before. Maybe we were more sloppy and returned
an all-zeros page?
 2022-03-13 01:14:58 +02:00
Heikki Linnakangas
4bd557ca61 Move RFC doc here. 
From https://github.com/zenithdb/rfcs/pull/17
 2022-03-12 11:35:28 +02:00
Heikki Linnakangas
d93fc371f3 Import all existing RFCs documents from the separate 'rfcs' repository. 2022-03-11 18:49:36 +02:00
Heikki Linnakangas
ee40297758 Refactor keyspace code 
Have separate classes for the KeySpace, a partitioning of the KeySpace
(KeyPartitioning), and a builder object used to construct the KeySpace.
Previously, KeyPartitioning did all those things, and it was a bit
confusing.
 2022-03-11 16:24:13 +02:00
Heikki Linnakangas
d5b8380dae Improve comments on image layer. 
Make it more explicit that if a key doesn't exist in an image layer, it
doesn't exist.
 2022-03-11 09:47:09 +02:00
Heikki Linnakangas
bce2da4e55 Another 'tablespace' test fix. 2022-03-11 00:53:46 +02:00
Dhammika Pathirana
5d7bd8643a Fix page reconstruct time histo 
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com>
 2022-03-10 14:42:28 -08:00
Dhammika Pathirana
a8a7dc9ca6 Fix zid encoding 
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com>
 2022-03-10 14:42:28 -08:00
Dhammika Pathirana
b2ad8342d2 Add zid stringify bench test 
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com>
 2022-03-10 14:42:28 -08:00
Dhammika Pathirana
27dadba52c Fix retain references to layer histograms 
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com>
 2022-03-10 14:42:28 -08:00