Commit Graph

34 Commits

Author SHA1 Message Date
Konstantin Knizhnik
e9c7665c81 Fix 2PC support 2021-05-21 20:01:44 +03:00
Konstantin Knizhnik
2bf2dd1d88 Add nonrelfile_utils.rs file 2021-05-20 14:48:55 +03:00
Konstantin Knizhnik
20b6279beb Fix restoring non-relational data during compute node startup 2021-05-20 14:14:52 +03:00
Konstantin Knizhnik
06f96f9600 Do not transfer WAL to computation nodes: use pg_resetwal for node startup 2021-05-20 14:13:47 +03:00
Eric Seppanen
9c94a34ae7 cargo fmt 2021-05-19 15:26:21 -07:00
Eric Seppanen
8f43d7637c wal_service: move code around some more
Move ReceiveWalConn into its own file. Shuffle constants around so they
are close to the protocol they're associated with, or move them into
postgres_ffi if they seem to be global constants.
2021-05-19 14:36:41 -07:00
Heikki Linnakangas
538f903861 Optimize parse_relfilename() function.
Compiling a Regex is very expensive, so let's not do it on every
invocation. This was consuming a big fraction of the time in creating
a new base backup at "zenith pg create". This commits brings down the
time to run "zenith pg create" on a freshly created repository from
about 2 seconds to 1 second.

It's not worth spending much effort on optimizing things at this stage
in general, but might as well pick low-hanging fruit like this.
2021-05-19 14:08:37 +03:00
Heikki Linnakangas
aa8debf4e8 Add test for a relation that's larger than 1 GB.
This isn't very exciting with the current RocksDB implementation, because
it doesn't care about the PostgreSQL 1 GB segment boundaries at all.
But I think we will care about this in the future, and more tests is
generally better anyway.
2021-05-19 09:22:17 +03:00
Heikki Linnakangas
2b2d24433a Don't panic on zenith-specific "fork numbers".
These can be printed as part of log messages, if you display a RelTag
that contains one.
2021-05-18 20:49:59 +03:00
Konstantin Knizhnik
04dc698d4b Add support of twophase transactions 2021-05-16 00:03:20 +03:00
Konstantin Knizhnik
9ece1e863d Compute and restore pg_xact, pg_multixact and pg_filenode.map files 2021-05-14 16:35:09 +03:00
Heikki Linnakangas
4dccdb33ab Fix comment formatting.
The module comment should use "//!" instead of "///". Otherwise, it is
considered to apply to the *next* thing, in this case the "use" statement
that follows, not the file as whole. "cargo fmt" revealed this by insisting
to move the "use crate::pg_constants" line to before the comment.
2021-05-13 22:06:05 +03:00
Konstantin Knizhnik
2f2dff4c8d Merge with main brnach 2021-05-12 10:46:01 +03:00
Konstantin Knizhnik
22e7fcbf2d Handle visbility map updates in WAL redo 2021-05-12 10:38:43 +03:00
Eric Seppanen
bace19ffbe wal_service: switch to Lsn type
Replace XLogRecPtr with Lsn in wal_service.rs .

This removes the last use of XLogSegmentOffset and XLByteToSeg, so
delete them. (replaced by Lsn::segment_offset and Lsn::segment_number.)
2021-05-10 16:21:05 -07:00
Eric Seppanen
df5a55c445 add workspace_hack crate
Our builds can be a little inconsistent, because Cargo doesn't deal well
with workspaces where there are multiple crates which have different
dependencies that select different features. As a workaround, copy what
other big rust projects do: add a workspace_hack crate.

This crate just pins down a set of dependencies and features that
satisfies all of the workspace crates.

The benefits are:
- running `cargo build` from one of the workspace subdirectories now
  works without rebuilding anything.
- running `cargo install` works (without rebuilding anything).
- making small dependency changes is much less likely to trigger large
  dependency rebuilds.
2021-05-07 13:08:31 -07:00
Heikki Linnakangas
e5e5c3e067 Tidy up the parse_relfilename function.
A few things that Eric commented on at PR #96:

- Use thiserror to simplify the implemention of FilePathError
- Add unit tests
- Fix a few complaints from clippy
2021-05-07 11:01:34 +03:00
Heikki Linnakangas
61af9bb889 Move a few functions that have been copy-pasted around to shared module. 2021-05-06 21:57:10 +03:00
Heikki Linnakangas
33d126ecbe Tidy up usage of a few constants from PostgreSQL headers. 2021-05-06 21:57:01 +03:00
anastasia
15db0d1d6f refactor walreciever and restore_local_repo 2021-05-06 12:58:08 +03:00
Konstantin Knizhnik
eea6f0898e Restore CLOG from snapshot 2021-04-30 14:22:47 +03:00
anastasia
b49164a1d4 cargo fmt 2021-04-29 18:41:42 +03:00
anastasia
e7b112aacc Refactor pg_constants. Move them to postgres_ffi/ 2021-04-29 18:41:42 +03:00
Heikki Linnakangas
41a3772e90 Replace pgbuild.sh with a Makefile
This allows building both Zenith and PostgreSQL in one command. The
command is 'make'

Reviewed-by: Arseny Sher <sher-ars@yandex.ru>
2021-04-28 16:54:45 +03:00
anastasia
421d586953 code cleanup for XLogRecord decoding 2021-04-28 13:56:27 +03:00
anastasia
ef37eb96b9 refactor XLogRecord reading 2021-04-28 13:56:27 +03:00
Konstantin Knizhnik
28f2800275 Merge branch 'main' into rocksdb_pageserver 2021-04-22 14:00:57 +03:00
Heikki Linnakangas
8af5cbedb1 Move xlog_utils.rs to postgres_ffi module.
I had copy-pasted these functions to a few other places. Clean that up,
move them to a common module, and add some comments.
2021-04-22 13:22:34 +03:00
Konstantin Knizhnik
9e7c45cb72 Merge with master 2021-04-22 09:45:13 +03:00
Eric Seppanen
1f3f4cfaf5 clippy cleanup #2
- remove needless return
- remove needless format!
- remove a few more needless clone()
- from_str_radix(_, 10) -> .parse()
- remove needless reference
- remove needless `mut`

Also manually replaced a match statement with map_err() because after
clippy was done with it, there was almost nothing left in the match
expression.
2021-04-21 17:56:58 -07:00
Konstantin Knizhnik
d8fa2ec367 Merge with main branch 2021-04-21 16:10:05 +03:00
Eric Seppanen
92e4f4b3b6 cargo fmt 2021-04-20 17:59:56 -07:00
Heikki Linnakangas
7f777a485e Fix caching of 'postgres' build in github action.
The postgres_ext.h isn't found in vendor/postgres, if the Postgres
was restored from cache instead of building it in vendor/postgres.
To fix, change include path to point into tmp_install/include where the
headers are installed, instead of the vendor/postgres source dir.
2021-04-20 20:27:22 +03:00
Heikki Linnakangas
b451ede199 Use rust bindgen for reading/writing the PostgreSQL control file. 2021-04-20 15:57:05 +03:00