Commit Graph

441 Commits

Author SHA1 Message Date
Konstantin Knizhnik
ce54133ec4 Use zenith smgr for global relations 2021-05-21 12:46:17 +03:00
Konstantin Knizhnik
610e14a7fc Use new version of postgres 2021-05-21 12:36:24 +03:00
Konstantin Knizhnik
35a1c3d521 Specify right LSN in test_createdb.py 2021-05-21 12:20:38 +03:00
Heikki Linnakangas
22b7e74c83 Add test for following relmapper files at CREATE DATABASE 2021-05-21 12:13:47 +03:00
Konstantin Knizhnik
d95e1da742 Fix issue with propagation of CREATE DATABASE to the branch 2021-05-21 12:06:46 +03:00
Stas Kelvich
40d047c146 Remove double "zenith" mention in CLI header 2021-05-21 00:26:31 +03:00
Stas Kelvich
42f3dd47d2 update README.md according to recent CLI changes 2021-05-21 00:26:31 +03:00
Stas Kelvich
c2b2ab974c Hide initdb output from "zenith init" command 2021-05-21 00:26:31 +03:00
Stas Kelvich
6ad6e5bd84 Add --destroy flag to "pg stop" CLI command 2021-05-21 00:26:31 +03:00
Stas Kelvich
d534aeb9e1 Properly propagate control plane errors to CLI.
That allows to show decent error whenever we try to start already
started postgres.
2021-05-21 00:26:31 +03:00
Stas Kelvich
d45839879c Bind to socket earlier during pageserver init.
That allows printing reasonable error message instead of panicking if
address is already in use.
2021-05-21 00:26:31 +03:00
Stas Kelvich
1f6ca23db6 Be slightly more vocal about pageserver start/stop 2021-05-21 00:26:31 +03:00
Heikki Linnakangas
2127a65e27 Tidy up the code to launch WAL redo process a little bit
- if removing the old datadir fails, throw an error
- obey PageServerConf.workdir
2021-05-20 19:29:00 +03:00
Heikki Linnakangas
ecf2d181c4 Tidy up the code to create PageServerConf
Parse all the command line options before calling "zenith init" and
changing current working dir. The rest of the options don't make any
difference if we're initializing a new repository, but it seems strange
and error-prone to parse some arguments at different times.
2021-05-20 19:28:57 +03:00
Alexey Kondratov
c1bfa32771 Follow up for issue #144, better graphics in branches tree 2021-05-20 19:25:25 +03:00
Konstantin Knizhnik
8465738aa5 [refer #167] Fix handling of pg_filenode.map files in page server 2021-05-20 19:16:16 +03:00
Konstantin Knizhnik
87d7ce816d Add pause in pageserver stop method to avoid address already in use errors 2021-05-20 17:42:07 +03:00
anastasia
f38c2e620e Add test_zenith_regress.py that runs pg_regress styled tests from test_runner/zenith_regress.
TODO: remove similar tests from vendor/postgres testset
2021-05-20 17:24:39 +03:00
Konstantin Knizhnik
86056abd0e Fix merge conflict: set initial WAL position to second segment because of pg_resetwal 2021-05-20 15:26:39 +03:00
Konstantin Knizhnik
2bf2dd1d88 Add nonrelfile_utils.rs file 2021-05-20 14:48:55 +03:00
Konstantin Knizhnik
874d82fd4c Fix tests in lsn.rs after changing wal_seg_size type 2021-05-20 14:45:09 +03:00
Konstantin Knizhnik
3645133700 Fix conflicts with main branch 2021-05-20 14:39:27 +03:00
Konstantin Knizhnik
20b6279beb Fix restoring non-relational data during compute node startup 2021-05-20 14:14:52 +03:00
Konstantin Knizhnik
06f96f9600 Do not transfer WAL to computation nodes: use pg_resetwal for node startup 2021-05-20 14:13:47 +03:00
Alexey Kondratov
b5f60f3874 Issue #144: Refactor errors handling during branches tree printing 2021-05-20 12:49:04 +03:00
Alexey Kondratov
0ec56cd21f Issue #144: Branching output of zenith branch
* Add ancestor_id to pg_list->branch_list output of pageserver.
* Display branching point (LSN) for each non-root branch.
* Add tests for `zenith branch`.
2021-05-20 12:49:04 +03:00
Heikki Linnakangas
600e1a0080 Pass PageServerConf as static ref.
It's created once early in server startup, after parsing the
command-line options, and never modified afterwards. To simplify
things, pass it around as static ref, instead of making copies in all
the different structs. We still pass around a reference to it, rather
than putting it in a global variable, to allow unit testing with
different configs in the same process.
2021-05-20 09:11:36 +03:00
Eric Seppanen
9c94a34ae7 cargo fmt 2021-05-19 15:26:21 -07:00
Stas Kelvich
9c0ac251df Describe BeMessage::ErrorResponse format in comments 2021-05-20 00:37:46 +03:00
Stas Kelvich
872ed24408 Manually handle all error throwing for branch and init commands 2021-05-20 00:37:46 +03:00
Stas Kelvich
2f25d17e11 Set more error fields to satisfy rust-postgres parser 2021-05-20 00:37:46 +03:00
Stas Kelvich
8faa6fa392 Accept semicolon right after branch_create command 2021-05-20 00:37:46 +03:00
Stas Kelvich
4d5a41301d Support returning errors from page service 2021-05-20 00:37:46 +03:00
Eric Seppanen
4c35b22626 Remove FIXME about buffer pools
If I'm not going to do anything about it soon, it's not worth keeping
this comment.
2021-05-19 14:36:41 -07:00
Eric Seppanen
9fe3b73e13 walkeeper replication: remove the lock from the send stream.
I originally thought there would be multiple threads sending here, but
that's not currently the case, so remove the lock.
2021-05-19 14:36:41 -07:00
Eric Seppanen
e0146304e6 timeline: make SharedState and some constructors private
This was pointed out in code review: no need for these to be public.
2021-05-19 14:36:41 -07:00
Eric Seppanen
fbb04c592a wal_service: change error message at thread exit
Because many errors are propagated to this point, use a better message
than "socket error".
2021-05-19 14:36:41 -07:00
Eric Seppanen
8f43d7637c wal_service: move code around some more
Move ReceiveWalConn into its own file. Shuffle constants around so they
are close to the protocol they're associated with, or move them into
postgres_ffi if they seem to be global constants.
2021-05-19 14:36:41 -07:00
Eric Seppanen
cf30303d8f extract protocol peek code; rename Connection -> ReceiveWalConn
It may be more robust to use the TcpStream::peek function, so do all
protocol peeking before creating the protocol object. This reveals the
next cleanup step: rename Connection, since it's no longer the parent of
SendWalConn. Now we peek at the first bytes and choose which kind of
connection object to create.
2021-05-19 14:36:41 -07:00
Eric Seppanen
1ec157653e bin_ser: expand serialize error type, add serialized_size 2021-05-19 14:36:41 -07:00
Eric Seppanen
858ca3a4ce bin_ser: simplify ser_into_slice
The conversion of &mut [u8] into Write is a little tricky.

Also, remove an unused generic parameter.
2021-05-19 14:36:41 -07:00
Eric Seppanen
d744ddee7c bin_ser: preserve IO errors on deserialization
We're starting to deserialize directly from the TcpStream now, which
means that a socket error gets logged as "deserialize error". That's not
very helpful; preserve the io::Error so it can be logged.
2021-05-19 14:36:41 -07:00
Eric Seppanen
3296b7d770 wal_service: permit I/O errors while reading control file
I'm not sure why ignoring this error is a good idea, but the
test_embedded_wal_proposer test fails if we propagate the error upward.
2021-05-19 14:36:41 -07:00
Eric Seppanen
2148ae78ab wal_service: remove manual output buffering
Serialize objects directly to the stream. This allows us to remove a
bunch of buffer management code, along with the NewSerializer trait that
was a temporary bridge between the old code and the new.
2021-05-19 14:36:41 -07:00
Eric Seppanen
78dcf2207e replace manual deserialization with serde + BeSer
This struct is a little awkward, because in other places it is
serialized/deserialized as little-endian, but here it's big-endian.
2021-05-19 14:36:41 -07:00
Eric Seppanen
74b78608d9 split timeline code into a separate file 2021-05-19 14:36:41 -07:00
Eric Seppanen
a11558b84f break wal_service into multiple files
+ misc cleanups
2021-05-19 14:36:41 -07:00
Eric Seppanen
513696a485 break wal_service into multiple pieces
The pieces are:
base Connection
SendWal
ReplicationHandler

There are lots of other changes here:
- Put the replication reader in a background thread; this gets rid
  of some hacks with nonblocking mode.
- Stop manually buffering input data; use BufReader instead.
- Use BytesMut a lot less; use Read/Write traits where possible.
2021-05-19 14:36:41 -07:00
Eric Seppanen
cedc2eb5c2 wal_service: add BufReader
If we try to read a few bytes at a time, we will perform a lot more
syscalls than necessary. Wrap the socket in a BufReader, which will
buffer bytes as needed.
2021-05-19 14:36:41 -07:00
Heikki Linnakangas
e3e593f571 Don't send spurious ReadyForQuery messages in extended query protocol.
libpq tolerates and ignores them, but the Rust postgres client gets
confused by them in certain states. This explained the strange failure
I saw with the Copy Out protocol. I'm not sure what the condition was
exactly, but somehow the rust client got confused if it received a
ReadyForQuery message that it was not expecting.

Fixes https://github.com/zenithdb/zenith/issues/148.
2021-05-19 22:31:28 +03:00