Commit Graph

1388 Commits

Author SHA1 Message Date
Heikki Linnakangas
2c5fb6d6c8 Change 'relsize_inc' signature to be a bit nicer.
Don't add 1 to the argument in the function, the callers must do it now.
And don't accept None argument, pass 0 instead for an empty relation.
2021-04-20 19:10:37 +03:00
Konstantin Knizhnik
936cad17e4 LSN-aware smgrnblock/smgrexists implementations 2021-04-20 18:28:35 +03:00
Konstantin Knizhnik
95160dee6d Merge with main branch 2021-04-19 17:00:30 +03:00
Konstantin Knizhnik
8aa3013ec2 Merge branch 'main' into rocksdb_pageserver 2021-04-19 16:28:29 +03:00
Eric Seppanen
8d1bf152cf fix up logged error for walreceiver connection failed
For some reason printing the Result made the error string print twice,
along with some annoying newlines. Extracting the error first gets the
expected result (just one explanation, no newlines)
2021-04-18 23:06:35 -07:00
Eric Seppanen
3725815935 pageserver: propage errors instead of calling .unwrap()
Just a few more places where we can drop the .unwrap() call in favor of
`?`.

Also include a fix to the log file handling: don't open the file twice.
Writing to two fds would result in one message overwriting another.
Presumably `File.try_clone()` reduces down to `dup` on Linux.
2021-04-18 23:06:35 -07:00
Eric Seppanen
b32cc6a088 pageserver: change over some error handling to anyhow+thiserror
This is a first attempt at a new error-handling strategy:
- Use anyhow::Error as the first choice for easy error handling
- Use thiserror to generate local error types for anything that
  needs it (no error type is available to us) or will be inspected
  or matched on by higher layers.
2021-04-18 23:06:35 -07:00
Eric Seppanen
3c7f810849 clippy cleanup #1
Resolve some basic warnings from clippy:
- useless conversion to the same type
- redundant field names in struct initialization
- redundant single-component path imports
2021-04-18 19:15:06 -07:00
Konstantin Knizhnik
1e65848551 Add pgbench test 2021-04-18 13:13:13 +03:00
Eric Seppanen
e03417a7c9 suppress dead_code warnings on nightly
We don't need the nightly compiler, but there's no reason it shouldn't
compile without warnings, either. I don't know why stable doesn't warn
about these, but it's cheap to suppress them.
2021-04-17 14:14:27 -07:00
Konstantin Knizhnik
33ee5b6ba0 Skip truncate records when calculating relation size 2021-04-17 15:54:57 +03:00
Eric Seppanen
52d6275812 drop nonfunctional attributes allow(dead_code)
These had no effect, so remove them.
2021-04-16 15:59:32 -07:00
Eric Seppanen
35e0099ac6 pin remote rust-s3 dependency to a git hash
Using the hash should allow us to change the remote repo and propagate
that change to user builds without that change becoming visible at a
random time.

It's unfortunate that we can't declare this dependency once in the
top-level Cargo.toml; that feature request is rust-lang rfc 2906.
2021-04-16 15:26:11 -07:00
Eric Seppanen
2246b48348 handle keepalive messages
When postgres sends us a keepalive message, send a reply so it doesn't
time out and close the connection.

The LSN values sent may need to change in the future. Currently we send:
write_lsn <= page_cache last_valid_lsn
flush_lsn <= page_cache last_valid_lsn
apply_lsn <= 0
2021-04-16 08:29:47 -07:00
Eric Seppanen
e8032f26e6 adopt new tokio-postgres:replication branch
This PR has evolved a lot; jump to the newer version. This should make
it easier to handle keepalive messages.
2021-04-16 08:29:47 -07:00
Heikki Linnakangas
d2c3ad162a Prefer passing PageServerConf by reference.
It seems more idiomatic Rust.
2021-04-16 10:42:41 +03:00
Heikki Linnakangas
b4c5cb2773 Clean up error types a little bit.
Don't use std::io::Error for errors that are not I/O related. Prefer
anyhow::Result instead.
2021-04-16 10:42:25 +03:00
Konstantin Knizhnik
b67df00bff Fix bug in decoding of truncate record 2021-04-15 17:09:30 +03:00
anastasia
05886b33e5 fix typo 2021-04-15 16:43:03 +03:00
anastasia
d7eeaec706 add test for restore from local pgdata 2021-04-15 16:43:03 +03:00
anastasia
1190030872 handle SLRU in restore_datadir 2021-04-15 16:43:03 +03:00
Konstantin Knizhnik
24b925d528 Support truncate WAL record 2021-04-15 15:50:47 +03:00
lubennikovaav
82dc1e82ba Restore pageserver from s3 or local datadir (#9)
* change pageserver --skip-recovery option to --restore-from=[s3|local]
* implement restore from local pgdata 
* add simple test for local restore
2021-04-14 21:14:10 +03:00
anastasia
2e9c730dd1 Cargo fmt pass 2021-04-14 20:12:50 +03:00
Heikki Linnakangas
6266fd102c Avoid short writes if a buffer is full.
write_buf() tries to write as much as it can in one go, and can return
without writing the whole buffer. We need to use write_all() instead.
2021-04-14 18:18:38 +03:00
Konstantin Knizhnik
d9bc2109bb Reduce size of shared buffers for wal-redo-postgtes 2021-04-14 14:33:55 +03:00
Eric Seppanen
3c4ebc4030 init_logging: return Result, print error on file create
Instead of panicking if the file create fails, print the filename and
error description to stderr; then propagate the error to our caller.
2021-04-13 14:06:14 -07:00
Konstantin Knizhnik
a606336074 Fix bug in WALRecord serializer 2021-04-09 20:31:34 +03:00
Konstantin Knizhnik
1816c4ca0a Merge with master 2021-04-09 15:00:33 +03:00
Konstantin Knizhnik
07fb30747a Store pageserver data in RocksDB 2021-04-08 19:39:30 +03:00
Heikki Linnakangas
0c6471ca0d Print a few more LSNs in the standard format. 2021-04-07 19:05:28 +03:00
Heikki Linnakangas
198fc9ea53 Capture initdb's stdout/stderr, to avoid messing with log formatting.
Especially with --interactive.
2021-04-07 18:51:34 +03:00
Heikki Linnakangas
6b9fc3aff0 Fix minor typos and copy-pastos 2021-04-07 16:39:37 +03:00
Stas Kelvich
c0fcbbbe0c Cargo fmt pass over a codebase 2021-04-06 14:42:13 +03:00
Heikki Linnakangas
494b95886b Move launch.sh into 'pageserver' directory.
It's a script to launch the Page Server.
2021-04-06 14:05:43 +03:00
Heikki Linnakangas
1367332447 Separate walkeeper and pageserver sources into different directories.
The integration tests, which depend on both walkeeper and pageserver,
are moved into yet another directory, 'integration_tests'.
2021-04-06 13:15:26 +03:00
Stas Kelvich
e2ce9e562e remove unused modules 2021-04-02 10:38:51 +03:00
Arun Sharma
253fc123e7 Initial repo structure 2021-03-26 10:54:25 -07:00