rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-01-07 13:32:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
955 Commits 1,487 Branches 282 Tags
100da024b60a5409ea078aaff2a9f648deb965bb
Commit Graph

955 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric Seppanen
533087fd5d cargo fmt 2021-04-19 23:26:13 -07:00
Konstantin Knizhnik
95160dee6d Merge with main branch 2021-04-19 17:00:30 +03:00
Konstantin Knizhnik
8aa3013ec2 Merge branch 'main' into rocksdb_pageserver 2021-04-19 16:28:29 +03:00
Konstantin Knizhnik
8879f747ee Add multitenancy test for wal_acceptor 2021-04-19 14:30:42 +03:00
Heikki Linnakangas
9809613c6f Don't try to read from two WAL files in one read() call. 
That obviously won't work, you have to stop at the WAL file boundary,
and open the next file.
 2021-04-19 10:34:51 +03:00
Eric Seppanen
8d1bf152cf fix up logged error for walreceiver connection failed 
For some reason printing the Result made the error string print twice,
along with some annoying newlines. Extracting the error first gets the
expected result (just one explanation, no newlines)
 2021-04-18 23:06:35 -07:00
Eric Seppanen
3725815935 pageserver: propage errors instead of calling .unwrap() 
Just a few more places where we can drop the .unwrap() call in favor of
`?`.

Also include a fix to the log file handling: don't open the file twice.
Writing to two fds would result in one message overwriting another.
Presumably `File.try_clone()` reduces down to `dup` on Linux.
 2021-04-18 23:06:35 -07:00
Eric Seppanen
b32cc6a088 pageserver: change over some error handling to anyhow+thiserror 
This is a first attempt at a new error-handling strategy:
- Use anyhow::Error as the first choice for easy error handling
- Use thiserror to generate local error types for anything that
  needs it (no error type is available to us) or will be inspected
  or matched on by higher layers.
 2021-04-18 23:06:35 -07:00
Eric Seppanen
3c7f810849 clippy cleanup #1 
Resolve some basic warnings from clippy:
- useless conversion to the same type
- redundant field names in struct initialization
- redundant single-component path imports
 2021-04-18 19:15:06 -07:00
Konstantin Knizhnik
1e65848551 Add pgbench test 2021-04-18 13:13:13 +03:00
Eric Seppanen
e03417a7c9 suppress dead_code warnings on nightly 
We don't need the nightly compiler, but there's no reason it shouldn't
compile without warnings, either. I don't know why stable doesn't warn
about these, but it's cheap to suppress them.
 2021-04-17 14:14:27 -07:00
Konstantin Knizhnik
33ee5b6ba0 Skip truncate records when calculating relation size 2021-04-17 15:54:57 +03:00
Eric Seppanen
52d6275812 drop nonfunctional attributes allow(dead_code) 
These had no effect, so remove them.
 2021-04-16 15:59:32 -07:00
Eric Seppanen
639c9e8266 .gitignore vscode files 2021-04-16 15:36:36 -07:00
Eric Seppanen
35e0099ac6 pin remote rust-s3 dependency to a git hash 
Using the hash should allow us to change the remote repo and propagate
that change to user builds without that change becoming visible at a
random time.

It's unfortunate that we can't declare this dependency once in the
top-level Cargo.toml; that feature request is rust-lang rfc 2906.
 2021-04-16 15:26:11 -07:00
Eric Seppanen
4ff248515b remote unnecessary dependencies between peer crates 
These dependencies make cargo rebuild more than is strictly necessary.
Removing them makes the build a little faster.
 2021-04-16 15:25:43 -07:00
Konstantin Knizhnik
8b70ea4d79 Undo debugg settings for computation node 2021-04-16 22:54:11 +03:00
Eric Seppanen
2246b48348 handle keepalive messages 
When postgres sends us a keepalive message, send a reply so it doesn't
time out and close the connection.

The LSN values sent may need to change in the future. Currently we send:
write_lsn <= page_cache last_valid_lsn
flush_lsn <= page_cache last_valid_lsn
apply_lsn <= 0
 2021-04-16 08:29:47 -07:00
Eric Seppanen
e8032f26e6 adopt new tokio-postgres:replication branch 
This PR has evolved a lot; jump to the newer version. This should make
it easier to handle keepalive messages.
 2021-04-16 08:29:47 -07:00
Heikki Linnakangas
d2c3ad162a Prefer passing PageServerConf by reference. 
It seems more idiomatic Rust.
 2021-04-16 10:42:41 +03:00
Heikki Linnakangas
b4c5cb2773 Clean up error types a little bit. 
Don't use std::io::Error for errors that are not I/O related. Prefer
anyhow::Result instead.
 2021-04-16 10:42:25 +03:00
Konstantin Knizhnik
b67df00bff Fix bug in decoding of truncate record 2021-04-15 17:09:30 +03:00
anastasia
24c3e961e4 Always run cargo build before tests in CI 2021-04-15 16:43:03 +03:00
anastasia
92fb7a1641 don't ignore multitenancy test 2021-04-15 16:43:03 +03:00
anastasia
05886b33e5 fix typo 2021-04-15 16:43:03 +03:00
anastasia
d7eeaec706 add test for restore from local pgdata 2021-04-15 16:43:03 +03:00
anastasia
1190030872 handle SLRU in restore_datadir 2021-04-15 16:43:03 +03:00
anastasia
913a91c541 bump vendor/postgres 2021-04-15 16:20:23 +03:00
Konstantin Knizhnik
24b925d528 Support truncate WAL record 2021-04-15 15:50:47 +03:00
lubennikovaav
82dc1e82ba Restore pageserver from s3 or local datadir (#9) 
* change pageserver --skip-recovery option to --restore-from=[s3|local]
* implement restore from local pgdata 
* add simple test for local restore
 2021-04-14 21:14:10 +03:00
anastasia
2e9c730dd1 Cargo fmt pass 2021-04-14 20:12:50 +03:00
Heikki Linnakangas
6266fd102c Avoid short writes if a buffer is full. 
write_buf() tries to write as much as it can in one go, and can return
without writing the whole buffer. We need to use write_all() instead.
 2021-04-14 18:18:38 +03:00
Konstantin Knizhnik
d9bc2109bb Reduce size of shared buffers for wal-redo-postgtes 2021-04-14 14:33:55 +03:00
Eric Seppanen
d1d6c968d5 control_plane: add error handling to reading pid files 
print file errors to stderr; propagate the io::Error to the caller.
This error isn't handled very gracefully in WalAcceptorNode::drop(),
but there aren't any good options there since drop can't fail.
 2021-04-13 14:30:48 -07:00
Eric Seppanen
3c4ebc4030 init_logging: return Result, print error on file create 
Instead of panicking if the file create fails, print the filename and
error description to stderr; then propagate the error to our caller.
 2021-04-13 14:06:14 -07:00
Eric Seppanen
46543f54a6 pgbuild.sh: halt if a subcommand fails 
This is helpful when first setting up a build machine, just in case
build dependencies are missing.
 2021-04-13 11:45:27 -07:00
Stas Kelvich
b07fa4c896 update readme 2021-04-13 18:58:22 +03:00
Stas Kelvich
f35d13183e fixup, check testing in CI 2021-04-13 18:58:22 +03:00
Stas Kelvich
c5f379bff3 [WIP] Implement CLI pg part 2021-04-13 18:58:22 +03:00
Stas Kelvich
39ebec51d1 Return Result<()> from pageserver start/stop. 
To provide meaningful error messages when it is called by CLI.
 2021-04-10 19:03:40 +03:00
Stas Kelvich
6264dc6aa3 Move control_plane code out of lib.rs and split up control plane 
into compute and storage parts.

Before that code was concentrated in lib.rs which was unhandy to
open by name.
 2021-04-10 13:56:19 +03:00
Stas Kelvich
59163cf3b3 Rework controle_plane code to reuse it in CLI. 
Move all paths from control_plane to local_env which can set them
for testing environment or for local installation.
 2021-04-10 12:09:20 +03:00
Konstantin Knizhnik
a606336074 Fix bug in WALRecord serializer 2021-04-09 20:31:34 +03:00
Konstantin Knizhnik
1816c4ca0a Merge with master 2021-04-09 15:00:33 +03:00
Konstantin Knizhnik
542dffa4a6 Set LD_LIBRARY_PATH for tests 2021-04-08 20:34:24 +03:00
Konstantin Knizhnik
07fb30747a Store pageserver data in RocksDB 2021-04-08 19:39:30 +03:00
Heikki Linnakangas
ba4f8e94aa Minor comment fixes. 2021-04-08 13:30:42 +03:00
Heikki Linnakangas
3e09cb5718 Add README to give an overview of the WAL safekeeper. 2021-04-08 13:30:10 +03:00
Heikki Linnakangas
0c6471ca0d Print a few more LSNs in the standard format. 2021-04-07 19:05:28 +03:00
Heikki Linnakangas
198fc9ea53 Capture initdb's stdout/stderr, to avoid messing with log formatting. 
Especially with --interactive.
 2021-04-07 18:51:34 +03:00