rust/neon
1
0
Fork 0
mirror of https://github.com/neondatabase/neon.git synced 2026-01-16 01:42:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
5,518 Commits 1,487 Branches 282 Tags
compress-proxy-node-cache
Commit Graph

5518 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dmitry Ivanov
96c7594d29 Enable some kind of gradual typing in test_runner (#222) 
It's not realistic to enable full-blown type checks
within test_runner's codebase, since the amount of
warnings revealed by mypy is overwhelming.

Tests are supposed to be easy to use, so we can't
cripple everybody's workflow for the sake of imaginary benefit.

Ultimately, the purpose of this attempt is three-fold:

* Facilitate code navigation when paired with python-language-server.
* Make method signatures apparent to a fellow programmer.
* Occasionally catch some obvious type errors.
 2021-06-10 22:53:15 +03:00
Eric Seppanen
7a3794ef18 tui_event: unwrap result of stdin.keys() 
Clear a clippy warning about manual flatten.

This isn't good error handling, but panicking is probably better than
spinning forever if stdin returns EOF.
 2021-06-10 12:04:52 -07:00
Stas Kelvich
bf56ea8c43 Locate postgres binary and libs for 'postgres --wal-redo' 
based on POSTGRES_DISTRIB_DIR.
 2021-06-09 20:17:27 +03:00
Dmitry Ivanov
bb1446e33a Change behavior of ComputeControlPlane::new_node() (#235) 
Previously, transaction commit could happen regardless of whether
pageserver has caught up or not. This patch aims to fix that.

There are two notable changes:

1. ComputeControlPlane::new_node() now sets the
`synchronous_standby_names = 'pageserver'` parameter to delay
transaction commit until pageserver acting as a standby has
fetched and ack'd a relevant portion of WAL.

2. pageserver now has to:
    - Specify the `application_name = pageserver` which matches the
    one in `synchronous_standby_names`.
    - Properly reply with the ack'd LSNs.

This means that some tests don't need sleeps anymore.

TODO: We should probably make this behavior configurable.

Fixes #187.
 2021-06-09 11:24:55 +03:00
anastasia
0969574d48 Use bindgen for various xlog structures and checkpoint. 
Implement encode/decode methods for them.

Some methods are unused now. This is a preparatory commit for nonrel_wal
 2021-06-09 01:00:42 +03:00
anastasia
05a681be2c add createuser test to test shared catalog restore 2021-06-09 00:31:09 +03:00
anastasia
507c1fbbac bump vendor/postgres 2021-06-08 17:46:43 +03:00
Arseny Sher
b2f51026aa Consolidate PG proto parsing-deparsing and backend code. 
Now postgres_backend communicates with the client, passing queries to the
provided handler; we have two currently, for wal_acceptor and pageserver.

Now BytesMut is again used for writing data to avoid manual message length
calculation.

ref #118
 2021-06-08 17:31:40 +03:00
anastasia
2b0193e6bf implement from_bytes for XLogPageHeader structs 2021-06-08 13:08:57 +03:00
anastasia
c31a5e2c8f move XLogPageHeader structs to xlog_utils 2021-06-08 13:08:57 +03:00
anastasia
d85d67a6f1 use constants defined in xlog_utils for waldecoder 2021-06-08 13:08:57 +03:00
anastasia
9b8e82c6cf Fix makefile for out-of-tree compiling of contrib/zenith 2021-06-07 19:24:20 +03:00
Heikki Linnakangas
434374ebb4 Turn encode/decode into methods 
Like in PR #208
 2021-06-04 23:05:30 +03:00
Heikki Linnakangas
a7ae552851 Use rust memoffset crate to replace C offsetof(). 
Cherry-picked from Eric's PR #208
 2021-06-04 23:05:28 +03:00
Heikki Linnakangas
8b5a061c8e Add comments on the unsafe use of transmute in encode/decode_pg_control 
Note the unsafety of the unsafe block, with a link to the ongoing
discussion. This doesn't try to solve the problem, but let's at least
document the status quo.
 2021-06-04 23:05:26 +03:00
Heikki Linnakangas
8147aa7e93 Use u8 slice instead of Bytes in function argument. 
Bytes is handy, but in decode_pg_control's case it's just complicating
things. Also, pass ControlFileData by ref to encode_pg_control().
 2021-06-04 23:05:20 +03:00
Heikki Linnakangas
d18cc8a3a8 Update 'postgres_ffi' module's readme file and comments. 
Explain the purpose of of the 'postgres_ffi' module, explain what the
PostgreSQL control file is, and some other minor cleanup.
 2021-06-04 23:05:11 +03:00
Heikki Linnakangas
762e9859d6 Move functions for reading/writing control file to separate source file. 
To follow the precedent of xlog_utils.rs and relfile_utils.rs.
 2021-06-04 23:05:05 +03:00
Heikki Linnakangas
924261f7db Remove unused ControlFile::new() constructor. 
It has never been used, AFAICS.
 2021-06-04 23:05:02 +03:00
Konstantin Knizhnik
063429aade Implement GC for new object_store API (#229) 
* Implement GC for new object_store API

* Add comments for GC

* Revert postgres module version reference
 2021-06-04 20:11:56 +03:00
anastasia
445e88f50b Minor fix: preserve log of regression tests in circleci 2021-06-03 15:54:59 +03:00
Patrick Insinger
47694ea4f5 zenith push 2021-06-02 17:20:49 -04:00
Patrick Insinger
3364a8d442 pageserver - timeline history api 2021-06-02 16:20:26 -04:00
Dmitry Ivanov
f2243d7459 Merge pull request #219 from zenithdb/tidy-up-tests 
Tidy up pytest-based tests
 2021-06-01 22:06:13 +03:00
Dmitry Ivanov
244fcffc50 Fix typos found by codespell 2021-06-01 21:43:26 +03:00
Dmitry Ivanov
8c3c9c3394 Update README.md 2021-06-01 21:31:29 +03:00
Dmitry Ivanov
00ce635da9 Reformat tests using yapf 2021-06-01 21:09:09 +03:00
Dmitry Ivanov
7d5f7462c1 Tidy up pytest-based tests 2021-06-01 21:09:09 +03:00
Alexey Kondratov
bed75f800f Bump vendor/postgres to use rebased on top of REL_14_BETA1 version 2021-06-01 20:04:22 +03:00
anastasia
0e423d481e Update rustdoc comments and README for pageserver crate 2021-06-01 19:38:42 +03:00
anastasia
0c74f6fa4e Update README about source tree layout 2021-06-01 19:38:42 +03:00
Heikki Linnakangas
fc01fae9b4 Remove leftover references to safekeeper_proxy. 
We don't use it anymore. The WAL proposer is now a background worker that
runs as part of the primary Postgres server.
 2021-06-01 18:50:24 +03:00
Heikki Linnakangas
558a2214bc Fix comment 2021-06-01 18:28:01 +03:00
Heikki Linnakangas
31815bccc0 Remove obsolete script. 
It was used to launch the page server a long time ago. Doesn't work,
and isn't needed anymore.
 2021-06-01 18:27:54 +03:00
Stas Kelvich
e12cab7c17 fix dockerfile comments 2021-06-01 17:07:10 +03:00
Stas Kelvich
002cd8ed5b Dockerfile for pageserver. 2021-06-01 16:08:32 +03:00
Stas Kelvich
588a030803 Clean up control_plane's Cargo.toml 2021-06-01 16:08:32 +03:00
Stas Kelvich
0b9bbdc4ec Remove libxml2 and curl from postgres dependencies. 
As lazyrestore was removed from our postgres main branch.
 2021-06-01 16:08:32 +03:00
Heikki Linnakangas
922715a923 Use Timeline::get_last_record_lsn() to find end-of-WAL. 
That's less hacky than scanning the WAL directory.
 2021-06-01 09:37:54 +03:00
Heikki Linnakangas
ac60b68d50 Handle VM and FSM truncation WAL records in the page server. 
Fixes issue #190.

Original patch by Konstantin Knizhnik.
 2021-05-31 23:36:17 +03:00
Konstantin Knizhnik
1aceea1bdd Shutdown socket in ReplicationConn 2021-05-31 21:37:07 +03:00
Konstantin Knizhnik
e0cc4dee4f [refer #182] Make walkeeper periodically send callme requests to pageserver 2021-05-31 21:37:07 +03:00
Arseny Sher
fd20101e5c Configure pipenv for python tests. 2021-05-31 16:43:01 +03:00
anastasia
5a73a6fdfc add -w flag to wait till pg_ctl actually finishes what was asked 2021-05-28 20:33:16 +03:00
Stas Kelvich
4608b1ec70 Set wal_log_hints=on 
That is mandatory to correctly maintain visibility map (see issue#192).
It also makes sense to check that wal_log_hints is enabled at the pageserver side,
but for now let just check that tests will pass with this on.
 2021-05-28 11:38:46 +03:00
anastasia
ccb2eea7fd Add test_isolation that runs pg_isolation_regress for zenith 2021-05-28 11:38:46 +03:00
Heikki Linnakangas
6b615cbde1 Remove Copy marker from large ServerInfo struct. 
We don't want to encourage passing it by value. Doesn't matter much in
practice, but let's be tidy.

Per discussion at https://github.com/zenithdb/zenith/pull/195#issuecomment-849897327
 2021-05-27 23:16:54 +03:00
Heikki Linnakangas
adc0e04205 Misc cleanup of the 'zenith_regress' tests 
- Remove serial_schedule. As was alluded to in the README, it's really
  quote pointless.
- Remove unused PORT/HOST variables
- Fix typos
 2021-05-27 23:12:02 +03:00
Heikki Linnakangas
78e3edf2e9 Add CONTRIBUTING.md with some ground rules for submitting PRs. 
Let's try to rachet up our code quality, but at the same time keep the
bureaucracy low.
 2021-05-27 23:07:37 +03:00
Heikki Linnakangas
4b6563a55d Little cleanup on save_decoded_record function 
Pass DecodedWALRecord by reference, seems nicer, and probably marginally
faster.
 2021-05-27 20:50:45 +03:00