Arseny Sher
5865f85ae2
Add --id argument to safekeeper setting its unique u64 id.
...
In preparation for storage node messaging. IDs are supposed to be monotonically
assigned by the console. In tests it is issued by ZenithEnv; at the zenith cli
level and fixtures, string name is completely replaced by integer id. Example
TOML configs are adjusted accordingly.
Sequential ids are chosen over Zid mainly because they are compact and easy to
type/remember.
2022-02-23 08:33:50 +03:00
Dhammika Pathirana
b815f5fb9f
Add no_sync check in storage
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-02-22 12:01:12 -08:00
anastasia
74a0942a77
Fix zenith feedback processing at compute node.
...
Add test for backpressure
2022-02-22 13:56:21 +03:00
anastasia
1a4682a04a
Add 'walreceiver-after-ingest' failpoint. Use sleep at this point to imitate slow walreceiver.
2022-02-22 13:56:21 +03:00
Heikki Linnakangas
993b544ad0
Change default parameters for back pressure
...
Fixes issue #1238 and #1189 . Extracted from PR #1194 , with some comment
editorialization by me.
Author: Konstantin Knizhnik <knizhnik@zenith.tech >
2022-02-22 13:56:21 +03:00
Arthur Petukhovsky
dba1d36a4a
Refactor WAL utils in safekeeper ( #1290 )
...
wal_storage.rs was split up from timeline.rs, safekeeper.rs and send_wal.rs,
and now contains all WAL related code from the safekeeper. Now there are
PhysicalStorage for persisting WAL to disk and WalReader for reading it.
This allows optimizing PhysicalStorage without affecting too much of other
code.
Also there is a separate structure for persisting control file now in
control_file.rs.
2022-02-21 17:20:53 +03:00
Bojan Serafimov
ca81a550ef
Fmt
2022-02-21 16:43:28 +03:00
Bojan Serafimov
65a0b2736b
Add static router
2022-02-21 16:43:28 +03:00
Bojan Serafimov
cca886682b
Undo cplane change
2022-02-21 16:43:28 +03:00
Bojan Serafimov
c8f47cd38e
Fix param name
2022-02-21 16:43:28 +03:00
Bojan Serafimov
92787159f7
Add client auth method option
2022-02-21 16:43:28 +03:00
anastasia
abb422d5de
Fix SafekeeperMetrics parsing in python tests
2022-02-21 13:45:22 +03:00
bojanserafimov
fdc15de8b2
Add perf test: test_random_writes ( #1292 )
2022-02-18 15:46:29 -05:00
Stas Kelvich
207286f2b8
Actualize branching parts of openapi spec.
...
Previous version of spec caused parsing errors in generated clients
as return type is object not array, also one field was missing. In
a passing set `format: hex` on ancestor_id too as value conforms to
that format.
2022-02-18 20:22:21 +02:00
Dhammika Pathirana
d2b896381a
Add safekeeper tenant tags in lsn/wal metrics
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
Add tenant_id in lsn/wal metrics (#1234 )
2022-02-18 08:26:37 -08:00
Dhammika Pathirana
009f6d4ae8
Fix safekeeper metric tags
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
Use separate tags in sk storage file histo (#1234 )
2022-02-18 08:26:37 -08:00
Kirill Bulatov
1b31379456
Log postgres errors with ERROR level
2022-02-17 13:42:09 +02:00
Bojan Serafimov
4c64b10aec
Revert removal of ignore hint
2022-02-17 13:41:49 +02:00
Bojan Serafimov
ad262a46ad
Remove redundant pytest_plugins assignment
2022-02-17 13:41:49 +02:00
Kirill Bulatov
ce533835e5
Use uuid.UUID types for tenants and timelines more
2022-02-17 13:41:19 +02:00
Kirill Bulatov
e5bf520b18
Use types in zenith cli invocations in Python tests
2022-02-17 13:41:19 +02:00
Dmitry Rodionov
9512e21b9e
fix python formatting
2022-02-17 13:22:14 +03:00
Dmitry Ivanov
a26d565282
[proxy] Replace private static map with a public CancelMap
...
This is a cleaner approach which might facilitate testing.
2022-02-17 11:54:27 +03:00
Dmitry Ivanov
a47dade622
[proxy] Migrate to async
...
This change makes most parts of the code asynchronous, except
for the `mgmt` subsystem (we're going to drop it anyway).
Co-authored-by: bojanserafimov <bojan.serafimov7@gmail.com >
2022-02-17 11:54:27 +03:00
Dmitry Rodionov
9cce430430
remove several obsolete management api commands from pageserver's libpq
...
api
these commands are now available via http api
2022-02-17 11:26:28 +03:00
Dhammika Pathirana
4bf4bacf01
Add cli start/stop test
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
Add a test for #1260
2022-02-16 13:19:12 -08:00
bojanserafimov
335abfcc28
Add slow seqscan perf test ( #1283 )
2022-02-16 10:59:51 -05:00
bojanserafimov
afb3342e46
Add vanilla pg baseline tests ( #1275 )
2022-02-15 13:44:22 -05:00
Kirill Bulatov
5563ff123f
Reuse tenant-timeline id struct from utils
2022-02-15 17:45:23 +02:00
Dhammika Pathirana
0a557b2fa9
Add cli v4 loopback listener ports test
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
Add a test for #1247
2022-02-15 17:01:22 +02:00
Heikki Linnakangas
9632c352ab
Avoid having multiple records for the same page and LSN.
...
If a heap UPDATE record modified two pages, and both pages needed to have
their VM bits cleared, and the VM bits were located on the same VM page,
we would emit two ZenithWalRecord::ClearVisibilityMapFlags records for
the same VM page. That produced warnings like this in the pageserver log:
Page version Wal(ClearVisibilityMapFlags { heap_blkno: 18, flags: 3 }) of rel 1663/13949/2619_vm blk 0 at 2A/346046A0 already exists
To fix, change ClearVisibilityMapFlags so that it can update the bits
for both pages as one operation.
This was already covered by several python tests, so no need to add a
new one. Fixes #1125 .
Co-authored-by: Konstantin Knizhnik <knizhnik@zenith.tech >
2022-02-15 14:26:16 +02:00
Arseny Sher
328e3b4189
bump vendor/postgres to fix compiler warnings
2022-02-15 06:51:16 +03:00
Arseny Sher
47f6a1f9a8
Add -Werror to CI builds.
2022-02-15 06:51:16 +03:00
Dmitry Rodionov
a4829712f4
merge directories in git-upload instead of removing existing files for perf test result uploads
2022-02-15 03:47:06 +03:00
Arseny Sher
d4d26f619d
bump vendor/postgres to fix compilation warning
2022-02-14 21:00:11 +03:00
Arseny Sher
36481f3374
bump vendor/postgres to init pgxactoff in walproposer
...
ref #1244
2022-02-14 15:57:38 +03:00
Dhammika Pathirana
d951dd8977
Fix cli start ( #1260 )
...
Signed-off-by: Dhammika Pathirana <dhammika@gmail.com >
2022-02-10 18:36:02 -05:00
bojanserafimov
ea13838be7
Add pgbench baseline test ( #1204 )
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2022-02-10 15:33:36 -05:00
Dmitry Rodionov
b51f23cdf0
pass perf test cluster connstr to circle ci jobs
2022-02-10 17:49:54 +03:00
Kirill Bulatov
3cfcdb92ed
Fix tokio features in zenith utils to enable its standalone compilation
2022-02-10 08:33:22 -05:00
Kirill Bulatov
d7af965982
Do not leak decoding_key in JwtAuth's Debug representation
2022-02-10 08:33:22 -05:00
Kirill Bulatov
7c1c7702d2
Code review fixes
2022-02-10 08:33:22 -05:00
Kirill Bulatov
6eef401602
Move routerify behind zenith_utils
2022-02-10 08:33:22 -05:00
Kirill Bulatov
c5b5905ed3
Remove parking_lot dependency from workspace
2022-02-10 08:33:22 -05:00
Kirill Bulatov
76b74349cb
Bump pageserver dependencies
2022-02-10 08:33:22 -05:00
Dmitry Rodionov
b08e340f60
point perf results back from testing to master
2022-02-10 14:18:34 +03:00
Dmitry Rodionov
a25fa29bc9
modify git-upload for generate_and_push_perf_report.sh needs
2022-02-10 13:12:19 +03:00
Dmitry Rodionov
ccf3c8cc30
store performance test results in our staging cluster to be able to
...
visualize them in grafana
2022-02-10 13:12:19 +03:00
Heikki Linnakangas
c45ee13b4e
Bump vendor/postgres, to fix memory leak.
...
See https://github.com/zenithdb/postgres/pull/129
2022-02-10 11:29:38 +02:00
anastasia
f1e7db9d0d
Bump vendor/postgres rebased to 14.2
2022-02-10 11:19:10 +03:00
