Bojan Serafimov
bf76f43ea4
Wait for tasks to complete
2022-06-27 17:54:31 -04:00
Bojan Serafimov
98062865f4
Handle errors inside loop
2022-06-24 17:08:44 -04:00
Bojan Serafimov
cdc81996b4
Add metrics
2022-06-24 15:28:53 -04:00
Bojan Serafimov
1169e9ea4c
Rename threads to tasks
2022-06-24 11:08:55 -04:00
Bojan Serafimov
3a23869780
Log errors
2022-06-24 11:05:07 -04:00
Bojan Serafimov
796ee4d8af
Instrument the task, not the await
2022-06-24 10:29:42 -04:00
Bojan Serafimov
b31ce411d2
Remove unnecessary map_err
2022-06-24 09:50:14 -04:00
Bojan Serafimov
24a5bd10a0
Add cancellation
2022-06-23 23:25:48 -04:00
Bojan Serafimov
2c029d9803
Remove redundant error context
2022-06-23 22:32:15 -04:00
Bojan Serafimov
763b00ccee
Merge branch 'tenant-tasks' of github.com:neondatabase/neon into tenant-tasks
2022-06-23 14:15:14 -04:00
Bojan Serafimov
c44c8a0ea0
Add comment
2022-06-23 14:14:39 -04:00
Bojan Serafimov
692496d733
Cancel tasks
2022-06-23 13:20:53 -04:00
Bojan Serafimov
0f4552a544
Update TODO
2022-06-23 12:59:20 -04:00
Bojan Serafimov
d7d4cc8c77
Error instead of panic
2022-06-23 12:56:21 -04:00
Bojan Serafimov
9aab1d0f2b
Expand blocking scope
2022-06-22 16:04:42 -04:00
Bojan Serafimov
83dc93ab0f
Merge branch 'main' into tenant-tasks
2022-06-22 14:49:45 -04:00
KlimentSerafimov
d059e588a6
Added invariant check for project name. ( #1921 )
...
Summary: Added invariant checking for project name. Refactored ClientCredentials and TlsConfig.
* Added formatting invariant check for project name:
**\forall c \in project_name . c \in [alnum] U {'-'}.
** sni_data == <project_name>.<common_name>
* Added exhaustive tests for get_project_name.
* Refactored TlsConfig to contain common_name : Option<String>.
* Refactored ClientCredentials construction to construct project_name directly.
* Merged ProjectNameError into ClientCredsParseError.
* Tweaked proxy tests to accommodate refactored ClientCredentials construction semantics.
* [Pytests] Added project option argument to test_proxy_select_1.
* Removed project param from Api since now it's contained in creds.
* Refactored &Option<String> -> Option<&str>.
Co-authored-by: Dmitrii Ivanov <dima@neon.tech >.
2022-06-22 09:34:24 -04:00
Sergey Melnikov
6222a0012b
Migrate from CircleCI to Github Actions: python codestyle, build and tests ( #1647 )
...
Duplicate postgres and neon build and test jobs from CircleCI to Github actions.
2022-06-22 11:40:59 +03:00
bojanserafimov
1ca28e6f3c
Import basebackup into pageserver ( #1925 )
...
Allow importing basebackup taken from vanilla postgres or another pageserver via psql copy in protocol.
2022-06-21 11:04:10 -04:00
Arthur Petukhovsky
6c4d6a2183
Remove timeline_start_lsn check temporary. ( #1964 )
2022-06-21 02:02:24 +03:00
Thang Pham
37465dafe3
Add wal backpressure tests ( #1919 )
...
Resolves #1889 .
This PR adds new tests to measure the WAL backpressure's performance under different workloads.
## Changes
- add new performance tests in `test_wal_backpressure.py`
- allow safekeeper's fsync to be configurable when running tests
2022-06-20 11:40:55 -04:00
Joshua D. Drake
ec0064c442
Small README.md changes ( #1957 )
...
* Update make instructions for release and debug build. Update PostgreSQL glossary to proper version (14)
* Continued cleanup of build instructions including removal of redundancies
2022-06-20 10:05:10 -04:00
Heikki Linnakangas
83c7e6ce52
Bump vendor/postgres.
...
This brings in the change to not use a shared memory in the WAL redo
process, to avoid running out of sysv shmem segments in the page server.
Also, removal of callmemaybe bits.
2022-06-20 15:28:43 +03:00
Arthur Petukhovsky
f862373ac0
Fix WAL timeout in test_s3_wal_replay ( #1953 )
2022-06-17 20:43:54 +03:00
Arthur Petukhovsky
699f46cd84
Download WAL from S3 if it's not available in safekeeper dir ( #1932 )
...
`send_wal.rs` and `WalReader` are now async. `test_s3_wal_replay` checks that WAL can be replayed after offloaded.
2022-06-17 15:33:39 +03:00
Anastasia Lubennikova
36ee182d26
Implement page servise 'fullbackup' endpoint ( #1923 )
...
* Implement page servise 'fullbackup' endpoint that works like basebackup, but also sends relational files
* Add test_runner/batch_others/test_fullbackup.py
Co-authored-by: bojanserafimov <bojan.serafimov7@gmail.com >
2022-06-16 14:07:11 +03:00
Anastasia Lubennikova
d11c9f9fcb
Use random ports for the proxy and local pg in tests
...
Fixes #1931
Author: Dmitry Ivanov
2022-06-15 20:21:58 +03:00
Bojan Serafimov
9a9a58d52c
Fmt
2022-06-15 09:49:07 -04:00
Bojan Serafimov
865e8740a7
Add docs
2022-06-15 09:35:41 -04:00
Bojan Serafimov
1a5d1a15d0
Use tokio sleep instead
2022-06-15 09:33:56 -04:00
Bojan Serafimov
02a9883f0f
Add TenantTaskManager
2022-06-15 09:24:46 -04:00
Bojan Serafimov
ee36ca54d5
Run compaction as task
2022-06-15 09:16:57 -04:00
Bojan Serafimov
36cc6d2928
Fmt
2022-06-14 15:07:36 -04:00
Bojan Serafimov
e1a4c06918
Fix init
2022-06-14 15:05:30 -04:00
Bojan Serafimov
ec4528505e
Simplify
2022-06-14 12:38:09 -04:00
Bojan Serafimov
c79e72e835
Add runtime
2022-06-14 11:08:17 -04:00
Bojan Serafimov
a1f85715ac
WIP
2022-06-14 09:39:55 -04:00
Kirill Bulatov
d8a37452c8
Rename ZenithFeedback ( #1912 )
2022-06-11 00:44:05 +03:00
chaitanya sharma
e1336f451d
renamed .zenith data-dir to .neon.
2022-06-09 18:19:18 +02:00
Arseny Sher
a4d8261390
Save Postgres log in test_find_end_of_wal_* tests.
2022-06-09 19:16:43 +04:00
Egor Suvorov
e2a5a31595
Safekeeper HTTP router: add comment about /v1/timeline
2022-06-09 17:14:46 +02:00
Egor Suvorov
0ac0fba77a
test_runner: test Safekeeper HTTP API Auth
...
All endpoints except for POST /v1/timeline are tested, this one is not tested in any way yet.
Three attempts for each endpoint: correctly authenticated, badly authenticated, unauthenticated.
2022-06-09 17:14:46 +02:00
Egor Suvorov
a001052cdd
test_runner: SafekeeperHttpClient: support auth
2022-06-09 17:14:46 +02:00
Egor Suvorov
1f1d852204
ZenithEnvBuilder: rename pageserver_auth_enabled --> auth_enabled
2022-06-09 17:14:46 +02:00
Egor Suvorov
f7b878611a
Implement JWT authentication in Safekeeper HTTP API ( #1753 )
...
* `control_plane` crate (used by `neon_local`) now parses an `auth_enabled` bool for each Safekeeper
* If auth is enabled, a Safekeeper is passed a path to a public key via a new command line argument
* Added TODO comments to other places needing auth
2022-06-09 17:14:46 +02:00
Arseny Sher
a51b2dac9a
Don't s3 offload from newly joined safekeeper not having required WAL.
...
I made the check at launcher level with the perspective of generally moving
election (decision who offloads) there.
Also log timeline 'active' changes.
2022-06-09 18:30:16 +04:00
Thang Pham
e22d9cee3a
fix ZeroDivisionError in scripts/generate_perf_report_page ( #1906 )
...
Fixes the `ZeroDivisionError` error by adding `EPS=1e-6` when doing the calculation.
2022-06-08 09:15:12 -04:00
Arthur Petukhovsky
a01999bc4a
Replace most common remote logs with metrics ( #1909 )
2022-06-08 13:36:49 +03:00
chaitanya sharma
32e64afd54
Use better parallel build instructions in readme.md ( #1908 )
2022-06-08 11:25:37 +03:00
Kirill Bulatov
8a53472e4f
Force etcd broker keys to not to intersect
2022-06-08 11:21:05 +03:00
