Bojan Serafimov
9ee5f17c41
Fix test
2022-06-29 17:40:48 -04:00
Bojan Serafimov
170c882c84
Check that tasks finish
2022-06-29 16:27:22 -04:00
Bojan Serafimov
a5bacd42f5
Merge branch 'main' into tenant-tasks-test
2022-06-29 14:52:33 -04:00
Bojan Serafimov
bfeaa4957f
WIP
2022-06-29 14:24:22 -04:00
Bojan Serafimov
633b1762f6
Fix metric name
2022-06-29 11:39:15 -04:00
Dmitry Ivanov
5ee19b0758
Fix bloated coverage uploads ( #2005 )
...
Move coverage data to a better directory, merge it better and don't publish it from CircleCI pipeline
2022-06-29 17:59:19 +03:00
Kirill Bulatov
cef90d9220
Disable cachepot for GH Actions builds ( #2007 )
2022-06-29 17:56:02 +03:00
Bojan Serafimov
2b4c3cb932
Handle errors on shutdown
2022-06-28 09:29:59 -04:00
Bojan Serafimov
bf76f43ea4
Wait for tasks to complete
2022-06-27 17:54:31 -04:00
Kirill Bulatov
4a05413a4c
More code coverage fixes in GH Actions ( #2002 )
2022-06-27 22:40:20 +03:00
Kirill Bulatov
dd61f3558f
Fix coverage upload credentials retrieval ( #2001 )
2022-06-27 20:41:09 +03:00
Kirill Bulatov
8a714f1ebf
Add coverage to GH actions and rework part of them ( #1987 )
2022-06-27 19:15:56 +03:00
Arseny Sher
137291dc24
Push to etcd from safekeeper many timelines concurrently.
...
Mitigates latency fee, making push throughput 1-1.5 order of magnitude bigger.
Also make leases per timeline, not per whole safekeeper, avoiding storing
garbage in etcd for deleted timelines while safekeeper is alive.
2022-06-27 16:30:21 +03:00
Kirill Bulatov
eb8926083e
Use the updated base build Docker image ( #1972 )
2022-06-27 13:12:58 +03:00
Johan Eliasson
26bca6ddba
Add openssl to OSX dependencies ( #1994 )
2022-06-26 21:54:07 +03:00
Bojan Serafimov
98062865f4
Handle errors inside loop
2022-06-24 17:08:44 -04:00
Bojan Serafimov
cdc81996b4
Add metrics
2022-06-24 15:28:53 -04:00
Bojan Serafimov
1169e9ea4c
Rename threads to tasks
2022-06-24 11:08:55 -04:00
Bojan Serafimov
3a23869780
Log errors
2022-06-24 11:05:07 -04:00
Bojan Serafimov
796ee4d8af
Instrument the task, not the await
2022-06-24 10:29:42 -04:00
Bojan Serafimov
b31ce411d2
Remove unnecessary map_err
2022-06-24 09:50:14 -04:00
Arthur Petukhovsky
55192384c3
Fix zero timeline_start_lsn ( #1981 )
...
* Fix zero timeline_start_lsn
* Log more info on control file upgrade
* Fix formatting
Co-authored-by: Anastasia Lubennikova <anastasia@neon.tech >
2022-06-24 13:59:37 +03:00
KlimentSerafimov
392cd8b1fc
Refactored extracting project_name in console.rs. ( #1982 )
2022-06-24 05:57:33 -04:00
Bojan Serafimov
24a5bd10a0
Add cancellation
2022-06-23 23:25:48 -04:00
Bojan Serafimov
2c029d9803
Remove redundant error context
2022-06-23 22:32:15 -04:00
Alexey Kondratov
3cc531d093
Fix CREATE EXTENSION for non-db-owner users ( #1408 )
...
Previously, we were granting create only to db owner, but now we have a
dedicated 'web_access' role to connect via web UI and proxy link auth.
We anyway grant read / write all data to all roles, so let's grant
create to everyone too. This creates some provelege objects in each db,
which we need to drop before deleting the role. So now we reassign all
owned objects to each db owner before deletion. This also fixes deletion
of roles that created some data in any db previously. Will be tested by
https://github.com/neondatabase/cloud/pull/1673
Later we should stop messing with Postgres ACL that much.
2022-06-23 21:36:53 +02:00
Bojan Serafimov
763b00ccee
Merge branch 'tenant-tasks' of github.com:neondatabase/neon into tenant-tasks
2022-06-23 14:15:14 -04:00
Bojan Serafimov
c44c8a0ea0
Add comment
2022-06-23 14:14:39 -04:00
Bojan Serafimov
692496d733
Cancel tasks
2022-06-23 13:20:53 -04:00
Bojan Serafimov
0f4552a544
Update TODO
2022-06-23 12:59:20 -04:00
Bojan Serafimov
d7d4cc8c77
Error instead of panic
2022-06-23 12:56:21 -04:00
bojanserafimov
84b9fcbbd5
Increase a few test timeouts ( #1977 )
2022-06-23 11:51:56 -04:00
Bojan Serafimov
93e050afe3
Don't require project name for link auth
2022-06-23 15:38:05 +03:00
Anastasia Lubennikova
6d7dc384a5
Add zenith-us-stage-ps-3 to deploy
2022-06-23 14:52:32 +03:00
Anastasia Lubennikova
3c2b03cd87
Update timeline size on dropdb. Add the test ( #1973 )
...
In addition, fix database size calculation:
count not only main fork of the relation, but also vm and fsm.
2022-06-23 12:28:12 +03:00
Kirill Bulatov
7c49abe7d1
Rework etcd timeline updates and their handling
2022-06-23 09:11:27 +03:00
Bojan Serafimov
9aab1d0f2b
Expand blocking scope
2022-06-22 16:04:42 -04:00
Bojan Serafimov
83dc93ab0f
Merge branch 'main' into tenant-tasks
2022-06-22 14:49:45 -04:00
KlimentSerafimov
d059e588a6
Added invariant check for project name. ( #1921 )
...
Summary: Added invariant checking for project name. Refactored ClientCredentials and TlsConfig.
* Added formatting invariant check for project name:
**\forall c \in project_name . c \in [alnum] U {'-'}.
** sni_data == <project_name>.<common_name>
* Added exhaustive tests for get_project_name.
* Refactored TlsConfig to contain common_name : Option<String>.
* Refactored ClientCredentials construction to construct project_name directly.
* Merged ProjectNameError into ClientCredsParseError.
* Tweaked proxy tests to accommodate refactored ClientCredentials construction semantics.
* [Pytests] Added project option argument to test_proxy_select_1.
* Removed project param from Api since now it's contained in creds.
* Refactored &Option<String> -> Option<&str>.
Co-authored-by: Dmitrii Ivanov <dima@neon.tech >.
2022-06-22 09:34:24 -04:00
Sergey Melnikov
6222a0012b
Migrate from CircleCI to Github Actions: python codestyle, build and tests ( #1647 )
...
Duplicate postgres and neon build and test jobs from CircleCI to Github actions.
2022-06-22 11:40:59 +03:00
bojanserafimov
1ca28e6f3c
Import basebackup into pageserver ( #1925 )
...
Allow importing basebackup taken from vanilla postgres or another pageserver via psql copy in protocol.
2022-06-21 11:04:10 -04:00
Arthur Petukhovsky
6c4d6a2183
Remove timeline_start_lsn check temporary. ( #1964 )
2022-06-21 02:02:24 +03:00
Thang Pham
37465dafe3
Add wal backpressure tests ( #1919 )
...
Resolves #1889 .
This PR adds new tests to measure the WAL backpressure's performance under different workloads.
## Changes
- add new performance tests in `test_wal_backpressure.py`
- allow safekeeper's fsync to be configurable when running tests
2022-06-20 11:40:55 -04:00
Joshua D. Drake
ec0064c442
Small README.md changes ( #1957 )
...
* Update make instructions for release and debug build. Update PostgreSQL glossary to proper version (14)
* Continued cleanup of build instructions including removal of redundancies
2022-06-20 10:05:10 -04:00
Heikki Linnakangas
83c7e6ce52
Bump vendor/postgres.
...
This brings in the change to not use a shared memory in the WAL redo
process, to avoid running out of sysv shmem segments in the page server.
Also, removal of callmemaybe bits.
2022-06-20 15:28:43 +03:00
Arthur Petukhovsky
f862373ac0
Fix WAL timeout in test_s3_wal_replay ( #1953 )
2022-06-17 20:43:54 +03:00
Arthur Petukhovsky
699f46cd84
Download WAL from S3 if it's not available in safekeeper dir ( #1932 )
...
`send_wal.rs` and `WalReader` are now async. `test_s3_wal_replay` checks that WAL can be replayed after offloaded.
2022-06-17 15:33:39 +03:00
Anastasia Lubennikova
36ee182d26
Implement page servise 'fullbackup' endpoint ( #1923 )
...
* Implement page servise 'fullbackup' endpoint that works like basebackup, but also sends relational files
* Add test_runner/batch_others/test_fullbackup.py
Co-authored-by: bojanserafimov <bojan.serafimov7@gmail.com >
2022-06-16 14:07:11 +03:00
Anastasia Lubennikova
d11c9f9fcb
Use random ports for the proxy and local pg in tests
...
Fixes #1931
Author: Dmitry Ivanov
2022-06-15 20:21:58 +03:00
Bojan Serafimov
9a9a58d52c
Fmt
2022-06-15 09:49:07 -04:00
