Commit Graph

533 Commits

Author SHA1 Message Date
Alexey Masterov
d753fd9c80 tc 2024-09-18 14:39:06 +02:00
Alexey Masterov
24431cfbdf Finally add the mask 2024-09-18 10:33:57 +02:00
Alexey Masterov
0ec1d6f73f an attempt 2024-09-18 10:30:57 +02:00
Alexey Masterov
cdd6a978b9 an attempt 2024-09-18 10:21:34 +02:00
Alexey Masterov
b2a8d96262 an attempt 2024-09-17 17:17:55 +02:00
Alexey Masterov
caaa399293 an attempt 2024-09-17 17:12:34 +02:00
Alexey Masterov
81e1001ae5 Fix add mask 2024-09-17 17:09:48 +02:00
Alexey Masterov
76016391ee Fix the actionlint warnings 2024-09-17 16:45:06 +02:00
Alexey Masterov
f4493a65db Refactor the workflow 2024-09-17 16:35:38 +02:00
a-masterov
9b6e46969e Update .github/workflows/cloud-regress.yml
Co-authored-by: Alexander Bayandin <alexander@neon.tech>
2024-09-17 16:23:01 +02:00
a-masterov
f8b9ec0dd0 Merge branch 'main' into amasterov/regress-arm 2024-09-10 14:29:04 +02:00
Alexey Masterov
84ca400dcf We are going to calculate md5 sum dynamically 2024-09-10 11:10:11 +02:00
Alexey Masterov
e127e97797 Fix the security problem 2024-09-09 17:00:51 +02:00
Alexey Masterov
b04995f10f Fix the security problem 2024-09-09 16:44:33 +02:00
Alexey Masterov
c29f78acc4 Fix errors 2024-09-09 14:17:30 +02:00
Alexey Masterov
b7fd9e5123 Fix an error 2024-09-09 13:55:00 +02:00
Alexey Masterov
b4d9632b19 Fix an error 2024-09-09 13:28:51 +02:00
Alexey Masterov
3b6449fb7b Fix an error 2024-09-09 12:48:37 +02:00
Alexey Masterov
130066898e Fix an error 2024-09-09 12:09:55 +02:00
Alexey Masterov
6f2d7b4662 Try to avoid passwords in clean text 2024-09-09 11:58:18 +02:00
Cihan Demirci
16c200d6d9 push images to prod ACR (#8940)
Used `vars` for new storing non-sensitive information, changed dev
secrets to vars as well but
didn't cleanup any secrets.

https://github.com/neondatabase/cloud/issues/16925

---------

Co-authored-by: Alexander Bayandin <alexander@neon.tech>
2024-09-07 00:20:36 +01:00
Alexander Bayandin
30583cb626 CI(label-for-external-users): add retry logic for unexpected errors (#8938)
## Problem

One of the PRs opened by a `neondatabase` org member got labelled as
`external` because the `gh api` call failed in the wrong way:
```
Get "https://api.github.com/orgs/neondatabase/members/<username>": dial tcp 140.82.114.5:443: i/o timeout
is-member=false
```

## Summary of changes
- Check that the error message is expected before labelling PRs
- Retry `gh api` call for 10 times in case of unexpected error messages
- Add `workflow_dispatch` trigger
2024-09-06 17:42:35 +01:00
Alexey Masterov
ac72832589 Change the runner 2024-09-06 14:33:35 +02:00
Alexey Masterov
6f6d5f1ea3 Add AWS access keys 2024-09-06 12:03:54 +02:00
Alexey Masterov
7cd76ee351 add an allure report and slack posting 2024-09-06 11:52:04 +02:00
Alexey Masterov
0510676a3f Some refactoring 2024-09-06 11:30:21 +02:00
Alexey Masterov
6140e3b6b1 Some refactoring 2024-09-06 11:09:26 +02:00
Alexey Masterov
74eec88125 Some refactoring 2024-09-06 11:08:33 +02:00
Alexey Masterov
243db8ab4a Some refactoring 2024-09-05 17:06:56 +02:00
a-masterov
815d7d6ab1 Merge branch 'main' into amasterov/regress-arm 2024-09-05 15:30:05 +02:00
Joonas Koivunen
efe03d5a1c build: sync between benchies (#8919)
Sometimes, the benchmarks fail to start up pageserver in 10s without any
obvious reason. Benchmarks run sequentially on otherwise idle runners.
Try running `sync(2)` after each bench to force a cleaner slate.

Implement this via:
- SYNC_AFTER_EACH_TEST environment variable enabled autouse fixture
- autouse fixture seems to be outermost fixture, so it works as expected
- set SYNC_AFTER_EACH_TEST=true for benchmarks in build_and_test
workflow

Evidence:
https://neon-github-public-dev.s3.amazonaws.com/reports/main/10678984691/index.html#suites/5008d72a1ba3c0d618a030a938fc035c/1210266507534c0f/

---------

Co-authored-by: Alexander Bayandin <alexander@neon.tech>
2024-09-05 14:29:48 +01:00
Alexey Masterov
e4dc7fe4a5 Remove running the cloud test on a pull request 2024-09-05 12:28:07 +02:00
Alexey Masterov
bfb7bf92f2 fix linters' warnings 2024-09-05 11:07:51 +02:00
Alexey Masterov
c34323eb80 Fix the test selection 2024-09-04 13:48:19 +02:00
Alexey Masterov
4104b1cbd4 Add CONNSTR 2024-09-04 13:29:08 +02:00
Alexey Masterov
6ff6843dbb add submodules 2024-09-04 11:23:35 +02:00
Alexey Masterov
c14d53b923 debug 2024-09-04 11:20:32 +02:00
Alexey Masterov
c7dde2e784 fix an obvious error 2024-09-04 11:05:05 +02:00
Alexey Masterov
173aef925c directory change 2024-09-04 11:03:06 +02:00
Alexey Masterov
b2af44f027 debug 2024-09-04 11:00:03 +02:00
Alexey Masterov
a07fda3a86 debug 2024-09-04 10:57:22 +02:00
Alexey Masterov
6b5d33de7d debug 2024-09-04 10:55:36 +02:00
Alexey Masterov
16450111c9 Fix a syntax error 2024-09-04 10:53:52 +02:00
Alexey Masterov
e8775dda76 Add patch 2024-09-04 10:42:45 +02:00
Alexey Masterov
8959cb1219 change on: 2024-09-03 19:16:23 +02:00
Alexey Masterov
ecf20bb6fa Add the workflow file 2024-09-03 17:21:33 +02:00
Alexander Bayandin
e58e045ebb CI(promote-compatibility-data): fix job (#8871)
## Problem

`promote-compatibility-data` job got broken and slightly outdated after 
- https://github.com/neondatabase/neon/pull/8552 -- we don't upload
artifacts for ARM64
- https://github.com/neondatabase/neon/pull/8561 -- we don't prepare
`debug` artifacts in the release branch anymore

## Summary of changes
- Promote artifacts from release PRs to the latest version (but do it
from `release` branch)
- Upload artifacts for both X64 and ARM64
2024-08-30 13:18:30 +01:00
Conrad Ludgate
72aa6b02da chore: speed up testing (#8874)
`safekeeper::random_test test_random_schedules` debug test takes over 2
minutes to run on our arm runners. Running it 6 times with pageserver
settings seems redundant.
2024-08-30 11:34:23 +01:00
Yuchen Liang
a889a49e06 pageserver: do vectored read on each dio-aligned section once (#8763)
Part of #8130, closes #8719.

## Problem

Currently, vectored blob io only coalesce blocks if they are immediately
adjacent to each other. When we switch to Direct IO, we need a way to
coalesce blobs that are within the dio-aligned boundary but has gap
between them.

## Summary of changes

- Introduces a `VectoredReadCoalesceMode` for `VectoredReadPlanner` and
`StreamingVectoredReadPlanner` which has two modes:
  - `AdjacentOnly` (current implementation)
  - `Chunked(<alignment requirement>)`
- New `ChunkedVectorBuilder` that considers batching `dio-align`-sized
read, the start and end of the vectored read will respect
`stx_dio_offset_align` / `stx_dio_mem_align` (`vectored_read.start` and
`vectored_read.blobs_at.first().start_offset` will be two different
value).
- Since we break the assumption that blobs within single `VectoredRead`
are next to each other (implicit end offset), we start to store blob end
offsets in the `VectoredRead`.
- Adapted existing tests to run in both `VectoredReadCoalesceMode`.
- The io alignment can also be live configured at runtime.

Signed-off-by: Yuchen Liang <yuchen@neon.tech>
2024-08-28 15:54:42 +01:00
Alexander Bayandin
75175f3628 CI(build-and-test): run regression tests on arm (#8552)
## Problem

We want to run our regression test suite on ARM.

## Summary of changes
- run regression tests on release ARM builds
- run `build-neon` (including rust tests) on debug ARM builds
- add `arch` parameter to test to distinguish them in the allure report
and in a database
2024-08-21 14:29:11 +01:00