rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-07 22:02:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,204 Commits 103 Branches 226 Tags
4534e4c31dfd62b7d1275d515fded980a78bea1a
Commit Graph

148 Commits

Author SHA1 Message Date
Weny Xu
74222c3070 chore: improve CI debugging and resource configuration (#6274) 
* chore: describe pods on CI failure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: increase memory limit for main pod template from 2Gi to 3Gi

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-06-09 11:40:32 +00:00
Ruihang Xia
bf496e05cc ci: turn off fail fast strategy (#6157) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-05-23 02:38:25 +00:00
yinheli
8d36ffb4e1 chore: enable github folder typo check and fix typos (#6128) 2025-05-20 04:20:07 +00:00
liyang
51f2cb1053 ci: run only in the GreptimeTeam/greptimedb repository (#6064) 
ci: run only in the GreptimeTeam/greptimedb repository
 2025-05-08 08:39:13 +00:00
Yohan Wal
1dc4a196bf feat: add mysql election logic (#5694) 
* feat: add mysql election

* feat: add mysql election

* chore: fix deps

* chore: fix deps

* fix: duplicate container

* fix: duplicate setup for sqlness

* fix: call once

* fix: do not use NOWAIT for mysql 5.7

* chore: apply comments

* fix: no parallel sqlness for mysql

* chore: comments and minor revert

* chore: apply comments

* chore: apply comments

* chore: add  to table name

* ci: use 2 metasrv to detect election bugs

* refactor: better election logic

* chore: apply comments

* chore: apply comments

* feat: version check before startup
 2025-03-19 11:31:18 +00:00
Yohan Wal
af1920defc feat: add mysql kvbackend (#5528) 
* feat: add mysql kvbackend txn support

* chore: error handling

* chore: follow review comments

* chore: follow review comments

* chore: follow review comments

* revert: mysql QAQ

* revert: revert changes to sqls

This reverts commit cf98c50dd9.

* chore: add comments
 2025-03-12 06:52:56 +00:00
Ning Sun
e0065a5159 ci: remove ubuntu 20.04 runners (#5545) 
* ci: remove ubuntu 20.04 runners

* chore: update ec2-github-runner action as author suggests

* fix: use latest ubuntu image for fuzz test

* Update action.yml

* Update action.yml

---------

Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
Co-authored-by: liyang <daviderli614@gmail.com>
 2025-03-05 00:40:29 +00:00
yihong
342883e922 ci: safe ci using zizmor check (#5491) 
* ci: safe ci using zizmor check

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: lines empty

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

* fix: delete useless code

Signed-off-by: yihong0618 <zouzou0208@gmail.com>

---------

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-02-11 02:38:14 +00:00
yihong
1e61d05211 fix: arm actions test failed (#5433) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-01-24 03:43:59 +00:00
Ning Sun
5daac5fe3d ci: revert coverage runner (#5403) 2025-01-20 03:52:38 +00:00
Ning Sun
f53b6777cc ci: use arm builders for tests (#5395) 2025-01-20 02:12:12 +00:00
Ning Sun
121ec7936f ci: do not collect coverage from pull request any more (#5364) 
* ci: do not collect coverage from pull request any more

* fix: disable toolchain cache
 2025-01-15 09:33:49 +00:00
Ning Sun
f5829364a2 fix: security fix, sqlx, hashbrown, idna and CI updates (#5330) 
* fix: security fix, sqlx, hashbrown, idna

* ci: optimize ci cache generation

* feat: update pprof
 2025-01-14 08:19:33 +00:00
Ning Sun
0685ba265c ci: disable cache for some tasks, create cache in nightly build (#5324) 
* ci: disable cache for some tasks

* ci: add a nightly test to create rust cache on main
 2025-01-09 11:14:13 +00:00
Ning Sun
815ce59a3a ci: use mold for tests (#5319) 
* ci: use mold for tests

* ci: enable rust cache saving for merge group
 2025-01-08 12:41:04 +00:00
Ning Sun
e18416a726 ci: do not trigger tests when there is a merge conflict (#5318) 
* ci: do not trigger tests when there is a merge conflict

* Update .github/workflows/develop.yml

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* ci: disable cache from rust toolchain action

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2025-01-08 09:02:49 +00:00
Weny Xu
3d9df822ad refactor: refactor PgStore (#5309) 
* refactor: refactor PgStore

* fix: election use bytea and txn use Serializable to avoid read unrepeatable (#4)

* fix: election use bytea as well

* fix: use Serializable to avoid read unrepeatable

* chore: remove unused error

* ci: enable pg kvbackend and sqlness

* ci: switch on pg_kvbackend feature

* fix: fix sqlness runner

* chore: add pg_kvbackend feature gate

* build(ci): add feature gate

* fix: add retry for `PgStore` txn

* fix: correct `SET_IDLE_SESSION_TIMEOUT`

---------

Co-authored-by: Yohan Wal <1035325592@qq.com>
Co-authored-by: CookiePieWw <profsyb@gmail.com>
 2025-01-07 07:27:58 +00:00
Ning Sun
ddf36c8324 ci: disable docker/rust cache temporarily and merge docker compose files (#5293) 
* ci: disable docker cache temporarily and merge docker compose files

* ci: fix compose file name and options

* ci: try to disable rust cache
 2025-01-06 08:23:46 +00:00
Ning Sun
cc5b1d42b0 ci: disable pyo3 build tasks (#5256) 
* ci: disable pyo3 build tasks

* ci: skip installing python for windows

* ci: also removed python dependencies from docker base image
 2024-12-31 04:53:41 +00:00
Ning Sun
1287d4cb9f ci: make sure clippy passes before running tests (#5253) 
* ci: make sure clippy passes before running tests

* ci: do not run ci on main branch
 2024-12-30 07:01:15 +00:00
Yingwen
f1eb76f489 fix: implement a CacheStrategy to ensure compaction use cache correctly (#5254) 
* feat: impl CacheStrategy

* refactor: replace Option<CacheManagerRef> with CacheStrategy

* feat: add disabled strategy

* ci: force update taplo

* refactor: rename CacheStrategy::Normal to CacheStrategy::EnableAll

* ci: force install cargo-gc-bin

* ci: force install

* chore: use CacheStrategy::Disabled as ScanInput default

* chore: fix compiler errors
 2024-12-30 06:24:53 +00:00
Yohan Wal
ef13c52814 feat: init PgElection with candidate registration (#5209) 
* feat: init PgElection

fix: release advisory lock

fix: handle duplicate keys

chore: update comments

fix: unlock if acquired the lock

chore: add TODO and avoid unwrap

refactor: check both lock and expire time, add more comments

chore: fmt

fix: deal with multiple edge cases

feat: init PgElection with candidate registration

chore: fmt

chore: remove

* test: add unit test for pg candidate registration

* test: add unit test for pg candidate registration

* chore: update pg env

* chore: make ci happy

* fix: spawn a background connection thread

* chore: typo

* fix: shadow the election client for now

* fix: fix ci

* chore: readability

* chore: follow review comments

* refactor: use kvbackend for pg election

* chore: rename

* chore: make clippy happy

* refactor: use pg server time instead of local ones

* chore: typo

* chore: rename infancy to leader_infancy for clarification

* chore: clean up

* chore: follow review comments

* chore: follow review comments

* ci: unit test should test all features

* ci: fix

* ci: just test pg
 2024-12-26 12:39:32 +00:00
Weny Xu
d0245473a9 fix: correct set_region_role_state_gracefully behaviors (#5171) 
* fix: reduce default max rows for fuzz testing

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): increase resource limits for GreptimeDB cluster

* chore(fuzz): increase resource limits for kafka

* fix: correct `set_region_role_state_gracefully` behaviors

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): redue resource limits for GreptimeDB & kafka
 2024-12-16 14:01:40 +00:00
discord9
d91517688a chore: fix aws_lc not in depend tree check in CI (#5121) 
* chore: fix aws_lc check in CI

* chore: update lock file
 2024-12-11 07:02:03 +00:00
Weny Xu
83eb777d21 test: add fuzz test for metric region migration (#4862) 
* test: add fuzz tests for migrate metric regions

* test: insert values before migrating metric region

* feat: correct table num

* chore: apply suggestions from CR
 2024-10-29 15:47:48 +00:00
discord9
f3e5a5a7aa ci: install numpy in CI (#4895) 
chore: install numpy in CI
 2024-10-29 07:57:40 +00:00
Weny Xu
6e776d5f98 feat: support to reject write after flushing (#4759) 
* refactor: use `RegionRoleState` instead of `RegionState`

* feat: introducing `RegionLeaderState::Downgrading`

* refactor: introduce `set_region_role_state_gracefully`

* refactor: use `set_region_role` instead of `set_writable`

* feat: support to reject write after flushing

* fix: fix unit tests

* test: add unit test for `should_reject_write`

* chore: add comments

* chore: refine comments

* fix: fix unit test

* test: enable fuzz tests for Local WAL

* chore: add logs

* chore: rename `RegionStatus` to `RegionState`

* feat: introduce `DowngradingLeader`

* chore: rename

* refactor: refactor `set_role_state` tests

* test: ensure downgrading region will reject write

* chore: enhance logs

* chore: refine name

* chore: refine comment

* test: add tests for `set_role_role_state`

* fix: fix unit tests

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-09-30 08:28:51 +00:00
discord9
0274e752ae chore: make sure aws-lc-sys wouldn't be built (#4767) 
* chore: make sure aws-lc-sys wouldn't be built

* fix: bash

* refactor: multiple line bash
 2024-09-25 17:11:37 +00:00
Ruihang Xia
93f202694c refactor: remove unused error variants (#4666) 
* add python script

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unused errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix all negative cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* setup CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add license header

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-09-03 13:19:38 +00:00
Ning Sun
0b102ef846 ci: improve toolchain resolution in ci (#4614) 
* ci: improve toolchain resolution in ci

* fix: yaml format
 2024-08-27 07:46:51 +00:00
Ruihang Xia
a3b8d2fe8f chore: bump rust toolchain to 2024-08-21 (#4606) 
* chore: bump rust toolchain to 2024-08-22

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update workflow

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* try 20240606

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-08-22 15:38:10 +00:00
Weny Xu
25cd61b310 chore: upgrade toolchain to nightly-2024-08-07 (#4549) 
* chore: upgrade toolchain to `nightly-2024-08-07`

* chore(ci): upgrade toolchain

* fix: fix unit test
 2024-08-22 11:02:18 +00:00
Weny Xu
8036b44347 chore: setup kafka before downloading binary step (#4582) 2024-08-19 06:44:33 +00:00
Lanqing Yang
93be81c041 feat: implement postgres kvbackend (#4421) 2024-08-14 22:49:32 +00:00
Weny Xu
2e2eacf3b2 feat: add SASL and TLS config for Kafka client (#4536) 
* feat: add SASL and TLS config

* feat: add SASL/PLAIN and TLS config for Kafka client

* chore: use `ring`

* feat: support SASL SCRAM-SHA-256 and SCRAM-SHA-512

* fix: correct unit test

* test: add integration test

* chore: apply suggestions from CR

* refactor: introduce `KafkaConnectionConfig`

* chore: refine toml examples

* docs: add missing fields

* chore: refine examples

* feat: allow no server ca cert

* chore: refine examples

* chore: fix clippy

* feat: load system ca certs

* chore: fmt toml

* chore: unpin version

* Update src/common/wal/src/error.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-08-12 12:27:11 +00:00
LFC
a75cfaa516 chore: update snafu to make clippy happy (#4507) 
* chore: update snafu to make clippy happy

* fix ci
 2024-08-07 16:12:00 +00:00
LFC
685aa7dd8f ci: squeeze some disk space for complex fuzz tests (#4519) 
* ci: squeeze some disk space for complex fuzz tests

* Update .github/workflows/develop.yml

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-08-06 11:52:34 +00:00
Weny Xu
79f10d0415 chore: reduce fuzz tests in CI (#4505) 2024-08-05 06:56:41 +00:00
Weny Xu
dd23d47743 chore(ci): bring back chaos tests (#4456) 
* Revert "chore: temporarily disable fuzz chaos tests (#4457)"

This reverts commit f0c953f84a.

* chore: update config

* Update .github/actions/setup-greptimedb-cluster/with-remote-wal.yaml

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-31 07:29:31 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436) 
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-07-30 06:17:58 +00:00
Weny Xu
f0c953f84a chore: temporarily disable fuzz chaos tests (#4457) 2024-07-29 13:23:40 +00:00
Weny Xu
8a119aa0b2 feat: add naive region failover test for metric table (#4269) 
* feat: add failover test for metric table

* chore: introduce help marco

* chore: remove incorrect check
 2024-07-05 06:54:23 +00:00
Weny Xu
76fac359cd feat: implement naive fuzz test for region migration (#4252) 
* fix(fuzz): adapt for new partition rules

* feat: implement naive fuzz test for region migration

* chore(ci): add ci cfg

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-07-03 13:30:41 +00:00
Weny Xu
0f4b9e576d chore(ci): add timeout (60min) for fuzz tests (#4255) 2024-07-03 03:36:43 +00:00
Weny Xu
352cc9ddde test: add e2e test for region failover (#4188) 
* test: add e2e test for region failover

* chore: add ci cfg

* chore: reduce parallelism to 8

* fix(ci): enable region failure

* chore: set sqlx LogLevel to Off

* refactor: move help functions to utils
 2024-06-28 06:49:41 +00:00
Weny Xu
573e25a40f chore: run fuzz tests with disk cache (#4118) 
* chore: run fuzz tests with disk cache

* fix: print error messages correctly
 2024-06-13 09:07:12 +00:00
Weny Xu
09e0e1b246 chore: run fuzz tests with kafka remote wal (#4105) 
* chore: add fuzz tests with kafka

* chore(ci): use minio

* chore: add empty line

* chore(ci): refactor

* chore: add empty line

* fix: update config

* fix: add default value for `MetaClientOptions`

* fix: remove redundant `debug_assert`

* chore: run fuzz tests with disk cache

* chore: remove redundant minio setup

* chore: cache targets

* Revert "chore: run fuzz tests with disk cache"

This reverts commit d81783187d.

* chore: fix typo

* chore: apply suggestions from CR

* Revert "fix: remove redundant `debug_assert`"

This reverts commit 09b899eed1.
 2024-06-07 03:47:40 +00:00
Weny Xu
16b85b06b6 chore: remove gc before running fuzz tests (#4108) 2024-06-05 11:59:29 +00:00
Ruihang Xia
03cacf9948 ci: cargo gc all fuzz test runner (#4081) 
* ci: cargo gc all fuzz test runner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore(ci): disable caching targets

* chore(ci): remove .tar file after unzip

* fix cargo-gc command

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-06-05 09:58:29 +00:00
Weny Xu
e0a2c5a581 chore(ci): remove redundant sqlness test config (#4106) 2024-06-05 08:39:39 +00:00