Commit Graph

77 Commits

Author SHA1 Message Date
Yingwen
c19a56c79f feat!: Remove script crate and python feature (#5321)
* feat: exclude script crate

* chore: simplify feature

* feat: remove the script crate

* chore: remove python feature and some comments

* chore: fix warning
2025-01-08 12:11:53 +00:00
Weny Xu
05f115e047 fix(pg_backend): correct set idle_in_transaction_session_timeout statement (#5304)
* fix(metasrv): correct `backend` field configuration

* refactor!: added `#[serde(rename_all = "snake_case")]` macro to the `BackendImpl` enum

* fix(metasrv): correct `set idle_in_transaction_session_timeout` statement

* build: enable `pg_backend` by default
2025-01-06 12:07:35 +00:00
Ning Sun
cc5b1d42b0 ci: disable pyo3 build tasks (#5256)
* ci: disable pyo3 build tasks

* ci: skip installing python for windows

* ci: also removed python dependencies from docker base image
2024-12-31 04:53:41 +00:00
discord9
6a46f391cc ci: upload .pdb files too for better windows debug (#5224)
ci: upload .pdb files too
2024-12-25 08:10:57 +00:00
zyy17
267941bbb5 ci: support to pack multiple files in upload-artifacts action (#5228) 2024-12-25 02:37:32 +00:00
Weny Xu
1910d71cb3 chore: adjust fuzz tests cfg (#5207) 2024-12-20 06:58:51 +00:00
Weny Xu
d0245473a9 fix: correct set_region_role_state_gracefully behaviors (#5171)
* fix: reduce default max rows for fuzz testing

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): increase resource limits for GreptimeDB cluster

* chore(fuzz): increase resource limits for kafka

* fix: correct `set_region_role_state_gracefully` behaviors

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): redue resource limits for GreptimeDB & kafka
2024-12-16 14:01:40 +00:00
Weny Xu
b35221ccb6 ci: set meta replicas to 1 (#5111) 2024-12-09 07:22:47 +00:00
discord9
f3e5a5a7aa ci: install numpy in CI (#4895)
chore: install numpy in CI
2024-10-29 07:57:40 +00:00
Ruihang Xia
03f2fa219d feat: optimizer rule for windowed sort (#4874)
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement physical rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: install windowed sort physical rule and optimize partition ranges

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add logs and sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: introduce PartSortExec for partitioned sorting

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune exec nodes' properties and metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* debug: add more info on very wrong

* debug: also print overlap ranges

* feat: add check when emit PartSort Stream

* dbg: info on overlap working range

* feat: check batch range is inside part range

* set distinguish partition range param

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: more logs

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune optimizer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix windowed sort rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: early terminate sort stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: remove min/max check

* chore: remove unused windowed_sort module, uuid feature and refactor region_scanner to synchronous

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: print more fuzz log

* chore: more log

* fix: part sort should skip empty part

* chore: remove insert logs

* tests: empty PartitionRange

* refactor: testcase

* docs: update comment&tests: all empty

* ci: enlarge etcd cpu limit

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
2024-10-29 07:46:05 +00:00
liyang
1acda74c26 fix: cannot input tag for the dev-builder image (#4743) 2024-09-19 11:14:34 +00:00
Weny Xu
b52e3c694a chore(ci): set etcd resources limits (#4665) 2024-09-03 07:25:23 +00:00
zyy17
f0e2d6e663 fix: use 'target' for 'actions-rust-lang/setup-rust-toolchain' to fix cross build failed (#4661) 2024-09-02 06:11:12 +00:00
zyy17
ddafcc678c ci: disable macos integration test and some minor refactoring (#4658) 2024-09-02 03:06:17 +00:00
Ning Sun
0b102ef846 ci: improve toolchain resolution in ci (#4614)
* ci: improve toolchain resolution in ci

* fix: yaml format
2024-08-27 07:46:51 +00:00
liyang
55d65da24d ci: add push dev-build images to aws ecr (#4618)
* ci: add push dev-build images to aws ecr

* chore: use toolchain file generation dev-build image tag

* chore: change dev-build version

* Update .github/workflows/release-dev-builder-images.yaml

Co-authored-by: zyy17 <zyylsxm@gmail.com>

---------

Co-authored-by: zyy17 <zyylsxm@gmail.com>
2024-08-26 09:36:55 +00:00
Lanqing Yang
93be81c041 feat: implement postgres kvbackend (#4421) 2024-08-14 22:49:32 +00:00
Weny Xu
4a3982ca60 chore: use configData (#4522)
* chore: use `configData`

* chore: add an empty line
2024-08-07 07:43:04 +00:00
Yingwen
559219496d ci: fix windows temp path (#4518) 2024-08-06 13:53:12 +00:00
Weny Xu
dd23d47743 chore(ci): bring back chaos tests (#4456)
* Revert "chore: temporarily disable fuzz chaos tests (#4457)"

This reverts commit f0c953f84a.

* chore: update config

* Update .github/actions/setup-greptimedb-cluster/with-remote-wal.yaml

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
2024-07-31 07:29:31 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436)
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
2024-07-30 06:17:58 +00:00
Yingwen
78962015dd ci: keep sqlness log by default (#4449)
* ci: keep sqlness log by default

* chore: not preserve state in makefile by default

* ci: use make
2024-07-29 17:11:24 +00:00
Weny Xu
e39f49fe56 fix: ensure keep alive is completed in time (#4349)
* fix: ensure keep alive is completed in time

* chore: apply suggestions from CR

* chore: use write runtime

* refactor: set META_LEASE_SECS to 5

* chore: set etcd replicas to 1

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* fix: set `MissedTickBehavior::Delay`

* chore: apply suggestions from CR
2024-07-17 06:14:45 +00:00
Weny Xu
08c415c729 ci: retry on error or timeout during installing operator (#4308)
chore(ci): retry on error or timeout during installing operator
2024-07-08 03:31:13 +00:00
Weny Xu
0b624dc337 ci: retry on error during installing operator (#4295)
chore(ci): retry on error during installing operator
2024-07-05 07:54:31 +00:00
zyy17
c9177cceeb ci: push latest greptimedb nigthly build image (#4260) 2024-07-03 11:14:06 +00:00
Weny Xu
352cc9ddde test: add e2e test for region failover (#4188)
* test: add e2e test for region failover

* chore: add ci cfg

* chore: reduce parallelism to 8

* fix(ci): enable region failure

* chore: set sqlx LogLevel to Off

* refactor: move help functions to utils
2024-06-28 06:49:41 +00:00
Weny Xu
bf3ad44584 fix: fix release CI typo (#4147)
* fix: ci typo

* chore: use aws registry
2024-06-14 14:29:34 +00:00
Weny Xu
11a903f193 chore(ci): switch to aws registry (#4145)
chore: switch to aws registry
2024-06-14 11:46:57 +00:00
Weny Xu
acdfaabfa5 fix(ci): use ld_classic on macOS (#4143)
fix: use ld_classic on macos
2024-06-14 08:09:12 +00:00
Weny Xu
573e25a40f chore: run fuzz tests with disk cache (#4118)
* chore: run fuzz tests with disk cache

* fix: print error messages correctly
2024-06-13 09:07:12 +00:00
Weny Xu
09e0e1b246 chore: run fuzz tests with kafka remote wal (#4105)
* chore: add fuzz tests with kafka

* chore(ci): use minio

* chore: add empty line

* chore(ci): refactor

* chore: add empty line

* fix: update config

* fix: add default value for `MetaClientOptions`

* fix: remove redundant `debug_assert`

* chore: run fuzz tests with disk cache

* chore: remove redundant minio setup

* chore: cache targets

* Revert "chore: run fuzz tests with disk cache"

This reverts commit d81783187d.

* chore: fix typo

* chore: apply suggestions from CR

* Revert "fix: remove redundant `debug_assert`"

This reverts commit 09b899eed1.
2024-06-07 03:47:40 +00:00
LFC
c0aed1d267 feat: set global runtime size by config file (#4063)
* set global runtime size

* fix: resolve PR comments

* fix: log the whole option

* fix ci

* debug ci

* debug ci

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
2024-06-04 10:03:33 +00:00
Weny Xu
614643ef7b chore(ci): add more replicas (#4015) 2024-05-23 02:43:24 +00:00
Weny Xu
56aed6e6ff chore(ci): export kind logs (#3996)
* chore(ci): export kind logs

* chore: add empty line
2024-05-21 11:56:03 +00:00
Weny Xu
7a8222dd97 fix: try to fix broken CI (#3998)
* fix: try to fix broken CI

* chore: using loop to check status
2024-05-21 07:02:18 +00:00
Weny Xu
d7ade3c854 chore(ci): add fuzz tests for distributed mode (#3967)
* chore(ci): add cfg for setup GreptimeDB cluster

* chore: use kind

* chore: always print info

* chore: add debug print

* chore: set etcd replica to 1

* ci: refactor e2e cfg

* ci: add Fuzz Test for distributed mode

* Apply suggestions from code review

* chore: apply suggestions from CR

* chore(ci): upload logs
2024-05-21 04:58:42 +00:00
Yingwen
e372e25e30 build: add RUSTUP_WINDOWS_PATH_ADD_BIN env (#3969)
build: add RUSTUP_WINDOWS_PATH_ADD_BIN: 1
2024-05-17 06:01:46 +00:00
tison
6c621b7fcf ci: implement docbot in cyborg (#3937)
* ci: implement docbot in cyborg

Signed-off-by: tison <wander4096@gmail.com>

* allow remove non-existing label

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* fixup org name

Signed-off-by: tison <wander4096@gmail.com>

* fixup step name

Signed-off-by: tison <wander4096@gmail.com>

* remove unused file

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-05-15 12:55:49 +00:00
Weny Xu
f6e2039eb8 test: introduce unstable fuzz create table test (#3788)
* feat: implement unstable_fuzz_create_table_standalone

* chore: use drop database

* docs: update docs

* chore: add ci config

* chore: add feature gate

* fix: fix clippy

* chore: update ci

* Apply suggestions from code review

* feat: reduce num

* Apply suggestions from code review

* chore: apply suggestions from CR

* Apply suggestions from code review

* chore: reduce `wait_timeout` in health check

* Update .env.example

* refactor: use `init_greptime_connections_via_env`

* refactor: use `init_greptime_connections_via_env`

---------

Co-authored-by: tison <wander4096@gmail.com>
2024-05-01 06:08:49 +00:00
Yingwen
0aaf7621bd build: only build amd64 image for CentOS (#3784)
build: only build amd64 for centos
2024-04-23 13:44:26 +00:00
tison
7ca0fa52d4 ci: secrets.GITHUB_TOKEN not exist on self-hosted runner (#3765) 2024-04-22 11:29:08 +00:00
tison
2fca45b048 ci: setup-protoc always with token (#3674)
Signed-off-by: tison <wander4096@gmail.com>
2024-04-08 18:13:24 +00:00
Weny Xu
3a326775ee ci: add bin options to reduce build burden (#3518)
chore: add bin options
2024-03-14 11:05:35 +00:00
Weny Xu
a218f12bd9 test: add fuzz test for create table (#3441)
* feat: add create table fuzz test

* chore: add ci cfg for fuzz tests

* refactor: remove redundant nightly config

* chore: run fuzz test in debug mode

* chore: use ubuntu-latest

* fix: close connection

* chore: add cache in fuzz test ci

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: refactor the fuzz test action
2024-03-07 06:51:19 +00:00
tison
fa08085119 ci: upgrade actions to node20-based version (#3345)
* ci: upgrade actions to node20-based version

Signed-off-by: tison <wander4096@gmail.com>

* distinguish artifact name

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-21 08:09:09 +00:00
tison
d9a96344ee ci: try fix log location (#3342)
Signed-off-by: tison <wander4096@gmail.com>
2024-02-21 07:01:51 +00:00
tison
29f11d7b7e ci: upgrade actions to avoid node16 deprecation warning (#3317)
* ci: upgrade actions to avoid node16 deprecantion warning

Signed-off-by: tison <wander4096@gmail.com>

* try delete size label action

Signed-off-by: tison <wander4096@gmail.com>

* one more action

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-02-18 15:20:26 +00:00
LFC
902570abf6 ci: fix nightly build (#3276)
* ci: fix nightly build

* ci: fix nightly build
2024-02-03 03:20:47 +00:00
LFC
50d16d6330 ci: build GreptimeDB binary for later use (#3244)
* ci: build GreptimeDB binaries for later use

* debug CI

* try larger runner host

* Revert "try larger runner host"

This reverts commit 03c18c0f51.

* fix: resolve PR comments

* revert some unrelated action yamls

* fix CI

* use artifact upload v4 for faster upload and download speed
2024-01-31 03:02:27 +00:00