rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2025-12-27 00:19:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,922 Commits 103 Branches 223 Tags
feature/df-binary-operator-nested-data
Commit Graph

97 Commits

Author SHA1 Message Date
LFC
32168e8ca8 ci: dev-build with large page size (#7228) 
* ci: able to build greptimedb with large page size in dev-build

Signed-off-by: luofucong <luofc@foxmail.com>

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Signed-off-by: luofucong <luofc@foxmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-11-17 02:38:16 +00:00
Weny Xu
7cf47ccf54 fix: initialize remote WAL regions with correct flushed entry IDs (#6856) 
* fix: initialize remote WAL regions with correct flushed entry IDs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: add logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: correct latest offset

Signed-off-by: WenyXu <wenymedia@gmail.com>

* fix: update sqlness

Signed-off-by: WenyXu <wenymedia@gmail.com>

* feat: add replay checkpoint to catchup request

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: logs

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: apply suggestions

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-09-03 04:00:02 +00:00
liyang
e595885dc6 chore: use greptime dockerhub image (#6865) 
Signed-off-by: liyang <daviderli614@gmail.com>
 2025-09-01 11:07:26 +00:00
liyang
7bc669e991 chore: update bitnami config (#6847) 
* chore: update bitnami config

Signed-off-by: liyang <daviderli614@gmail.com>

* update postgresql chart version

Signed-off-by: liyang <daviderli614@gmail.com>

* fix ci

Signed-off-by: liyang <daviderli614@gmail.com>

* refactor: add pull-test-deps-images.sh to pull images one by one to avoid rate limit

Signed-off-by: zyy17 <zyylsxm@gmail.com>

---------

Signed-off-by: liyang <daviderli614@gmail.com>
Signed-off-by: zyy17 <zyylsxm@gmail.com>
Co-authored-by: zyy17 <zyylsxm@gmail.com>
 2025-08-29 02:45:14 +00:00
liyang
88c4409df4 ci: use the new meta backendStorage etcd structure (#6303) 
Signed-off-by: liyang <daviderli614@gmail.com>
 2025-06-12 03:17:32 +00:00
Weny Xu
74222c3070 chore: improve CI debugging and resource configuration (#6274) 
* chore: describe pods on CI failure

Signed-off-by: WenyXu <wenymedia@gmail.com>

* chore: increase memory limit for main pod template from 2Gi to 3Gi

Signed-off-by: WenyXu <wenymedia@gmail.com>

---------

Signed-off-by: WenyXu <wenymedia@gmail.com>
 2025-06-09 11:40:32 +00:00
liyang
0eb9e97f79 ci: increase upload s3 retry times (#6263) 
ci: increase upload s3 retry

Signed-off-by: liyang <daviderli614@gmail.com>
 2025-06-06 12:42:03 +00:00
localhost
92422dafca feat: add CLI tool to export metadata (#6150) 
* chore: add tool to export db meta

* chore: add meta restore command

* chore: fmt code

* chore: remove useless error

* chore: support key prefix

* chore: add clean check for meta restore

* chore: add more log for meta restore

* chore: resolve s3 and local file root in command meta-snapshot

* chore: remove the pg mysql features from the build script as they are already in the default feature

* chore: fix by pr comment
 2025-05-28 03:18:00 +00:00
Ruihang Xia
56e696bd55 chore: remove stale wal config entries (#6134) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2025-05-20 19:42:09 +00:00
Yuhan Wang
ee07b9bfa8 test: update configs to enable auto wal prune (#5938) 
* test: update configs to enable auto wal prune

* fix: add humantime_serde

* fix: enable overwrite_entry_start_id

* fix: not in metasrv

* chore: update default value name

* Apply suggestions from code review

Co-authored-by: jeremyhi <jiachun_feng@proton.me>

* fix: kafka use overwrite_entry_start_id

---------

Co-authored-by: jeremyhi <jiachun_feng@proton.me>
 2025-04-21 07:57:43 +00:00
Yuhan Wang
e817a65d75 feat: enable submitting wal prune procedure periodically (#5867) 
* feat: enable submitting wal prune procedure periodically

* chore: fix and add options

* test: add unit test

* test: fix unit test

* test: enable active_wal_pruning in test

* test: update default config

* chore: update config name

* refactor: use semaphore to control the number of prune process

* refactor: use split client for wal prune manager and topic creator

* chore: add configs

* chore: apply review comments

* fix: use tracker properly

* fix: use guard to track semaphore

* test: update unit tests

* chore: update config name

* chore: use prunable_entry_id

* refactor: semaphore to only limit the process of submitting

* chore: remove legacy sort

* chore: better configs

* fix: update config.md

* chore: respect fmt

* test: update unit tests

* chore: use interval_at

* fix: fix unit test

* test: fix unit test

* test: fix unit test

* chore: apply review comments

* docs: update config docs
 2025-04-18 16:02:33 +00:00
yihong
c5b55fd8cf fix: close issue #3902 since upstream fixed (#5801) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-03-30 12:34:52 +00:00
Yohan Wal
1dc4a196bf feat: add mysql election logic (#5694) 
* feat: add mysql election

* feat: add mysql election

* chore: fix deps

* chore: fix deps

* fix: duplicate container

* fix: duplicate setup for sqlness

* fix: call once

* fix: do not use NOWAIT for mysql 5.7

* chore: apply comments

* fix: no parallel sqlness for mysql

* chore: comments and minor revert

* chore: apply comments

* chore: apply comments

* chore: add  to table name

* ci: use 2 metasrv to detect election bugs

* refactor: better election logic

* chore: apply comments

* chore: apply comments

* feat: version check before startup
 2025-03-19 11:31:18 +00:00
Yohan Wal
af1920defc feat: add mysql kvbackend (#5528) 
* feat: add mysql kvbackend txn support

* chore: error handling

* chore: follow review comments

* chore: follow review comments

* chore: follow review comments

* revert: mysql QAQ

* revert: revert changes to sqls

This reverts commit cf98c50dd9.

* chore: add comments
 2025-03-12 06:52:56 +00:00
liyang
a71b93dd84 fix: unable to install software-properties-common in dev builder (#5643) 
* fix: unable to install software-properties-common in dev builder

* test dev builder

* improve dev-build image

* setup qemu action
 2025-03-05 07:07:06 +00:00
Ning Sun
e0065a5159 ci: remove ubuntu 20.04 runners (#5545) 
* ci: remove ubuntu 20.04 runners

* chore: update ec2-github-runner action as author suggests

* fix: use latest ubuntu image for fuzz test

* Update action.yml

* Update action.yml

---------

Co-authored-by: shuiyisong <113876041+shuiyisong@users.noreply.github.com>
Co-authored-by: liyang <daviderli614@gmail.com>
 2025-03-05 00:40:29 +00:00
liyang
009dbad581 ci: don't push nightly latest image (#5551) 
* ci: don't push nightly latest image

* add push release latest image
 2025-02-17 04:34:49 +00:00
liyang
a2047b096c ci: use s5cmd upload artifacts (#5550) 2025-02-17 02:57:13 +00:00
zyy17
58c6274bf6 fix: use fixed tonistiigi/binfmt:qemu-v7.0.0-28 image version instead of latest version to avoid segmentation fault (#5516) 
fix: use fixed tonistiigi/binfmt:qemu-v7.0.0-28 image version instead of latest version to avoid segmentation fault

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2025-02-12 19:29:49 +00:00
Yingwen
859717c309 ci: always build standard db (#5390) 
ci: remove dev-mode = false in building standard db
 2025-01-17 13:10:46 +00:00
Yingwen
c19a56c79f feat!: Remove script crate and python feature (#5321) 
* feat: exclude script crate

* chore: simplify feature

* feat: remove the script crate

* chore: remove python feature and some comments

* chore: fix warning
 2025-01-08 12:11:53 +00:00
Weny Xu
05f115e047 fix(pg_backend): correct set idle_in_transaction_session_timeout statement (#5304) 
* fix(metasrv): correct `backend` field configuration

* refactor!: added `#[serde(rename_all = "snake_case")]` macro to the `BackendImpl` enum

* fix(metasrv): correct `set idle_in_transaction_session_timeout` statement

* build: enable `pg_backend` by default
 2025-01-06 12:07:35 +00:00
Ning Sun
cc5b1d42b0 ci: disable pyo3 build tasks (#5256) 
* ci: disable pyo3 build tasks

* ci: skip installing python for windows

* ci: also removed python dependencies from docker base image
 2024-12-31 04:53:41 +00:00
discord9
6a46f391cc ci: upload .pdb files too for better windows debug (#5224) 
ci: upload .pdb files too
 2024-12-25 08:10:57 +00:00
zyy17
267941bbb5 ci: support to pack multiple files in upload-artifacts action (#5228) 2024-12-25 02:37:32 +00:00
Weny Xu
1910d71cb3 chore: adjust fuzz tests cfg (#5207) 2024-12-20 06:58:51 +00:00
Weny Xu
d0245473a9 fix: correct set_region_role_state_gracefully behaviors (#5171) 
* fix: reduce default max rows for fuzz testing

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): increase resource limits for GreptimeDB cluster

* chore(fuzz): increase resource limits for kafka

* fix: correct `set_region_role_state_gracefully` behaviors

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): redue resource limits for GreptimeDB & kafka
 2024-12-16 14:01:40 +00:00
Weny Xu
b35221ccb6 ci: set meta replicas to 1 (#5111) 2024-12-09 07:22:47 +00:00
discord9
f3e5a5a7aa ci: install numpy in CI (#4895) 
chore: install numpy in CI
 2024-10-29 07:57:40 +00:00
Ruihang Xia
03f2fa219d feat: optimizer rule for windowed sort (#4874) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement physical rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: install windowed sort physical rule and optimize partition ranges

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add logs and sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: introduce PartSortExec for partitioned sorting

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune exec nodes' properties and metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* debug: add more info on very wrong

* debug: also print overlap ranges

* feat: add check when emit PartSort Stream

* dbg: info on overlap working range

* feat: check batch range is inside part range

* set distinguish partition range param

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: more logs

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune optimizer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix windowed sort rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: early terminate sort stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: remove min/max check

* chore: remove unused windowed_sort module, uuid feature and refactor region_scanner to synchronous

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: print more fuzz log

* chore: more log

* fix: part sort should skip empty part

* chore: remove insert logs

* tests: empty PartitionRange

* refactor: testcase

* docs: update comment&tests: all empty

* ci: enlarge etcd cpu limit

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
 2024-10-29 07:46:05 +00:00
liyang
1acda74c26 fix: cannot input tag for the dev-builder image (#4743) 2024-09-19 11:14:34 +00:00
Weny Xu
b52e3c694a chore(ci): set etcd resources limits (#4665) 2024-09-03 07:25:23 +00:00
zyy17
f0e2d6e663 fix: use 'target' for 'actions-rust-lang/setup-rust-toolchain' to fix cross build failed (#4661) 2024-09-02 06:11:12 +00:00
zyy17
ddafcc678c ci: disable macos integration test and some minor refactoring (#4658) 2024-09-02 03:06:17 +00:00
Ning Sun
0b102ef846 ci: improve toolchain resolution in ci (#4614) 
* ci: improve toolchain resolution in ci

* fix: yaml format
 2024-08-27 07:46:51 +00:00
liyang
55d65da24d ci: add push dev-build images to aws ecr (#4618) 
* ci: add push dev-build images to aws ecr

* chore: use toolchain file generation dev-build image tag

* chore: change dev-build version

* Update .github/workflows/release-dev-builder-images.yaml

Co-authored-by: zyy17 <zyylsxm@gmail.com>

---------

Co-authored-by: zyy17 <zyylsxm@gmail.com>
 2024-08-26 09:36:55 +00:00
Lanqing Yang
93be81c041 feat: implement postgres kvbackend (#4421) 2024-08-14 22:49:32 +00:00
Weny Xu
4a3982ca60 chore: use configData (#4522) 
* chore: use `configData`

* chore: add an empty line
 2024-08-07 07:43:04 +00:00
Yingwen
559219496d ci: fix windows temp path (#4518) 2024-08-06 13:53:12 +00:00
Weny Xu
dd23d47743 chore(ci): bring back chaos tests (#4456) 
* Revert "chore: temporarily disable fuzz chaos tests (#4457)"

This reverts commit f0c953f84a.

* chore: update config

* Update .github/actions/setup-greptimedb-cluster/with-remote-wal.yaml

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-07-31 07:29:31 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436) 
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-07-30 06:17:58 +00:00
Yingwen
78962015dd ci: keep sqlness log by default (#4449) 
* ci: keep sqlness log by default

* chore: not preserve state in makefile by default

* ci: use make
 2024-07-29 17:11:24 +00:00
Weny Xu
e39f49fe56 fix: ensure keep alive is completed in time (#4349) 
* fix: ensure keep alive is completed in time

* chore: apply suggestions from CR

* chore: use write runtime

* refactor: set META_LEASE_SECS to 5

* chore: set etcd replicas to 1

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* fix: set `MissedTickBehavior::Delay`

* chore: apply suggestions from CR
 2024-07-17 06:14:45 +00:00
Weny Xu
08c415c729 ci: retry on error or timeout during installing operator (#4308) 
chore(ci): retry on error or timeout during installing operator
 2024-07-08 03:31:13 +00:00
Weny Xu
0b624dc337 ci: retry on error during installing operator (#4295) 
chore(ci): retry on error during installing operator
 2024-07-05 07:54:31 +00:00
zyy17
c9177cceeb ci: push latest greptimedb nigthly build image (#4260) 2024-07-03 11:14:06 +00:00
Weny Xu
352cc9ddde test: add e2e test for region failover (#4188) 
* test: add e2e test for region failover

* chore: add ci cfg

* chore: reduce parallelism to 8

* fix(ci): enable region failure

* chore: set sqlx LogLevel to Off

* refactor: move help functions to utils
 2024-06-28 06:49:41 +00:00
Weny Xu
bf3ad44584 fix: fix release CI typo (#4147) 
* fix: ci typo

* chore: use aws registry
 2024-06-14 14:29:34 +00:00
Weny Xu
11a903f193 chore(ci): switch to aws registry (#4145) 
chore: switch to aws registry
 2024-06-14 11:46:57 +00:00
Weny Xu
acdfaabfa5 fix(ci): use ld_classic on macOS (#4143) 
fix: use ld_classic on macos
 2024-06-14 08:09:12 +00:00