rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2025-12-25 15:40:02 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,749 Commits 104 Branches 223 Tags
release/v0.12-internal
Commit Graph

58 Commits

Author SHA1 Message Date
jeremyhi
fac22575aa refactor!: unify the option names across all components (#5457) 
* refactor: rename grpc options

* refactor: make the arg clearly

* chore: comments on server_addr

* chore: fix test

* chore: remove the store_addr alias

* refactor: cli option rpc_server_addr

* chore: keep store-addr alias

* chore: by comment
 2025-02-06 06:37:14 +00:00
Ning Sun
f5829364a2 fix: security fix, sqlx, hashbrown, idna and CI updates (#5330) 
* fix: security fix, sqlx, hashbrown, idna

* ci: optimize ci cache generation

* feat: update pprof
 2025-01-14 08:19:33 +00:00
Weny Xu
b229c94fba fix(fuzz): ensure all regions leases are renewed (#5294) 
* fix(fuzz): ensure all regions leases are renewed

* fix: fix clippy
 2025-01-06 06:21:41 +00:00
Weny Xu
1910d71cb3 chore: adjust fuzz tests cfg (#5207) 2024-12-20 06:58:51 +00:00
Weny Xu
66f0581f5b fix: ensure table route metadata is eventually rolled back on failure (#5174) 
* fix: ensure table route metadata is eventually rolled back on procedure failure

* fix(fuzz): enhance procedure condition checking

* chore: add logs

* feat: close downgraded leader region actively

* chore: apply suggestions from CR
 2024-12-19 03:29:34 +00:00
Weny Xu
d0245473a9 fix: correct set_region_role_state_gracefully behaviors (#5171) 
* fix: reduce default max rows for fuzz testing

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): increase resource limits for GreptimeDB cluster

* chore(fuzz): increase resource limits for kafka

* fix: correct `set_region_role_state_gracefully` behaviors

* chore: remove Postgres setup from fuzz test workflow

* chore(fuzz): redue resource limits for GreptimeDB & kafka
 2024-12-16 14:01:40 +00:00
Yohan Wal
03ad6e2a8d feat(fuzz): add alter table options for alter fuzzer (#5074) 
* feat(fuzz): add set table options to alter fuzzer

* chore: clippy is happy, I'm sad

* chore: happy ci happy

* fix: unit test

* feat(fuzz): add unset table options to alter fuzzer

* fix: unit test

* feat(fuzz): add table option validator

* fix: make clippy happy

* chore: add comments

* chore: apply review comments

* fix: unit test

* feat(fuzz): add more ttl options

* fix: #5108

* chore: add comments

* chore: add comments
 2024-12-12 04:21:38 +00:00
Weny Xu
4b263ef1cc fix: obsolete wal entires while opening a migrated region (#4993) 
* fix: delete obsolete wal entrie while opening a migrated region

* chore: add logs

* chore: rust fmt

* fix: fix fuzz test
 2024-11-15 10:55:22 +00:00
Yingwen
0da112b335 chore: provide more info in check batch message (#4906) 
* chore: provide more info in check message

* chore: set timeout to 240s

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-10-30 11:56:10 +00:00
Weny Xu
83eb777d21 test: add fuzz test for metric region migration (#4862) 
* test: add fuzz tests for migrate metric regions

* test: insert values before migrating metric region

* feat: correct table num

* chore: apply suggestions from CR
 2024-10-29 15:47:48 +00:00
Ruihang Xia
03f2fa219d feat: optimizer rule for windowed sort (#4874) 
* basic impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement physical rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: install windowed sort physical rule and optimize partition ranges

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add logs and sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: introduce PartSortExec for partitioned sorting

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune exec nodes' properties and metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* debug: add more info on very wrong

* debug: also print overlap ranges

* feat: add check when emit PartSort Stream

* dbg: info on overlap working range

* feat: check batch range is inside part range

* set distinguish partition range param

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: more logs

* update sqlness

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tune optimizer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix windowed sort rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: early terminate sort stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: remove min/max check

* chore: remove unused windowed_sort module, uuid feature and refactor region_scanner to synchronous

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: print more fuzz log

* chore: more log

* fix: part sort should skip empty part

* chore: remove insert logs

* tests: empty PartitionRange

* refactor: testcase

* docs: update comment&tests: all empty

* ci: enlarge etcd cpu limit

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: discord9 <discord9@163.com>
Co-authored-by: evenyag <realevenyag@gmail.com>
 2024-10-29 07:46:05 +00:00
Weny Xu
49004391d3 chore(fuzz): print table name for debugging (#4738) 
* chore(fuzz): print table name for debugging

* chore: apply suggestions
 2024-09-19 09:40:10 +00:00
Weny Xu
d1dfffcdaf chore: enable fuzz test for append table (#4702) 
* chore: enable fuzz test for append table

* fix: fix mysql translator
 2024-09-18 03:01:30 +00:00
dennis zhuang
63f2463273 feat!: impl admin command (#4600) 
* feat: impl admin statement parser

* feat: introduce AsyncFunction and implements it for admin functions

* feat: execute admin functions

* fix: license header

* fix: panic in test

* chore: fixed by code review
 2024-08-26 07:53:40 +00:00
Weny Xu
90301a6250 fix: generate unique timestamp for inserting tests (#4472) 2024-07-31 12:19:43 +00:00
Ruihang Xia
7daf24c47f feat: remove dedicated runtime for grpc, mysql and pg protocols (#4436) 
* feat: remove dedicated runtime for grpc, mysql and pg protocols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove other runtimes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* spawn compact task into compact_runtime

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refine naming

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/servers/tests/mysql/mysql_server_test.rs

Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* turnoff fuzz test matrix fail fast option

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: update rt config for ci tests

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-07-30 06:17:58 +00:00
Weny Xu
8a119aa0b2 feat: add naive region failover test for metric table (#4269) 
* feat: add failover test for metric table

* chore: introduce help marco

* chore: remove incorrect check
 2024-07-05 06:54:23 +00:00
Weny Xu
b5c6c72b02 fix: enhance ColumnOption::DefaultValue formatting for string values (#4287) 2024-07-04 13:16:51 +00:00
Weny Xu
70f7baffda feat(fuzz): enhance condition check of region migration finish (#4283) 2024-07-04 12:14:52 +00:00
Weny Xu
22f4d43b10 fix(fuzz): generate valid string (#4281) 
* fix: generate valid string

* refactor(fuzz): wait for procedure finish at first
 2024-07-04 08:22:39 +00:00
Weny Xu
76fac359cd feat: implement naive fuzz test for region migration (#4252) 
* fix(fuzz): adapt for new partition rules

* feat: implement naive fuzz test for region migration

* chore(ci): add ci cfg

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-07-03 13:30:41 +00:00
Weny Xu
be29e48a60 chore: reduce insertion size of fuzz test (#4243) 
* chore: reduce size of fuzz test

* chore: get env cfg variables
 2024-07-02 13:02:04 +00:00
Weny Xu
6a634f8e5d feat: register & deregister region failure detectors actively (#4223) 
* feat: Use DATANODE_LEASE_SECS from distributed_time_constants for heartbeat pause duration

* feat: introduce `RegionFailureDetectorController` to manage region failure detectors

* feat: add `RegionFailureDetectorController` to `DdlContext`

* feat: add `region_failure_detector_controller` to `Context` in region migration

* feat: register region failure detectors during rollback region migration procedure

* feat: deregister region failure detectors during drop table procedure

* feat: register region failure detectors during create table procedure

* fix: update meta config

* chore: apply suggestions from CR

* chore: avoid cloning

* chore: rename

* chore: reduce the size of the test

* chore: apply suggestions from CR

* chore: move channel initialization into `RegionSupervisor::channel`

* chore: minor refactor

* chore: rename ident
 2024-07-01 05:58:27 +00:00
Weny Xu
352cc9ddde test: add e2e test for region failover (#4188) 
* test: add e2e test for region failover

* chore: add ci cfg

* chore: reduce parallelism to 8

* fix(ci): enable region failure

* chore: set sqlx LogLevel to Off

* refactor: move help functions to utils
 2024-06-28 06:49:41 +00:00
Weny Xu
1e52ba325f feat: introduce chaos crds (#4173) 
feat: add chaos-mesh crds
 2024-06-21 06:28:51 +00:00
taobo
dd06e107f9 test: add fuzz tests for column data type alteration (#4076) 
* feat: support make fuzz-stable in Makefile

* test: add fuzz tests for column data type alteration

* fix: optimize code by cr
 2024-06-04 13:38:57 +00:00
Weny Xu
097f62f459 refactor(fuzz-tests): generate ts value separately (#4056) 
refactor: generate ts value separately
 2024-05-27 14:07:32 +00:00
Weny Xu
048368fd87 feat: invoke flush_table and compact_table in fuzz tests (#4045) 
* feat: invoke `flush_table` and `compact_table` in fuzz tests

* feat: support to flush and compact physical metric table

* fix: avoid to create tables with the same name

* feat: validate values after flushing or compacting table
 2024-05-27 09:26:50 +00:00
Weny Xu
75bddc0bf5 fix(fuzz-tests): avoid to drop in-use database (#4049) 
* fix(fuzz-tests): avoid to drop in-use database

* fix: correct datahome path

* fix: correct `schema_name`

* chore: apply suggestions from CR
 2024-05-27 07:44:59 +00:00
Yohan Wal
9800807fe5 fix(fuzz): sort inserted rows with primary keys and time index (#4008) 
* fix(fuzz): sort inserted rows with primary keys and time index

* fix: correct index when replacing default

* fix: put null behind all values
 2024-05-22 03:32:19 +00:00
Weny Xu
43bf7bffd0 fix: try to fix unstable fuzz test (#4003) 
fix: ignore PoolTimedOut
 2024-05-21 12:57:09 +00:00
Yohan Wal
cfae276d37 feat(fuzz): add validator for inserted rows (#3932) 
* feat(fuzz): add validator for inserted rows

* fix: compatibility with mysql types

* feat(fuzz): add datetime and date type in mysql for row validator
 2024-05-15 07:05:51 +00:00
Weny Xu
89da42dbc1 refactor: refactor frontend cache (#3912) 
* feat: implement the `TableCache`

* refactor: use `TableCache`

* refactor: replace `TableFlowManager` with `TableFlownodeSetCache`

* refactor: introduce cache crate

* chore: add comments

* chore: update comments

* chore: apply suggestions from CR

* chore: rename `cache_invalidator` to `local_cache_invalidator`

* chore(fuzz): set `acquire_timeout` to 30s
 2024-05-11 09:58:18 +00:00
Ruihang Xia
115c74791d build(deps): bump snafu to 0.8 (#3911) 
* change Cargo.toml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* global replace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle alias in script engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-10 13:36:25 +00:00
Weny Xu
aa2934b422 docs: run fuzz tests in dev mode & disable sanitizer (#3871) 
doc: run fuzz tests in dev mode & disable sanitizer
 2024-05-07 14:15:07 +08:00
Weny Xu
f6e2039eb8 test: introduce unstable fuzz create table test (#3788) 
* feat: implement unstable_fuzz_create_table_standalone

* chore: use drop database

* docs: update docs

* chore: add ci config

* chore: add feature gate

* fix: fix clippy

* chore: update ci

* Apply suggestions from code review

* feat: reduce num

* Apply suggestions from code review

* chore: apply suggestions from CR

* Apply suggestions from code review

* chore: reduce `wait_timeout` in health check

* Update .env.example

* refactor: use `init_greptime_connections_via_env`

* refactor: use `init_greptime_connections_via_env`

---------

Co-authored-by: tison <wander4096@gmail.com>
 2024-05-01 06:08:49 +00:00
Yohan Wal
573d369f77 feat(fuzz): add insert logical table target (#3842) 
* refactor: let upper caller control whether to omit column list

* feat(fuzz): add insert logical table target

* ci: add fuzz_insert_logical_table ci cfg
 2024-05-01 03:48:51 +00:00
Yohan Wal
f29aebf89f feat(fuzz): add alter logical table target (#3818) 
* feat(fuzz): add alter logical table target

* chore(ci): add fuzz_alter_logical_table ci cfg
 2024-04-28 06:40:37 +00:00
Yohan Wal
934c7e3fef feat(fuzz): add create logical table target (#3756) 
* feat(fuzz): add create logical table target

* fix: drop physical table after fuzz test

* fix: remove backticks of table name in with clause

* fix: create physical and logical table properly

* chore: update comments

* chore(ci): add fuzz_create_logical_table ci cfg

* fix: create one logical table once a time

* fix: avoid possible duplicate table and column name

* feat: use hard-code physical table

* chore: remove useless phantom

* refactor: create logical table with struct initialization

* chore: suggested changes and corresponding test changes

* chore: clean up
 2024-04-26 09:09:08 +00:00
Ruihang Xia
df01ac05a1 feat: add validate method to CreateExpr (#3772) 
* feat: add validate method to CreateExpr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness reproducer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* verify region create request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix existing test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add tailing empty line

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add more validation

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable metric table fuzz

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor refactor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-04-24 07:29:10 +00:00
Ruihang Xia
0c5f4801b7 build: update toolchain to nightly-2024-04-18 (#3740) 
* chore: update toolchain to nightly-2024-04-17

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix ut

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update fuzz test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update to nightly-2024-04-18

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add document

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* avoid unnecessary allow clippy attrs

Signed-off-by: tison <wander4096@gmail.com>

* help the compiler find the clone is unnecessary and make clippy happy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-04-19 05:42:34 +00:00
Yohan Wal
ffc8074556 feat(fuzz): enable create-if-not-exists option (#3732) 2024-04-18 02:50:57 +00:00
Weny Xu
2398918adf feat(fuzz): support to create metric table (#3617) 
Co-authored-by: tison <wander4096@gmail.com>
 2024-04-09 06:00:04 +00:00
Yohan Wal
682b04cbe4 feat(fuzz): add create database target (#3675) 
* feat(fuzz): add create database target

* chore(ci): add fuzz_create_database ci cfg
 2024-04-09 01:33:29 +00:00
dennis zhuang
87e0189e58 fix!: columns table in information_schema misses some columns (#3639) 
* fix: columns table in information_schema misses some columns

* fix: test_information_schema_dot_columns

* fix: fuzz test

* feat: adds srs_id and refactor some columns with constant vector

* fix: test_information_schema_dot_columns

* chore: update comment

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* build(deps): bump h2 from 0.3.24 to 0.3.26 (#3642)

Bumps [h2](https://github.com/hyperium/h2) from 0.3.24 to 0.3.26.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/v0.3.26/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.24...v0.3.26)

---
updated-dependencies:
- dependency-name: h2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* build(deps): bump whoami from 1.4.1 to 1.5.1 (#3643)

Bumps [whoami](https://github.com/ardaku/whoami) from 1.4.1 to 1.5.1.
- [Changelog](https://github.com/ardaku/whoami/blob/v1/CHANGELOG.md)
- [Commits](https://github.com/ardaku/whoami/compare/v1.4.1...v1.5.1)

---
updated-dependencies:
- dependency-name: whoami
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: adding victoriametrics remote write (#3641)

* feat: adding victoria metrics remote write

* test: add e2e tests for prom and vm remote writes

* fix: construct correct pk list with pre-existing pk (#3614)

* fix: construct correct pk list with pre-existing pk

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update UT

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* test(sqlness): release databases after tests (#3648)

* refactor: rename Greptime_Type to Greptime_type

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ning Sun <sunng@protonmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-04-08 03:20:49 +00:00
niebayes
d33435fa84 feat: introduce wal benchmarker (#3446) 
* feat: introduce wal benchmarker

* chore: add log store metrics

* chore: add some comments to wal benchmarker

* fix: ci

* chore: add more metrics for kafka logstore

* chore: add more timers for kafka logstore

* chore: add more configs

* chore: move humantime to common dependencies

* refactor: refactor wal benchmarker

* fix: apply suggestions from code review

* doc: add a simple README for wal benchmarker

* fix: Cargo.toml

* fix: clippy

* chore: rename wal.rs to wal_bench.rs

* fix: compile
 2024-04-03 03:16:05 +00:00
Weny Xu
e4333969b4 feat(fuzz): add alter table target (#3503) 
* feat(fuzz): validate semantic type of column

* feat(fuzz): add fuzz_alter_table target

* feat(fuzz): validate columns

* chore(ci): add fuzz_alter_table ci cfg
 2024-03-13 14:11:47 +00:00
Zhenchi
b55905cf66 feat(fuzz): add insert target (#3499) 
* fix(common-time): allow building nanos timestamp from parts split from i64::MIN

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(fuzz): add insert target

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: cleanup cargo.toml and polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2024-03-13 10:03:03 +00:00
Weny Xu
0a4444a43a feat(fuzz): validate columns (#3485) 2024-03-11 11:34:50 +00:00
Weny Xu
3f3ef2e7af refactor: separate the quote char and value (#3455) 
refactor: use ident instead of string
 2024-03-07 08:24:09 +00:00