rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 21:32:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,786 Commits 105 Branches 226 Tags
1f315e300f271dfc2fa57a0b5920eb4182849478
Commit Graph

2786 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Weny Xu
1f315e300f fix: retry on unknown error (#4138) 2024-06-13 11:15:38 +00:00
Weny Xu
573e25a40f chore: run fuzz tests with disk cache (#4118) 
* chore: run fuzz tests with disk cache

* fix: print error messages correctly
 2024-06-13 09:07:12 +00:00
Lei, HUANG
f8ec46493f refactor: simplify parquet writer (#4112) 
* refactor: simplify parquet writer

* chore: fix clippy

* refactor: use AsyncArrowWriter instead of BufferedWriter

* refactor: remove BufferedWriter

* fix: add chunk parameter to avoid entity too small issue

* refactor: use AtomicUsize instead of Mutex

* fix: add chunk argument to stream_to_parquet

* chore: fmt

* wip: fail check

* fix: check

* fmt

* refactor: use impl Future instead of async_trait

* fmt

* refactor: use associate types
 2024-06-13 07:32:47 +00:00
Weny Xu
14a2d83594 chore: remove unused code (#4135) 
* chore: remove unused code

* Update src/mito2/src/wal/entry_reader.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-06-12 13:18:33 +00:00
Yingwen
65f8b72d34 feat: Implement RegionScanner for SeqScan (#4060) 
* feat: ordered builder wip

* feat: impl RegionScanner for SeqScan

* feat: implement scan_partition and build_stream

* chore: return SeqScan as RegionScanner

* fix: group parts

* feat: split parts

* chore: reader metrics

* chore: metrics

* chore: remove unused codes

* chore: support holding a group of ranges in ScanPart

* feat: group ScanParts to ScanParts

* feat: impl SeqScanner again

* chore: observe build cost in ScannerMetrics

* chore: fix compiler warnings

* style: fix clippy

* docs: update config docs

* chore: forward DisplayAs to scanner

* test: update sqlness tests

* chore: update debug fmt

* chore: custom debug for timestamp

fix test compiling issue with common-macro when running
cargo nextest -p common-time

* chore: update debug format

* feat: update fmt for scan part

* chore: fix warning

* fix: sanitize parallelism

* feat: split parts

* test: fix config api test

* feat: update logs

* chore: Revert "chore: remove unused codes"

This reverts commit b548b30a01eeded59b1a0a8d89f9293ca63afc41.

* chore: Revert "docs: update config docs"

This reverts commit a7997e78d6ddcf635560574de8c1948c495bdd12.

* feat: each partition scan files in parallel

* test: fix config api test

* docs: fix typo

* chore: address comments, simplify tests

* feat: global semaphore

* feat: always spawn task

* chore: simplify default explain output format

* handle output partiton number is 0

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-06-12 08:21:30 +00:00
LFC
9473daab8b fix: explicitly set config instead of using changable default in tests (#4132) 
* fix: explicitly set config instead of using changable default in tests

* fix: resolve PR comments
 2024-06-11 10:51:01 +00:00
LFC
5a6021e34f refactor: remove substrait ser/de for region query in standalone (#3812) 
* refactor: remove substrait serde for region query in standalone

* fix ci

* move QueryRequest to common-query

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* format toml file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: format toml

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-06-11 09:33:47 +00:00
discord9
1b00526de5 fix(flow): infer table schema correctly (#4113) 
* refactor: make individual col name optional

* chore: rename TypedPlan's `typ` to `schema`

* feat: add optional col name to typed plan

* feat: pass col name all along

* feat: correct infer output table schema

* chore: unused import

* fix: error when key is not projected

* refactor: per review

* chore: fmt
 2024-06-11 08:57:47 +00:00
Yingwen
5533bd9293 chore(common-macro): remove features covered by full (#4131) 2024-06-11 07:44:53 +00:00
Ning Sun
587e99d806 fix: macro crate cannot be compiled alone (#4130) 
* fix: macro crate cannot be compiled alone

* Update src/common/macro/Cargo.toml

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: tison <wander4096@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-06-11 05:36:29 +00:00
Yingwen
9cae15bd1b fix: executes pending ddls if region memtable is empty while scheduling next flush (#4119) 
* ci: enable debug log

* chore: test to reproduce panic

* chore: Revert "ci: enable debug log"

This reverts commit 17eff2a045.

* test: add test for alter during flush

* fix: clear status if region has nothing to flush

It will also executes pending ddls and requests

* docs: fix typo
 2024-06-11 00:10:17 +00:00
cjw
d8b51cfaba refactor: remove double checks of memtable size (#4117) 
* refactor: remove unnecessary unwrap

* Update src/mito2/src/region/version.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: Kermit <chenjiawei1@xiaohongshu.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-06-07 23:34:03 +00:00
Weny Xu
e142ca40d7 feat: invoke handle_batch_open_requests (#4107) 
* feat: open all regions via invoking `handle_batch_open_requests`

* tests: add sqlness tests

* refactor: avoid cloning

* chore: apply suggestions from CR

* chore: update config.md

* chore: apply suggestions from CR
 2024-06-07 09:07:45 +00:00
Yingwen
e982d2e55c fix: Update region Version in the worker loop (#4114) 
* feat: handle region edit result

* feat: handle edit result

* feat: handle truncate result

* feat: flush compaction

* feat: invoke in worker

* feat: remove unused fields

* style: fix clippy

* feat: remove applier

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-06-07 06:27:16 +00:00
Weny Xu
09e0e1b246 chore: run fuzz tests with kafka remote wal (#4105) 
* chore: add fuzz tests with kafka

* chore(ci): use minio

* chore: add empty line

* chore(ci): refactor

* chore: add empty line

* fix: update config

* fix: add default value for `MetaClientOptions`

* fix: remove redundant `debug_assert`

* chore: run fuzz tests with disk cache

* chore: remove redundant minio setup

* chore: cache targets

* Revert "chore: run fuzz tests with disk cache"

This reverts commit d81783187d.

* chore: fix typo

* chore: apply suggestions from CR

* Revert "fix: remove redundant `debug_assert`"

This reverts commit 09b899eed1.
 2024-06-07 03:47:40 +00:00
irenjj
9c42825f5d feat: Implement SHOW CREATE FLOW (#4040) 
* feat: Implement SHOW CREATE FLOW

* fmt

* stmt for display

* Update src/operator/src/statement.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* test: add sqlness test

* fix test

* parse query in parser

* test: move test to standalone

* reuse ParserContext::new()

* Update tests/cases/standalone/show_create_flow.result

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* add line breaks

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-06-07 03:24:56 +00:00
Jeremyhi
4719569e4f feat: support gRPC cancellation (#4092) 
* feat: support cancellation

* chore: add unit test for cancellation

* chore: minor refactor

* feat: we do not need to spawn in distributed mode

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
v0.9.0-b03cb3860-20240606-1717661661 		2024-06-06 08:27:25 +00:00
Jeremyhi
b03cb3860e chore: reduce some burden on the write path (#4110) 
* chore: remove unnecessary checking

* chore: avoid do the same thing in a loop
 2024-06-06 06:45:19 +00:00
shuiyisong
2ade511f26 feat: introduce pipeline crate (#4109) 
* chore: introduce pipeline crate

* chore: fix typo
 2024-06-05 17:23:25 +00:00
Weny Xu
16b85b06b6 chore: remove gc before running fuzz tests (#4108) 2024-06-05 11:59:29 +00:00
Ruihang Xia
03cacf9948 ci: cargo gc all fuzz test runner (#4081) 
* ci: cargo gc all fuzz test runner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore(ci): disable caching targets

* chore(ci): remove .tar file after unzip

* fix cargo-gc command

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-06-05 09:58:29 +00:00
Weny Xu
c23f8ad113 feat: implement the handle_batch_open_requests (#4075) 
* feat: implement the `handle_batch_open_requests`

* refactor: refactor `handle_batch_open_requests` method signature

* test: add tests for `handle_batch_open_requests`

* chore: fmt code

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-06-05 09:22:34 +00:00
Weny Xu
e0a2c5a581 chore(ci): remove redundant sqlness test config (#4106) 2024-06-05 08:39:39 +00:00
zyy17
417ab3b779 ci: add 'make run-cluster-with-etcd' to run greptimedb cluster by using docker-compose (#4103) 2024-06-05 08:07:29 +00:00
tison
1850fe2956 feat: show create table only for base table (#4099) 
* feat: show create table only for base table

Signed-off-by: tison <wander4096@gmail.com>

* add new cases

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-06-04 21:29:07 +00:00
taobo
dd06e107f9 test: add fuzz tests for column data type alteration (#4076) 
* feat: support make fuzz-stable in Makefile

* test: add fuzz tests for column data type alteration

* fix: optimize code by cr
 2024-06-04 13:38:57 +00:00
sarailQAQ
98c19ed0fa feat: implement drop multiple tables (#4085) 
* feat: implement drop multiple tables

* fix: pass fmt and clippy checks

* add: drop multiple sqlness test

* update: accept review suggestions

* update: accept reviem suggestion

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* fix: pass clippy check

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-06-04 13:11:41 +00:00
LFC
c0aed1d267 feat: set global runtime size by config file (#4063) 
* set global runtime size

* fix: resolve PR comments

* fix: log the whole option

* fix ci

* debug ci

* debug ci

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-06-04 10:03:33 +00:00
discord9
0a07130931 fix(flow): mfp operator missing rows (#4084) 
* fix: mfp missing rows if run twice in same tick

* tests: run mfp for multiple times

* refactor: make mfp less hacky

* feat: make channel larger

* chore: typos
 2024-06-04 09:07:13 +00:00
Weny Xu
a6269397c8 fix: fix EntityTooSmall issue (#4100) 
* fix: fix EntityTooSmall issue

* chore(ci): add minio to coverage

* tests: add test for parquet writer

* chore: move tests to `common-datasource` crate
 2024-06-04 08:43:33 +00:00
Lei, HUANG
a80059b47f fix: recover memtable options when opening physical regions (#4102) 
* fix: recover memtable options when opening physical regions

* chore: fmt

* chore: merge data region options
 2024-06-04 08:20:29 +00:00
Weny Xu
b3a4362626 test: run test_flush_reopen_region and test_region_replay with KafkaLogStore (#4083) 
* feat: add `LogStoreFactory` to `TestEnv`

* feat: add `multiple_log_store_factories` template

* test: run `test_flush_reopen_region` and `test_region_replay` with `KafkaLogStore`

* chore: move deps to workspace

* chore: apply suggestions from CR
 2024-06-04 08:11:15 +00:00
Kelvin Wu
51e2b6e728 fix: display the PartitionBound and PartitionDef correctly (#4101) 
* fix: display the PartitionBound and PartitionDef correctly

* Update src/partition/src/partition.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* fix: fix unit test of partition definition

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-06-04 08:10:44 +00:00
shuiyisong
d1838fb28d refactor: move define_into_tonic_status to common-error (#4095) 
* chore: finish cherry-pick

* chore: remove unused code
 2024-06-04 03:29:15 +00:00
Weny Xu
cd97a39904 chore: enable strip for tests-fuzz crate (#4093) 2024-06-03 14:32:11 +00:00
Weny Xu
4e5dd1ebb0 ci: try to free space after fuzz tests (#4089) 
* chore(ci): remove .tar file after unzip

* chore: free space

* chore: include debug info
 2024-06-02 21:22:22 +00:00
Kelvin Wu
88cdefa41e feat: implement Display for PartitionExpr (#4087) 2024-06-02 21:09:00 +00:00
Ruihang Xia
c2218f8be8 build(deps): bump datafusion 20240528 (#4061) 
* build(deps): bump datafusion 20240528

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* another update

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update expected sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix first/last value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reformat comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix remaining errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert toml format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 feature

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* format file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-06-01 14:03:00 +00:00
Ruihang Xia
45fee948e9 fix: display error in correct format (#4082) 
* fix: display error in correct format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add address to RegionServer error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-31 09:25:14 +00:00
discord9
ea49f8a5c4 feat(flow): make write path faster with shared lock (#4073) 
* feat(WIP): make write faster

* feat: read lock on fast path

* chore: per review
 2024-05-31 06:50:22 +00:00
Jeremyhi
43afea1a9d refactor!: remove the tableid in ddl response since tableids is enough (#4080) 
* refactor: remove the tableid in ddl response since tableids is enough

* chore: upgrade proto
 2024-05-31 06:41:53 +00:00
Weny Xu
fcfcf86385 feat: implement WalEntryDistributor, WalEntryReciver (#4031) 
* feat: implement the `WalEntryDistributor` and `WalEntryReceiver`

* test: add tests for `WalEntryDistributor`

* refactor: use bounded channel

* chore: apply suggestions from CR
 2024-05-31 03:03:38 +00:00
Jeremyhi
26b112ab57 refactor: remove upgrade cli tool (#4077) 2024-05-31 00:40:27 +00:00
dennis zhuang
24612f62dd feat: querying from view works (#3952) 
* feat: querying from view works

* feat: use MemoryCatalogProviderList instead of DummyCatalogList

* refactor: revert src/query/src/dummy_catalog.rs

* chore: clean code

* fix: make clippy happy

* fix: toml format

* fix: sqlness

* fix: forgot files

* fix: make sqlness happy

* test: table source, serializer and decoder

* fix: fail to decode plan because of invalid table names

* test: adds more sqlness test for view

* chore: remove unused errors

* fix: comments

* fix: typo

* fix: invalidate view info cache after creating view successfully

* chore: apply suggestion

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: apply suggestion

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: compile error after rebeasing

* chore: style

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: don't export table_name in common-meta

* chore: change ViewInfo::new signature

* docs: leave a TODO for optimize param

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 21:45:56 +00:00
Ruihang Xia
85a231850d fix: add tailing separator to prefix (#4078) 
* fix: add tailing separator to prefix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* project select result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 15:29:26 +00:00
Ruihang Xia
f024054ed3 ci: cargo gc fuzz test runner (#4074) 
* ci: cargo gc fuzz test runner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change profile to dev

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 10:41:29 +00:00
discord9
05751084e7 chore: bump to v0.8.1 (#4055) v0.8.1 2024-05-30 07:59:52 +00:00
Jeremyhi
8b6596faa0 feat: avoid some cloning when mirror requests to flownode (#4068) 
* feat: some refactor mirror requests to flownode

* feat: use spawn_bg to avoid impact front-ground write

* feat: add mirror row count metric
 2024-05-30 07:29:13 +00:00
Weny Xu
eab309ff7e fix: avoid acquiring lock during reading stats (#4070) 
* fix: avoid acquiring lock during reading stats

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-05-30 07:08:04 +00:00
discord9
7de336f087 fix(flow): correctness bugs (#4018) 
* fix: optional args of tumble

* fix(WIP): choose

* feat: rename default ts to GREPTIME_TIMESTAMP

* fix: default timestamp name

* fix: reorder write requests

* fix: expire state

* fix: test of tumble

* fix: send buf clear

* fix: ts<start time correct window

* fix: window_start when ts<start_time

* mend

* fix: range begin>range end

* refactor: per reviews

* feat!: ts placeholder rename to __ts_placeholder

* refactor: better condition

* tests(WIP): func sig choose

* tests(WIP): tumble func

* feat: make run_available optional blocking

* tests: tumble transform

* chore: clippy

* fix?: lagged missing data

* fix: flow source break on empty chnl
 2024-05-30 03:49:11 +00:00