rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 21:32:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,759 Commits 105 Branches 226 Tags
c0aed1d267f7f1d638bedc68456ed7ecc794c6bf
Commit Graph

2759 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
LFC
c0aed1d267 feat: set global runtime size by config file (#4063) 
* set global runtime size

* fix: resolve PR comments

* fix: log the whole option

* fix ci

* debug ci

* debug ci

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2024-06-04 10:03:33 +00:00
discord9
0a07130931 fix(flow): mfp operator missing rows (#4084) 
* fix: mfp missing rows if run twice in same tick

* tests: run mfp for multiple times

* refactor: make mfp less hacky

* feat: make channel larger

* chore: typos
 2024-06-04 09:07:13 +00:00
Weny Xu
a6269397c8 fix: fix EntityTooSmall issue (#4100) 
* fix: fix EntityTooSmall issue

* chore(ci): add minio to coverage

* tests: add test for parquet writer

* chore: move tests to `common-datasource` crate
 2024-06-04 08:43:33 +00:00
Lei, HUANG
a80059b47f fix: recover memtable options when opening physical regions (#4102) 
* fix: recover memtable options when opening physical regions

* chore: fmt

* chore: merge data region options
 2024-06-04 08:20:29 +00:00
Weny Xu
b3a4362626 test: run test_flush_reopen_region and test_region_replay with KafkaLogStore (#4083) 
* feat: add `LogStoreFactory` to `TestEnv`

* feat: add `multiple_log_store_factories` template

* test: run `test_flush_reopen_region` and `test_region_replay` with `KafkaLogStore`

* chore: move deps to workspace

* chore: apply suggestions from CR
 2024-06-04 08:11:15 +00:00
Kelvin Wu
51e2b6e728 fix: display the PartitionBound and PartitionDef correctly (#4101) 
* fix: display the PartitionBound and PartitionDef correctly

* Update src/partition/src/partition.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* fix: fix unit test of partition definition

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-06-04 08:10:44 +00:00
shuiyisong
d1838fb28d refactor: move define_into_tonic_status to common-error (#4095) 
* chore: finish cherry-pick

* chore: remove unused code
 2024-06-04 03:29:15 +00:00
Weny Xu
cd97a39904 chore: enable strip for tests-fuzz crate (#4093) 2024-06-03 14:32:11 +00:00
Weny Xu
4e5dd1ebb0 ci: try to free space after fuzz tests (#4089) 
* chore(ci): remove .tar file after unzip

* chore: free space

* chore: include debug info
 2024-06-02 21:22:22 +00:00
Kelvin Wu
88cdefa41e feat: implement Display for PartitionExpr (#4087) 2024-06-02 21:09:00 +00:00
Ruihang Xia
c2218f8be8 build(deps): bump datafusion 20240528 (#4061) 
* build(deps): bump datafusion 20240528

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* another update

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update expected sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix first/last value

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reformat comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix remaining errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert toml format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix pyo3 feature

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove dead code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* format file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-06-01 14:03:00 +00:00
Ruihang Xia
45fee948e9 fix: display error in correct format (#4082) 
* fix: display error in correct format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add address to RegionServer error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-31 09:25:14 +00:00
discord9
ea49f8a5c4 feat(flow): make write path faster with shared lock (#4073) 
* feat(WIP): make write faster

* feat: read lock on fast path

* chore: per review
 2024-05-31 06:50:22 +00:00
Jeremyhi
43afea1a9d refactor!: remove the tableid in ddl response since tableids is enough (#4080) 
* refactor: remove the tableid in ddl response since tableids is enough

* chore: upgrade proto
 2024-05-31 06:41:53 +00:00
Weny Xu
fcfcf86385 feat: implement WalEntryDistributor, WalEntryReciver (#4031) 
* feat: implement the `WalEntryDistributor` and `WalEntryReceiver`

* test: add tests for `WalEntryDistributor`

* refactor: use bounded channel

* chore: apply suggestions from CR
 2024-05-31 03:03:38 +00:00
Jeremyhi
26b112ab57 refactor: remove upgrade cli tool (#4077) 2024-05-31 00:40:27 +00:00
dennis zhuang
24612f62dd feat: querying from view works (#3952) 
* feat: querying from view works

* feat: use MemoryCatalogProviderList instead of DummyCatalogList

* refactor: revert src/query/src/dummy_catalog.rs

* chore: clean code

* fix: make clippy happy

* fix: toml format

* fix: sqlness

* fix: forgot files

* fix: make sqlness happy

* test: table source, serializer and decoder

* fix: fail to decode plan because of invalid table names

* test: adds more sqlness test for view

* chore: remove unused errors

* fix: comments

* fix: typo

* fix: invalidate view info cache after creating view successfully

* chore: apply suggestion

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: apply suggestion

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: compile error after rebeasing

* chore: style

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: don't export table_name in common-meta

* chore: change ViewInfo::new signature

* docs: leave a TODO for optimize param

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 21:45:56 +00:00
Ruihang Xia
85a231850d fix: add tailing separator to prefix (#4078) 
* fix: add tailing separator to prefix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* project select result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 15:29:26 +00:00
Ruihang Xia
f024054ed3 ci: cargo gc fuzz test runner (#4074) 
* ci: cargo gc fuzz test runner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change profile to dev

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-30 10:41:29 +00:00
discord9
05751084e7 chore: bump to v0.8.1 (#4055) v0.8.1 2024-05-30 07:59:52 +00:00
Jeremyhi
8b6596faa0 feat: avoid some cloning when mirror requests to flownode (#4068) 
* feat: some refactor mirror requests to flownode

* feat: use spawn_bg to avoid impact front-ground write

* feat: add mirror row count metric
 2024-05-30 07:29:13 +00:00
Weny Xu
eab309ff7e fix: avoid acquiring lock during reading stats (#4070) 
* fix: avoid acquiring lock during reading stats

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-05-30 07:08:04 +00:00
discord9
7de336f087 fix(flow): correctness bugs (#4018) 
* fix: optional args of tumble

* fix(WIP): choose

* feat: rename default ts to GREPTIME_TIMESTAMP

* fix: default timestamp name

* fix: reorder write requests

* fix: expire state

* fix: test of tumble

* fix: send buf clear

* fix: ts<start time correct window

* fix: window_start when ts<start_time

* mend

* fix: range begin>range end

* refactor: per reviews

* feat!: ts placeholder rename to __ts_placeholder

* refactor: better condition

* tests(WIP): func sig choose

* tests(WIP): tumble func

* feat: make run_available optional blocking

* tests: tumble transform

* chore: clippy

* fix?: lagged missing data

* fix: flow source break on empty chnl
 2024-05-30 03:49:11 +00:00
Weny Xu
6e9a9dc333 refactor(log_store): remove associated type Namespace and Entry in LogStore (#4038) 
* refactor(log_store): remove associated type `Namespace` in `LogStore`

* fix(test): filter entries

* refactor: ignore incomplete parts

* refactor: simplify `RawEntryReaderFilter`

* chore: avoid cloning

* test: add tests for `maybe_emit_entry`

* refactor: remove `Namespace` trait and rename `LogStoreNamespace` to `Namespace`

* chore: apply suggestions from CR

* refactor: refine `entry` method signature

* feat: ingore  any potential incomplete parts

* refactor: rename `namespace` to `provider`

* chore: add debug assertion

* refactor: associated type `Entry` in `LogStore`

* refactor: renamse `namespace` to `provider`

* refactor: remove unwrap

* refactor: let `remaining_entries` return a optional vector

* test: add basic tests for kafka logstore

* refactor: move `append` method under `cfg(test)`

* refactor: rename `RawEntry` to `Entry`

* refactor: rename `CorruptedLogEntry` to `CorruptedEntry`

* test: add tests for handling corrupted raw entry stream

* refactor: rename `ns` to `provider`

* refactor: remove `entry_stream.rs` file

* chore: remove unused code

* chore: update comments

* chore: apply suggestions from CR

* chore: update comments

* chore: apply suggestions from CR

* chore: remove Deref

* chore: add comments

* fix: ignores tail corrupted data

* chore: add comments

* fix: add `MIN_BATCH_SIZE` limit
 2024-05-29 13:44:01 +00:00
Yingwen
848bd7e553 feat: Implements row group level parallel unordered scanner (#3992) 
* feat: unordered scanner

* feat: support compat

* chore: update debug print

fix: missing ranges in scan parts

* fix: ensure chunk size > 0

* fix: parallel is disabled if there is only one file and memtable

* chore: reader metrics

* chore: remove todo

* refactor: add ScanPartBuilder trait

* chore: pass file meta to the part builder

* chore: make part builder private

* docs: update comment

* chore: remove meta()

* refactor: only prune file ranges in ScanInput

replaces ScanPartBuilder with FileRangeCollector which only collect file
ranges

* chore: address typo

* fix: panic when no partition

* feat: Postpone part distribution

* chore: handle empty partition in mito

* style: fix clippy
 2024-05-29 11:06:08 +00:00
LYZJU2019
f0effd2680 feat: Implement SHOW STATUS (#4050) 
* show status returning empty contents

* return an empty set instead of affected rows

* chore: Update src/query/src/sql.rs

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2024-05-29 04:49:05 +00:00
dennis zhuang
aafb468547 fix: set local or session time_zone not work (#4064) 
* fix: set local or session time_zone not work

* chore: supports PostgreSQL-specific setting time zone
 2024-05-29 00:06:13 +00:00
Weny Xu
4aa756c896 feat: open region in background (#4052) 
* feat: open region in background

* feat: trace opening regions

* feat: wait for the opening region

* feat: let engine to handle the future open request

* fix: fix `test_region_registering`
 2024-05-28 13:58:15 +00:00
Weny Xu
d3860671a8 chore: add LAST_SENT_HEARTBEAT_ELAPSED metric (#4062) 2024-05-28 08:05:37 +00:00
tison
9dd6e033a7 refactor: move Database to client crate behind testing feature (#4059) 
* refactor: move Database to client crate behind testing feature

Signed-off-by: tison <wander4096@gmail.com>

* partial move

Signed-off-by: tison <wander4096@gmail.com>

* catch up more

Signed-off-by: tison <wander4096@gmail.com>

* fix imports

Signed-off-by: tison <wander4096@gmail.com>

* finish

Signed-off-by: tison <wander4096@gmail.com>

* tidy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-05-28 03:21:43 +00:00
Weny Xu
097f62f459 refactor(fuzz-tests): generate ts value separately (#4056) 
refactor: generate ts value separately
 2024-05-27 14:07:32 +00:00
Weny Xu
048368fd87 feat: invoke flush_table and compact_table in fuzz tests (#4045) 
* feat: invoke `flush_table` and `compact_table` in fuzz tests

* feat: support to flush and compact physical metric table

* fix: avoid to create tables with the same name

* feat: validate values after flushing or compacting table
 2024-05-27 09:26:50 +00:00
tison
f9db5ff0d6 build(deps): upgrade opendal to 0.46 (#4037) 
* build(deps): upgrade opendal to 0.46

Signed-off-by: tison <wander4096@gmail.com>

* migrate writes

Signed-off-by: tison <wander4096@gmail.com>

* migrate reads

Signed-off-by: tison <wander4096@gmail.com>

* fixup object safety

Signed-off-by: tison <wander4096@gmail.com>

* fixup names

Signed-off-by: tison <wander4096@gmail.com>

* fixup compilation

Signed-off-by: tison <wander4096@gmail.com>

* fixup compilation

Signed-off-by: tison <wander4096@gmail.com>

* a few Buffer to Vec

Signed-off-by: tison <wander4096@gmail.com>

* Make greptime buildable with opendal 0.46 (#5)

Signed-off-by: Xuanwo <github@xuanwo.io>

* fixup toml check

Signed-off-by: tison <wander4096@gmail.com>

* test_orc_opener

Signed-off-by: tison <wander4096@gmail.com>

* Fix lru cache (#6)

Signed-off-by: Xuanwo <github@xuanwo.io>

* clippy

Signed-off-by: tison <wander4096@gmail.com>

* improve comments

Signed-off-by: tison <wander4096@gmail.com>

* address comments

Signed-off-by: tison <wander4096@gmail.com>

* reduce buf copy

Signed-off-by: tison <wander4096@gmail.com>

* upgrade to reqwest 0.12

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Signed-off-by: Xuanwo <github@xuanwo.io>
Co-authored-by: Xuanwo <github@xuanwo.io>
 2024-05-27 09:12:23 +00:00
Weny Xu
20ce7d428d fix(metric-engine): missing catchup implementation (#4048) 
* fix(metric-engine): missing catchup implementation

* fix: should be `metadata_region_id`
 2024-05-27 07:56:46 +00:00
Weny Xu
75bddc0bf5 fix(fuzz-tests): avoid to drop in-use database (#4049) 
* fix(fuzz-tests): avoid to drop in-use database

* fix: correct datahome path

* fix: correct `schema_name`

* chore: apply suggestions from CR
 2024-05-27 07:44:59 +00:00
tison
c78043d526 build(deps): merge tower deps to workspace (#4036) 
Signed-off-by: tison <wander4096@gmail.com>
 2024-05-27 07:15:30 +00:00
LFC
297105266b feat: enable tcp keepalive for http server (#4019) 
* feat: enable tcp keepalive for http server

* chore: for enterprise's update

* resolve PR comments
 2024-05-27 04:07:36 +00:00
Ruihang Xia
1de17aec74 feat: change EXPIRE WHEN to EXPIRE AFTER (#4002) 
* feat: change EXPIRE WHEN to EXPIRE AFTER

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change remaining

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename create_if_not_exist to create_if_not_exists

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* parse interval expr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-05-27 04:05:55 +00:00
Weny Xu
389ded93d1 chore: add logs for setting the region to writable (#4044) 
* chore: add logs for setting the region to writable

* fix: ignore redundant logs
 2024-05-27 04:01:40 +00:00
Eugene Tolbakov
af486ec0d0 feat(opertor): check if a database is in use before dropping it (#4035) 
feat(opertor): check if database is in use before dropping it
 2024-05-27 03:31:58 +00:00
irenjj
25d64255a3 feat: support table level comment (#4042) 
* feat: support table level comment

* use constants

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-05-27 02:28:52 +00:00
tison
3790020d78 build(deps): upgrade promql-parser to 0.4 (#4047) 
* build(deps): upgrade promql-parser to 0.4

Signed-off-by: tison <wander4096@gmail.com>

* lock

Signed-off-by: tison <wander4096@gmail.com>

* catch up upgrades

Signed-off-by: tison <wander4096@gmail.com>

* concise method

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-05-27 01:51:59 +00:00
Weny Xu
5df3d4e5da feat: implement the LogStoreRawEntryReader and RawEntryReaderFilter (#4030) 
* feat: implement the `LogStoreRawEntryReader`

* feat: implement the `RawEntryReaderFilter`

* test: add tests
v0.9.0-nightly-20240527 		2024-05-24 11:53:15 +00:00
tison
af670df515 ci: skip notification for manual releases (#4033) 
Signed-off-by: tison <wander4096@gmail.com>
 2024-05-24 10:16:06 +00:00
Ruihang Xia
a58256d4d3 feat: round-robin selector (#4024) 
* feat: implement round robin peer selector

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add document and test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-24 07:29:07 +00:00
Weny Xu
466f7c6448 feat: add RawEntryReader and OneshotWalEntryReader trait (#4027) 
* feat: add `RawEntryReader` and `OneShotWalEntryReader` trait

* chore: rename `OneShot` to `Oneshot`

* refacotr: remove `region_id` from `OneshotWalEntryReader`
 2024-05-24 06:30:50 +00:00
Ruihang Xia
0101657649 feat: remove one clone on constructing partition (#4028) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-24 04:01:19 +00:00
taobo
a3a2c8d063 feat: Add TLS support for gRPC service (#3957) 
* feat: Add tls support for grpc service

* feat: add integration test

* fix: integration test

* fix: revert by suggestion

* fix: typos

* fix: optimize code

* fix: optimize code

* docs: update configs
 2024-05-23 19:00:16 +00:00
Yingwen
dfc1acbb2a fix: notifies all workers once a region is flushed (#4016) 
* fix: notify workers to handle stalled requests if flush is finished

* chore: change stalled count to gauge

* feat: process stalled requests eagerly
 2024-05-23 12:45:00 +00:00
Lei, HUANG
0d055b6ee6 refactor: remove unused log config (#4021) 2024-05-23 08:59:42 +00:00