rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-04 04:12:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,629 Commits 104 Branches 225 Tags
a5c499572c7434b1685c42323a8d339e7f70c00f
Commit Graph

1629 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Ruihang Xia
a5c499572c feat: open region in parallel (#2451) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-20 07:40:17 +00:00
JeremyHi
ca50ba5dc4 fix: remark region as inactive on leader changed (#2446) 
* fix: remark reigon as inactive on leader changed

* chore: by comment
 2023-09-20 06:37:27 +00:00
Yingwen
17e560c909 feat(mito): Allow to retry create request and alter request (#2447) 
* feat: RegionMetadataBuilder allow adding/dropping columns multiple times

* test: test add if not exists/drop if exists

* feat: change validator and add need_alter

* test: fix tests and test need_alter

* test: test alter retry

* feat: open before create

* style: fix clippy
 2023-09-20 06:36:46 +00:00
Weny Xu
339e12c64a fix: fix alter table verification (#2437) 
* fix: fix verify alter

* refactor: move AlterTable UpdateMetadata to last step

* refactor: send region request in parallel

* Update src/table/src/metadata.rs

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>

* Update src/table/src/metadata.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

---------

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-09-19 13:40:48 +00:00
Ruihang Xia
0f79ccab31 refactor: remove the old mito engine (#2443) 
Co-authored-by: Even Yag <realevenyag@gmail.com>
 2023-09-19 09:30:13 +00:00
Yingwen
7b606ed289 feat(mito): make use of options in RegionCreate/OpenRequest (#2436) 
* refactor: move RegionOptions to options mod

* refactor: define compaction strategy in region/options.rs

* feat: use duration for time window

* refactor: rename CompactionStrategy to CompactionOptions

* feat: use serde to parse options

* feat: parse options

* feat: set options on creation/opening

* test: test create/open with options

* chore: remove todo

* feat: get compaction ttl and options from RegionOptions

* style: fix clippy

* chore: Remove unused engine_options

* style: fix clippy

* chore: remove todo
 2023-09-19 09:06:09 +00:00
Weny Xu
1fb2d95c5f fix: fix open region missing path (#2441) 
* fix: fix open region missing path

* fix: correct log

* chore: apply suggestions from CR

* fix: fix tests
 2023-09-19 08:50:59 +00:00
Wei
8ee62a7d90 fix: parse i64 to different kinds of timestamp (#2440) 
* feat: support i64 to multi-timestamp.

* chore: fmt
 2023-09-19 08:26:25 +00:00
Ruihang Xia
802229de87 fix: type cast bugs found by sqlness (#2438) 
* update valid results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* accomplish datatype

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* cast null

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix unit tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-19 08:20:41 +00:00
Zhenchi
deac284973 refactor: RegionRequestHandler -> RegionQueryHandler (#2439) 
* refactor: RegionRequestHandler -> RegionQueryHandler

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: rename FrontendRegionQueryHandler

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: privte RegionInvoker

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-19 08:19:58 +00:00
Wei
5805e8d4b6 feat: type conversion between Values (#2394) 
* feat: add cast() in datatype trait.

* feat: add cast for primitive type

* feat: add unit test cases

* test: add datetime/time cases.

* refactor: time_type cast function.

* chore: typos.

* refactor code.

* feat: add can_cast_type func.

* chore: rename cast to try_cast

* feat: impl cast_with_opt

* chore: pub use cast_with_opt

* chore: add timezone for test

* Update src/common/time/src/date.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* chore: duration type

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-09-18 14:25:38 +00:00
dennis zhuang
342cc0a4c4 fix: compile error after updating protos (#2435) 2023-09-18 12:12:39 +00:00
Weny Xu
df6c79a378 fix: check version before alter region (#2433) 
* fix: check version before alter region

* chore: apply suggestions from CR

* Update src/mito2/src/worker/handle_alter.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-09-18 11:49:26 +00:00
dennis zhuang
5566f34bd1 feat: make scripts table work again (#2420) 
* feat: make scripts table work again

* chore: typo

* fix: license header

* Update src/table/src/metadata.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: cr comments

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-18 11:43:21 +00:00
Wei
14e6998d41 feat: impl duration datatype and vectors (#2180) 
* feat: impl datatype, vector traits for duration.

* feat: duration and grpc.

* test: add unit test cases.

* chore: style and test case.

* fix: update greptime-proto version and helper.rs

* chore: fix type name.

* Update src/datatypes/src/data_type.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: cr.

* chore: fix greptime-proto

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-09-18 11:28:06 +00:00
Weny Xu
43476e1ff9 refactor: rename coordination to require_lease_before_startup (#2431) 2023-09-18 11:07:42 +00:00
Weny Xu
c42cce57ca fix: fix incorrect matches (#2430) 
* fix: fix incorrect matches

* fix: fix incorrect status code
 2023-09-18 10:53:32 +00:00
Zhenchi
d6d46378a1 test: fix some integration tests (#2432) 
* test: fix some integration tests

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* test: add timezone setting

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-18 10:52:14 +00:00
Ruihang Xia
fbbf3978d9 fix: render comment in SHOW CREATE TABLE (#2427) 
* feat: add comment field to ColumnDef

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-18 10:51:10 +00:00
dennis zhuang
b0c56a3e23 feat: type alias (#2331) 
* fix: remove location from error msg

* feat: adds transformer for sqlparser statements

* feat: supports type alias

* fix: typo

* fix: license header

* test: adds timestamp_types test

* refactor: transform

* fix: rebase develop and fix tests

* fix: compile error

* chore: delete src/datanode/src/sql/create_external.rs
 2023-09-18 09:43:02 +00:00
Zhenchi
73af1368bd test: more integration test cases for external table (#2429) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-18 09:21:51 +00:00
Weny Xu
2c3ff90dbc feat: start services after first heartbeat response processed (#2424) 
* feat: start services after first heartbeat response processed

* refactor: watch changes in RegionAliveKeeper

* feat: add coordination to DatanodeOptions

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: enable coordination in sqlness
 2023-09-18 08:49:26 +00:00
Zhenchi
3a39215f11 feat: migrate file engine from table to reigon (#2365) 
* feat: migrate file engine from table to reigon

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update src/file-engine/src/engine.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* feat: specify ts index for file engine

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: handle time index for external table

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: some integration testsg

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add file schema and table schema compatibility

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: compatible file schema to region schema

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add error msg

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: simplify close

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: implement set_writable

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: tests-integration compilation

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>
 2023-09-18 08:02:43 +00:00
JeremyHi
e7e254cd11 feat: all distributed time together (#2423) 2023-09-17 15:18:52 +00:00
Yingwen
4a82926d72 docs: fix cargo doc errors and warnings (#2421) 
* docs: fix cargo doc warnings and errors

* docs: fix warnings

* docs: fix warnings

* chore: rm src/common/function-macro/src/lib.rs
 2023-09-17 11:45:15 +00:00
Ruihang Xia
92824d1c66 fix: update several sqlness results (#2422) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-17 11:33:40 +00:00
Yingwen
55ae5e5b66 feat(mito): Implements compaction scheduler (#2413) 
* feat: allow multiple waiters in compaction request

* feat: compaction status wip

* feat: track region status in compaction scheduler

* feat: impl compaction scheduler

* feat: call compaction scheduler

* feat: remove status if nothing to compact

* feat: schedule compaction after flush

* feat: set compacting to false after compaction finished

* refactor: flush status only needs region id and version control

* refactor: schedule_compaction don't need region as argument

* test: test flush/scheduler for empty requests

* test: trigger compaction in test

* feat: notify scheduler on truncated

* chore: Apply suggestions from code review

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

---------

Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-09-17 09:15:11 +00:00
Ruihang Xia
693e8de83a feat: scope spawned task with trace id (#2419) 
* feat: scope spawned task with trace id

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-17 09:05:28 +00:00
JeremyHi
542e863ecc fix: missing datanode id on keep lease (#2415) 2023-09-17 07:57:17 +00:00
Ruihang Xia
49310acea1 refactor: rename common-function-macro subcrate (#2418) 
* rename common-function-macro to common-macro

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* put impl into their own file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-17 07:56:41 +00:00
Weny Xu
5b08e03944 feat: sync regions between RegionServer and RegionAliveKeeper (#2417) 
* feat: sync regions between RegionServer and RegionAliveKeeper

* Apply suggestions from code review

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* refactor: rename event name

* chore: apply suggestions

---------

Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-09-17 07:55:44 +00:00
JeremyHi
98a40bae95 feat!: unify naming with options (#2416) 2023-09-17 07:24:57 +00:00
JeremyHi
342a6d071f feat: heartbeat request with header (#2412) 
* feat: heartbeat request with header

* chore: frontend send heartbeat with a longer interval
 2023-09-16 09:56:41 +00:00
Baasit
0a692aafb0 feat: clap wrapper around sqlness (#2400) 
* feat: wrapped sqlness with clap to provide nice interface

* fix: added spaces and changed -f flag to bool
 2023-09-16 08:53:08 +00:00
dennis zhuang
627c5b7419 feat: move table operations from frontend to operator crate (#2411) 
* feat: move table operations from frontend to operator crate

* chore: blank line

* fix: toml format

* chore: move constants
 2023-09-16 07:58:45 +00:00
Ruihang Xia
5e35087b67 fix: generate region path with given prefix (#2409) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-16 03:23:36 +00:00
Ruihang Xia
c149c123c3 feat: reopen corresponding regions on starting datanode (#2399) 
* separate config and datanode impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* partial implement of fetching region id list

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reopen all regions on starting region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness & assign default datanode id

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* set writable on lease

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply cr suggs.

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/datanode/src/datanode.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-09-15 13:30:20 +00:00
Vanish
0bd6b9bb39 feat: implement truncate region for mito2 (#2335) 
* feat: implement truncate region for mito2.

* chore: add license header and fix typos

* Update src/mito2/src/worker/handle_truncate.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* cr

* chore: consider the flush task being executed before truncating the region.

* test

* feat: check flush and compaction tasks

* chore: remove useless changes

* Update src/mito2/src/manifest/action.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/mito2/src/worker/handle_flush.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: CR, consider sequence number

* test: use EventListener to test the flush task during truncate

* fix: fix listener error

* Update src/mito2/src/engine/truncate_test.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: cr

* fix: remove set None

* Update src/mito2/src/region/version.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update src/mito2/src/worker/handle_flush.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update src/mito2/src/worker/handle_truncate.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* doc: add some doc for FlushTruncateListener and RegionTruncate

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-09-15 13:20:01 +00:00
Bamboo1
6aec30a1a8 feat: reserve internal column (#2396) 
* feat: reserve internal column

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* test: add function test

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* fix: spell typos

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

* chore: resolve conversation

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>

---------

Signed-off-by: ZhuZiyi <zyzhu2001@gmail.com>
 2023-09-15 11:19:40 +00:00
LFC
a688760563 fix: validate partition columns (#2393) 
fix: partition column must belong to primary keys or equals to time index
 2023-09-15 10:07:32 +00:00
LFC
4b13c88752 fix: resolve more integration tests (#2406) 
* fix: resolve more integration tests

* Update tests-integration/tests/http.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2023-09-15 09:43:16 +00:00
JeremyHi
9572b1edbb feat: region storage path (#2404) 
* feat: region storage path

* Update src/common/meta/src/key/datanode_table.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

* chore: by cr

* feat: upgrade proto

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2023-09-15 09:07:54 +00:00
dennis zhuang
43e3c94fd1 refactor: catalog managers (#2405) 
* feat: rename catalog::local to catalog::memory

* refactor: catalog managers

* chore: license header
 2023-09-15 08:48:14 +00:00
Ruihang Xia
364b99a14c fix: enable ignored promql unit tests (#2403) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-15 07:37:30 +00:00
shuiyisong
a8ae386a57 chore: add #[serde(default)] to new added engine field (#2402) 
chore: add serde default to new field
 2023-09-15 07:11:57 +00:00
LFC
fe5679e77e refactor: remove table ident (#2368) 
* refactor:
1. remove TableIdent, use TableId directly
2. use the latest greptime-proto
3. independently invalidate table id cache and table name cache

* rebase

* fix: resolve PR comments

* fix: resolve PR comments
 2023-09-15 05:14:40 +00:00
JeremyHi
8e70b9e982 feat: remove deprecated metadata keys (#2398) 
* feat: remove deprecated metadata keys

* feat: this time, weny indeed said [removes it]
 2023-09-15 02:11:21 +00:00
JeremyHi
d1adb915bf feat: set readonly first when deregister region (#2391) 
* feat: set readonly first when deregister region

* revert distxxx
 2023-09-14 12:12:38 +00:00
Yingwen
a84a8ad04f fix: alter table procedure panics while renaming table (#2397) 
* fix: procedure panic on renaming table

* test: fix test_insert_and_select invalid arguments

* test: fix test_standalone_insert_and_query using wrong semantic type

* test: fix test_distributed_insert_delete_and_query semantic type
 2023-09-14 11:50:00 +00:00
JeremyHi
7bb8a5999c feat!: add engine name to DatanodeTableValue (#2395) 
* feat: add engine name to DatanodeTableValue

* fix: by cr
 2023-09-14 09:50:35 +00:00