rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-25 09:20:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,633 Commits 133 Branches 254 Tags
27e376e892ac7f7ad6ea525b3e5e8cdae98869ce
Commit Graph

759 Commits

Author SHA1 Message Date
Weny Xu
27e376e892 feat: implement the CompositeTableRoute (#3918) 
* feat: implement the `CompositeTableRoute`

* chore: update comments
 2024-05-11 11:09:00 +00:00
Weny Xu
89da42dbc1 refactor: refactor frontend cache (#3912) 
* feat: implement the `TableCache`

* refactor: use `TableCache`

* refactor: replace `TableFlowManager` with `TableFlownodeSetCache`

* refactor: introduce cache crate

* chore: add comments

* chore: update comments

* chore: apply suggestions from CR

* chore: rename `cache_invalidator` to `local_cache_invalidator`

* chore(fuzz): set `acquire_timeout` to 30s
 2024-05-11 09:58:18 +00:00
Yingwen
d0820bb26d refactor: Remove PhysicalPlan trait and use ExecutionPlan directly (#3894) 
* refactor: remove PhysicalPlan

* refactor: remove physical_plan mod

* refactor: import

* fix merge error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-11 07:38:03 +00:00
Jeremyhi
9aa2182cb2 refactor: make txn easy to use (#3905) 
refactor: put_if_not_exists and compare_and_put API
 2024-05-11 04:45:04 +00:00
Weny Xu
bca2e393bf refactor: add procedure_loader macro (#3906) 2024-05-11 04:41:21 +00:00
zyy17
b1ef327bac refactor: remove MixOptions and use StandaloneOptions only (#3893) 
* refactor: remove MixOptions and use StandaloneOptions only

* refactor: refactor code by code review comments

1. Use '&self' in frontend_options() and datanode_options();

2. Remove unused 'clone()';

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* ci: fix integration error

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
 2024-05-10 16:08:34 +00:00
Ruihang Xia
115c74791d build(deps): bump snafu to 0.8 (#3911) 
* change Cargo.toml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* global replace

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* handle alias in script engine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-10 13:36:25 +00:00
Ruihang Xia
aec5cca2c7 feat: support distributed EXPLAIN ANALYZE (#3908) 
* feat: fetch and pass per-plan metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl DistAnalyzeExec

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/analyze.rs

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-05-10 12:31:29 +00:00
Weny Xu
06e1c43743 feat: support to drop flow (#3900) 
* feat: support to drop flow

* chore: apply suggestions from CR
 2024-05-10 12:09:37 +00:00
Weny Xu
9d36c31209 feat: introduce the CacheRegistry (#3896) 
* feat: implement the `CacheRegistry`

* refactor: change `CacheInvalidator` signature

* feat: implement `CacheInvalidator`

* feat: add `get_or_register`

* fix: fmt toml

* feat: implement the `CacheRegistryBuilder`

* chore: apply suggestions from CR

* chore: fmt code
 2024-05-10 11:53:42 +00:00
Weny Xu
c91132bd14 feat: introduce TableNameCache & TableInfoCache & TableRouteCache (#3895) 
* feat: implement the `TableNameCache`

* feat: implement the `TableInfoCache`

* feat: implement the `TableRouteCache`

* test: add tests for `TableInfoCache` & `TableRouteCache`

* chore: use `TableId`
 2024-05-10 09:50:44 +00:00
Ruihang Xia
66aa08d815 fix: memory leak in unit test of telemetry (#3897) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-09 14:29:22 +00:00
Weny Xu
b8a325d18c feat: implement CacheContainer & TableFlownodeSetCache (#3885) 
* feat: implement the `CacheContainer`

* feat: implement the `TableFlownodeSetCache`

* chore: remove unused feature

* chore: remove unused `Arc`

* refactor: refactor origin `get` to `get_by_ref`

* chore: update comments

* refactor: refactor `CacheContainer`

* chore: move `CacheContainer` to container.rs

* feat: add metrics

* chore: update tests

* test: add tests for value not exists

* test: add test for get

* chore: apply suggestions from CR
 2024-05-09 09:26:35 +00:00
Weny Xu
5545a8b023 feat: implement drop flow procedure (#3877) 
* feat: implement `destroy_flow_metadata` method

* chore: bump proto to 65c1364

* feat: implement the drop flow procedure

* feat: add `MockFlownodeManager`

* tests: add tests for create flow & drop flow procedure

* chore: apply suggestions from CR

* chore: use `ClusterId`
 2024-05-09 08:23:19 +00:00
Weny Xu
0e05f85a9d feat: pass QueryContext to FlowRequestHeader (#3878) 
* feat: pass `QueryContext` to `DdlTaskRequest`

* feat: pass `QueryContext` to `FlowRequestHeader`

* chore: fmt toml
 2024-05-09 04:57:33 +00:00
zyy17
cc8d6b1200 refactor: move Plugins to plugins.rs and use rwlock (#3862) 
* refactor: move `Plugins` to `plugins.rs` and add unit tests

* refactor: use rwlock instead of mutex to improve performance
 2024-05-07 23:32:22 +00:00
Weny Xu
1b5862223c fix: register regions during procedure recovery (#3859) 
* fix: register regions during procedure recovery

* feat: add `recover` to `Procedure` trait

* refactor: move recovery to `recover` method
 2024-05-07 08:30:41 +00:00
Kelvin Wu
154f561da1 refactor: passing QueryContext to RegionServer (#3829) 
* refactor: passing QueryContext to RegionServer

* refactor: change the return type of build() in QueryContextBuilder

* fix: update greptime-proto reference

* chore: apply suggestion

* chore: revert the last commit

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2024-05-07 15:10:59 +08:00
Ruihang Xia
530353785c refactor: remove re-export from logging (#3865) 
* refactor: remove re-export from logging

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix merge problem

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* run formatter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-05-06 13:26:01 +00:00
discord9
573c19be32 feat(flow): mirror insert req to flow node (#3858) 
* feat: mirror insert req to flow node

* refactor: group_requests_by_peer

* chore: rename `nodes` to `flows` to be more apt

* docs: add TODO

* refactor: split flow&data node grouping to two func

* refactor: mirror_flow_node_request

* chore: add some TODOs

* refactor: use Option in value

* feat: skip non-src table quickly

* docs: add TODO for  `Peer.address`

* fix: dedup
 2024-05-06 11:33:14 +00:00
Weny Xu
6e12e1b84b fix: retrieve all info instead of checking on demand (#3846) 
* fix: renew region lease bug

* refactor: only register regions once

* chore: apply suggestions from CR
 2024-05-06 04:44:47 +00:00
tison
7d447c20c5 chore: try avoid one slice (#3856) 
Signed-off-by: tison <wander4096@gmail.com>
 2024-05-06 03:44:12 +00:00
dennis zhuang
65d47bab56 feat: adds information_schema cluster_info table (#3832) 
* feat: adds server running mode to KvBackendCatalogManager

* feat: adds MetaClient to KvBackendCatalogManager

* feat: impl information_schema.cluster_info table

* fix: forgot files

* test: update information_schema result

* feat: adds start_time and uptime to cluster_info

* chore: tweak cargo and comment

* feat: rename greptime_region_peers to region_peers

* fix: cluster_info result

* chore: simplify sqlness commands

* chore: set peer_id to -1 for frontends

* fix: move cluster_info to greptime catalog

* chore: use official proto

* feat: adds active_time

* chore: apply suggestion

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* chore: STANDALONE for runtime_metrics

---------

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
Co-authored-by: tison <wander4096@gmail.com>
 2024-05-02 02:49:46 +00:00
dimbtp
3b89b9ddd8 refactor: move database client to test (#3820) 
* remove `Database` in `cli export`

* move `Database` to tests-integration

* fix clippy

* move `DatabaseClient` along with `Database`

* `cli export` now use http api

* add TODO for `tests-integration` dependencies

* cleanup code

* 'cli export' test use http api

* remove unsed dependencies in cmd crate

* apply review comments and clean code

* remove unused methods

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-05-01 05:55:13 +00:00
tison
695746193b ci: reduce workflow files by merging actions (#3848) 
* ci: merge license header checker into dev ci

Signed-off-by: tison <wander4096@gmail.com>

* ci: merge nightly-funtional-tests.yml into nightly-ci.yml

Signed-off-by: tison <wander4096@gmail.com>

* fix typos

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-05-01 04:44:00 +00:00
Yingwen
777bc3b89d fix: compiler warnings on Windows (#3844) 
* chore: fix windows warnings

* test: try fix create_metadata_region on windows

* style: fix clippy

* style: fix clippy
 2024-04-30 09:40:11 +00:00
Weny Xu
81f3007f6f refactor: remove TableMetaKey trait (#3837) 
* refactor: use MetaKey trait instead of TableMetaKey

* test: add tests

* test: add more tests

* refactor: use `&str` instead of `String`
 2024-04-30 06:31:45 +00:00
Weny Xu
20cbc039e6 refactor: remove catalog prefix (#3835) 
* refactor: remove catalog prefix

* refactor: remove scope.rs

* fix: fix tests

* chore: update comments

* chore: apply suggestions from CR
 2024-04-30 03:27:33 +00:00
tison
d11b1fa389 chore: correct RepeatedField origin (#3838) 
* chore: correct RepeatedField origin

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-04-30 03:13:54 +00:00
Ruihang Xia
a0f4881c6e feat: physical optimizer RemoveDuplicate to remove duplicate exec plans (#3839) 
* feat: physical optimizer RemoveDuplicate to remove duplicate exec plans

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update document

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness results

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-04-30 03:13:39 +00:00
Kould
aba5e41799 feat: support ALTER TABLE ... MODIFY COLUMN ... ... (#3796) 
* feat: support `ALTER COLUMN xxx TYPE xxx`

* fix: test `test_parse_alter_change_column_type`

* style: code fmt

* style: move to new test: `test_make_alter_column_type_region_request`

* style: simplify the code

* style: remove `v1::region::ChangeColumnType`

* resolve conflicts

* fix: test `test_make_alter_column_type_region_request`

* style: simplify the code

* rebase

* rebase

* rebase

* fix: `ALTER COLUMN ... TYPE` -> `MODIFY COLUMN`

* fix: `parser` -> `self.parser`

* Apply suggestions from code review

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: WenyXu <wenymedia@gmail.com>
 2024-04-30 03:13:33 +00:00
Weny Xu
701aba9cdb refactor: rename flow task to flow (#3833) 
* refactor: rename to `MIN_USER_FLOW_ID`

* refactor: rename to `FLOW_ID_SEQ`

* refactor: rename to `flow_id_sequence`

* refactor: rename to `FlowMetadataManager`

* refactor: rename flow_task.rs to flow.rs

* refactor: rename to FlowInfoManager

* refactor: rename to FlowName

* refactor: rename to FlownodeFlow

* refactor: rename to TableFlow

* refactor: remove TASK

* refactor: rename to __flow

* refactor: rename to flow_id

* refactor: rename to flow_name

* refactor: update comments

* refactor: rename to flow_metadata_manager

* refactor: rename to flow_metadata_allocator

* refactor: rename to FlowMetadataAllocator

* refactor: rename task suffix

* refactor: rename FlowTask to FlowInfo

* refactor: rename FlowTaskScoped to FlowScoped

* refactor: rename FlowTaskId to FlowId

* chore: bump proto to b5412f7

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2024-04-29 14:02:52 +00:00
Weny Xu
b493ea1b38 feat: implement the CreateFlowProcedure (#3810) 
* feat: implement `FlowTaskMetadataAllocator`

* feat: add `FlowTaskMetadataManagerRef` and `FlowTaskMetadataAllocatorRef`

* chore: fix clippy

* feat: add `FlowTaskNameLock`

* feat: implement the `CreateFlowTaskProcedure`

* chore: rename to `CreateFlowProcedure`

* chore: apply suggestions from CR

* feat: invoke create flow procedure

* chore: apply suggestions from CR

* refactor: rename TYPE_NAME

* feat: register the procedure

* chore: apply suggestions from CR

* feat: acquire the lock of sink table name
 2024-04-29 12:34:11 +00:00
Jeremyhi
336db38ce9 refactor!: remove duration type from gRPC (#3825) 
* refactor: remove duration type

* chore: proto
 2024-04-29 02:56:09 +00:00
tison
c387687262 refactor: use secrecy SerectString to hold secrets option (#3804) 
* build: centralize secrecy dependency

Signed-off-by: tison <wander4096@gmail.com>

* add secrecy to sql crate

Signed-off-by: tison <wander4096@gmail.com>

* try impl

Signed-off-by: tison <wander4096@gmail.com>

* update test

Signed-off-by: tison <wander4096@gmail.com>

* make linters happy

Signed-off-by: tison <wander4096@gmail.com>

* bundle secrecy

Signed-off-by: tison <wander4096@gmail.com>

* bundle secrecy

Signed-off-by: tison <wander4096@gmail.com>

* replace secrecy

Signed-off-by: tison <wander4096@gmail.com>

* tidy clones

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* updated

Signed-off-by: tison <wander4096@gmail.com>

* Apply suggestions from code review

Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>

* use BTreeMap

Signed-off-by: tison <wander4096@gmail.com>

* tidy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
 2024-04-29 02:18:18 +00:00
Ruihang Xia
7ef18c0915 feat: impl parser and operator for CREATE FLOW (#3806) 
* feat: impl parser for CREATE TASK

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finish parser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip expr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finish expr

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename output to sink

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix parser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove debug code

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* upload lock file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename symbol

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jeremyhi <jiachun_feng@proton.me>

* remove other task word

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* task name to flow name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* one more comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Jeremyhi <jiachun_feng@proton.me>
 2024-04-28 12:26:49 +00:00
Ruihang Xia
3dac7cbe37 refactor: remove location_opt and DebugFormat (#3830) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2024-04-28 11:18:55 +00:00
Weny Xu
08263995f6 feat: implement the FrontendInvoker (#3824) 
* chore: add `common-frontend`

* feat: add `FrontendInvoker` trait

* feat: implement the `FrontendInvoker`
 2024-04-28 11:11:34 +00:00
tison
e154dc5fd4 refactor: DfUdfAdapter to bridge ScalaUdf (#3814) 
* refactor: DfUdfAdapter to bridge ScalaUdf

Signed-off-by: tison <wander4096@gmail.com>

* tidy impl

Signed-off-by: tison <wander4096@gmail.com>

* for more

Signed-off-by: tison <wander4096@gmail.com>

* for more

Signed-off-by: tison <wander4096@gmail.com>

* for more

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2024-04-28 04:17:06 +00:00
Weny Xu
3112ced9c0 chore: rename all datanode_manager to node_manager (#3813) 2024-04-28 02:34:06 +00:00
Weny Xu
eb3d2ca759 refactor: refactor DatanodeManager to NodeManager (#3811) 
* chore: bump greptime-proto to 2c14c6e

* refactor: refactor `DatanodeManager` to `NodeManager`
 2024-04-26 11:31:03 +00:00
Weny Xu
1ec595134d feat: define CreateFlowTask and DropFlowTask (#3801) 
* chore: bump proto to cefc73f

* feat: add `CreateFlowTask` and `DropFlowTask`

* chore: bump to 87f2b38

* chore: bump to 783682f
 2024-04-25 12:30:31 +00:00
Weny Xu
9206f60b28 feat: implement FlowTaskMetadataManager (#3766) 
* feat: implement `FlowMetadataManager`

* chore: remove dead code

* refactor: change `sink_tables` to `sink_table`

* refactor: add `PartitionId`

* feat: implement FlowTaskNameManager

* refactor: update doc of keys

* fix: return partition id in `tasks`

* refactor: rename to `FlowTaskId`

* chore: add comments

* chore: add `task_id` in `TaskAlreadyExists`

* chore: add comments

* fix: fmt

* refactor: simplify the docoder

* chore: update comments

* feat: implement `FlowTaskScoped` and `CatalogScoped`

* refactor: refactor flow task keys

* refactor: remove metadata mod

* refactor: rename to `FlowTaskInfo`

* chore: add comments

* refactor: rename to `FlowTaskMetadataManager`

* chore: remove dead code

* Apply suggestions from code review

* chore: change to `pub(crate)`

* chore: apply suggestions from CR

* fix: fix fmt

* chore: fmt doc
 2024-04-25 11:59:24 +00:00
Jeremyhi
9524ec83bc fix!: use the right precision (#3794) 
* fix: use write precision

* chore: rename error

* chore: add doc

* chore: by comment

* chore: right test result

* chore: typo

* chore: add doc
 2024-04-25 03:20:10 +00:00
irenjj
62037ee4c8 feat: impl Display for Statement (#3744) 
* feat: impl Display for Statement

* fix: add license header

* fix: inline function manually

* fix: redacte options

* fix: check secret key and replace value

* test: add test for statement display

* fix: fix check

* fix: inline method

* fix: inline methods

* fix: format

* showcase how to write Display impl

Signed-off-by: tison <wander4096@gmail.com>

* for others

Signed-off-by: tison <wander4096@gmail.com>

* create and copy

Signed-off-by: tison <wander4096@gmail.com>

* create rest

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* address comments

Signed-off-by: tison <wander4096@gmail.com>

* fixup quote

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: tison <wander4096@gmail.com>
 2024-04-24 07:09:06 +00:00
Weny Xu
20a933e395 refactor: simplify the PaginationStream (#3787) 
* refactor: simplify the `PaginationStream`

* refactor: refactor decode fn

* fix: fix clippy
 2024-04-24 04:29:41 +00:00
Kould
b619950c70 feat: add ChangeColumnType for AlterKind (#3757) 
* feat: add `ModifyColumn` for `AlterKind`

* chore: additional code comments for `AlterKind::ModifyColumns`

* fix: add nullable check on `ModifyColumn`

* style: codefmt

* style: fix the code based on review suggestions

* style: fix the code based on review suggestions

* style: rename `ModifyColumn` -> `ChangeColumnType`

* style: code fmt

* style: `change_columns_type` -> `change_column_types`
 2024-04-24 04:27:23 +00:00
Yingwen
4685b59ef1 feat: write manifests in background tasks (#3709) 
* chore: truncate wip

* feat: truncate and edit write manifest in background

* refactor: wrap in manifest context

* feat: alter write manifest in background

* chore: fix compiler errors

* feat: flush update manifest in background

* feat: compaction update manifest in background

* feat: set dropping state

* feat: reset drop state

* feat: check state before updating manifest

* test: fix compaction test

* refactor: rename method

* chore: update comment

* chore: discard state guard

* refactor: use atomic cell to store state enum

* chore: fix clippy

* chore: update toml

* chore: remove unused type alias

* feat: check state after writing manifest

* chore: address CR comments

* chore: change status code

* chore: Update src/mito2/src/region.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* fix: executes applier

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2024-04-24 03:09:48 +00:00
LFC
86a989517e refactor: move the version string to common (#3783) 2024-04-23 14:21:34 +00:00
LFC
19a9035f4b fix: operating region guards should be dropped when procedure is done (#3775) 2024-04-23 06:21:53 +00:00