rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-07 22:02:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,274 Commits 103 Branches 226 Tags
8bea85395444e640cfef93cc38ff6445213677c8
Commit Graph

404 Commits

Author SHA1 Message Date
Weny Xu
37dad206f4 fix: fix wait procedure watcher bug (#1987) 2023-07-18 09:07:31 +00:00
dennis zhuang
b81570b99a feat: impl time type (#1961) 
* chore: remove useless Option type in plugins (#1544)

Co-authored-by: paomian <qtang@greptime.com>

* chore: remove useless Option type in plugins (#1544)

Co-authored-by: paomian <qtang@greptime.com>

* chore: remove useless Option type in plugins (#1544)

Co-authored-by: paomian <qtang@greptime.com>

* chore: remove useless Option type in plugins (#1544)

Co-authored-by: paomian <qtang@greptime.com>

* feat: first commit for time type

* feat: impl time type

* fix: arrow vectors type conversion

* test: add time test

* test: adds more tests for time type

* chore: style

* fix: sqlness result

* Update src/common/time/src/time.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* chore: CR comments

---------

Co-authored-by: localhost <xpaomian@gmail.com>
Co-authored-by: paomian <qtang@greptime.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-07-18 02:55:28 +00:00
LFC
dd62f4c407 feat: tool to migrate table metadata values (#1971) 
feat: tool to migrate table metadata values when upgrading to version 0.4
 2023-07-17 15:06:32 +08:00
LFC
7cf6c2bd5c refactor: trying to replace TableGlobalValue, part 1 (#1956) 
* refactor: trying to replace TableGlobalValue, part 1

* fix: resolve PR comments
 2023-07-17 11:32:46 +08:00
Yingwen
ef7c5dd311 feat(mito): Implement WorkerGroup to handle requests (#1950) 
* feat: engine worker framework

* feat: worder comments

* feat: divide worker requests by type

* feat: handlers for worker thread

* refactor: rename requests to ddl and dml requests

* feat: methods to stop and submit requests

* refactor: rename request queue to request buffer

* refactor: remove ddl and dml request

* feat: send request to worker

* test: test stop

* docs(mito): worker group docs

* style: fix clippy

* docs: update WorkerGroup comment

* chore: address CR comments

* chore: fix comment issues

* feat: use mpsc::channel

* feat: check is_running flag

* chore: Add stop request to notify a worker

* refactor: add join_dir to join paths

* feat: redefine region requests

* docs: more comments

* refactor: rename worker thread to worker loop

* chore: address CR comments
 2023-07-14 08:06:44 +00:00
JeremyHi
8b4145b634 feat: simplify the usage of channel_manage (#1949) 
* feat: simplify the usage of channelJ_manager by avoiding the external call of start_channel_recycle

* chore: fix unit test

* Update src/common/grpc/Cargo.toml

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>

---------

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>
 2023-07-13 06:11:11 +00:00
Weny Xu
735c6390ca feat: implement alter table procedure (#1878) 
* feat: implement alter table procedure

* fix: fix uncaught error

* refactor: move fetch_table/s to table_routes.rs

* refactor: refactor error handling

* chore: apply suggestions from CR

* feat: switch to using alter table procedure

* feat: add table_version

* chore: apply suggestions from CR

* feat: introduce ddl_channel_manager

* chore: update greptime-proto
 2023-07-13 10:41:46 +08:00
LFC
4fdb6d2f21 refactor: remote catalog uses memory (#1926) 
* refactor: remote catalog uses memory

* rebase develop

* fix: resolve PR comments
 2023-07-12 09:33:33 +00:00
shuiyisong
5422224530 chore: upgrade toml version (#1945) 2023-07-12 14:22:02 +08:00
Ruihang Xia
077785cf1e refactor(mito): define manifest related API (#1942) 
* refactor: port some manifest struct to mito2

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy and nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert lock file and resolve clippy warnings

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-07-12 03:42:55 +00:00
Weny Xu
264c5ea720 feat: meta procedure options (#1937) 
* feat: meta procedure options

* chore: tune meta procedure options in tests

* Update src/common/procedure/Cargo.toml

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-07-12 02:22:08 +00:00
Weny Xu
2d30f4c373 fix: fix broken CI (#1933) 2023-07-11 14:48:41 +08:00
Ning Sun
f293126315 feat: add logical plan based prepare statement for postgresql (#1813) 
* feat: add logical plan based prepare statement for postgresql

* refactor: correct more types

* Update src/servers/src/postgres/types.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* fix: address review issues

* test: add datetime in integration tests

---------

Co-authored-by: LFC <bayinamine@gmail.com>
 2023-07-11 11:07:18 +08:00
Yingwen
65f5349767 feat(mito2): Define basic structs for MitoEngine (#1928) 
* chore: metadata wip

* docs(mito2): Add struct relationships

* feat(mito2): define basic structs

* feat: add version and refactor other metadata

* docs: remove generics param from MitoEngine

* chore: Update src/mito2/Cargo.toml

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

* chore: Apply suggestions from code review

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-07-10 12:25:33 +00:00
Lei, HUANG
207d3d23a1 chore: bump latest greptime-proto version (#1920) 2023-07-10 16:28:22 +08:00
LFC
f19498f73e refactor: unify KvBackend and KvStore (#1890) 
* refactor: unify KvBackend and KvStore
 2023-07-07 19:53:49 +08:00
LFC
03e30652c8 refactor: TableNameKey and DatanodeTableKey (#1868) 
* refactor: TableNameKey and DatanodeTableKey
 2023-07-07 13:27:43 +08:00
Weny Xu
dc085442d7 chore: bump orc-rust to 0.2.4 (#1894) 
chore: bump orc-rust to 0.2
 2023-07-06 08:18:24 +00:00
Weny Xu
28748edb0d chore: bump proto to 917ead6 (#1892) 
* feat: add table_id for ddl exprs

* chore: bump proto to 917ead6
 2023-07-06 13:29:36 +08:00
Weny Xu
ee16262b45 feat: add create table procedure (#1845) 
* feat: add create table procedure

* feat: change table_info type from vec u8 to RawTableInfo

* feat: return create table status

* fix: fix uncaught error

* refactor: use a notifier to respond to callers

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* chore: add comment

* chore: apply suggestions from CR

* refacotr: make CreateMetadata step after DatanodeCreateTable step
 2023-07-04 22:24:43 +08:00
Yingwen
f37b394f1a fix: check table existence in create table procedure (#1880) 
* fix: check table existence in table procedures

* fix: use correct error variant

* chore: address view comments

* chore: address comments

* test: change error code
 2023-07-04 22:01:27 +08:00
Ruihang Xia
bee8323bae chore: bump sqlness to 0.5.0 (#1877) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-07-04 19:49:12 +08:00
Weny Xu
000df8cf1e feat: add ddl client (#1856) 
* feat: add ddl client

* chore: apply suggestions from CR

* chore: apply suggestions from CR
 2023-07-04 19:32:02 +08:00
Yingwen
884731a2c8 chore: initialize mito2 crate (#1875) 2023-07-04 17:55:00 +08:00
LFC
5b42e15105 refactor: add TableInfoKey and TableRegionKey (#1865) 
* refactor: add TableInfoKey and TableRegionKey

* refactor: move KvBackend to common-meta

* fix: resolve PR comments
 2023-07-03 18:01:20 +08:00
Ruihang Xia
25cb667470 fix: sort unstable sqlness result (#1858) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-30 09:25:24 +00:00
Weny Xu
acfa229641 chore: bump orc-rust to 0319acd (#1847) 2023-06-29 10:45:05 +08:00
Lei, HUANG
559d1f73a2 feat: push all possible filters down to parquet exec (#1839) 
* feat: push all possible filters down to parquet exec

* fix: project

* test: add ut for DatafusionArrowPredicate

* fix: according to CR comments
 2023-06-28 20:14:37 +08:00
fys
313121f2ae fix: block when stream insert (#1835) 
* fix: stream insert blocking

* fix: example link

* chore: Increase the default channel size "1024" -> "65536"
 2023-06-27 16:57:03 +08:00
shuiyisong
03057cab6c feat: physical plan wrapper (#1837) 
* test: add physical plan wrapper trait

* test: add plugins to datanode initialization

* test: add plugins to datanode initialization

* chore: add metrics method

* chore: update meter-core version

* chore: remove unused code

* chore: impl metrics method on df execution plan adapter

* chore: minor comment fix

* chore: add retry in create table

* chore: shrink keep lease handler buffer

* chore: add etcd batch size warn

* chore: try shrink

* Revert "chore: try shrink"

This reverts commit 0361b51670.

* chore: add create table backup time

* add metrics in some interfaces

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* calc elapsed time and rows

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: remove timer in scan batch

* chore: add back stream metrics wrapper

* chore: add timer to ready poll

* chore: minor update

* chore: try using df_plan.metrics()

* chore: remove table scan timer

* chore: remove scan timer

* chore: add debug log

* Revert "chore: add debug log"

This reverts commit 672a0138fd.

* chore: use batch size as row count

* chore: use batch size as row count

* chore: tune code for pr

* chore: rename to physical plan wrapper

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-27 14:04:04 +08:00
Weny Xu
223cf31409 feat: support to copy from orc format (#1814) 
* feat: support to copy from orc format

* test: add copy from orc test

* chore: add license header

* refactor: remove unimplemented macro

* chore: apply suggestions from CR

* chore: bump orc-rust to 0.2.3
 2023-06-25 14:07:16 +08:00
LFC
fa522bc579 fix: drop region alive countdown tasks when deregistering table (#1808) 2023-06-21 14:49:32 +08:00
Lei, HUANG
16c1ee2618 feat: incremental database backup (#1240) 
* feat: incremental database backup

* chore: rebase develop

* chore: move backup to StatementExecutor

* feat: copy database parser

* chore: remove some todos

* chore: use timestamp string instead of i64 string

* fix: typo
 2023-06-20 18:26:55 +08:00
JeremyHi
4fdee5ea3c feat: deal with node epoch (#1795) 
* feat: deal with node epoch

* feat: dn send node_epoch

* Update src/meta-srv/src/handler/persist_stats_handler.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update src/meta-srv/src/service/store/ext.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* chore: by cr

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-06-20 07:07:05 +00:00
dennis zhuang
30472cebae feat: prepare supports caching logical plan and infering param types (#1776) 
* feat: change do_describe function signature

* feat: infer param type and cache logical plan for msyql prepared statments

* fix: convert_value

* fix: forgot helper

* chore: comments

* fix: typo

* test: add more tests and test date, datatime in mysql

* chore: fix CR comments

* chore: add location

* chore: by CR comments

* Update tests-integration/tests/sql.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: remove the trace

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-20 04:07:28 +00:00
LFC
2dd86b686f feat: extend region leases in Metasrv (#1784) 
* feat: extend region leases in Metasrv

* fix: resolve PR comments
 2023-06-19 19:55:59 +08:00
LFC
128c6ec98c feat: region alive keeper in Datanode (#1780) 2023-06-19 14:50:33 +08:00
dennis zhuang
09747ea206 feat: use DataFrame to replace SQL for Prometheus remote read (#1774) 
* feat: debug QueryEngineState

* feat: impl read_table to create DataFrame for a table

* fix: clippy warnings

* feat: use DataFrame to handle prometheus remote read quries

* Update src/frontend/src/instance/prometheus.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* chore: CR comments

---------

Co-authored-by: LFC <bayinamine@gmail.com>
 2023-06-14 07:39:28 +00:00
Yingwen
5b8e54e60e feat: Add HTTP API for cpu profiling (#1694) 
* chore: print source error in mem-prof

* feat(common-pprof): add pprof crate

* feat(servers): Add pprof handler to router

refactor the mem_prof handler to avoid checking feature while
registering router

* feat(servers): pprof handler support different output type

* docs(common-pprof): Add readme

* feat(common-pprof): Build guard using code in pprof-rs's example

* feat(common-pprof): use prost

* feat: don't add timeout to perf api

* feat: add feature pprof

* feat: update readme

* test: fix tests

* feat: close region in TestBase

* feat(pprof): addres comments
 2023-06-07 15:25:16 +08:00
Lei, HUANG
8cda1635cc feat: make jemalloc the default allocator (#1733) 
* feat: add jemalloc metrics

* fix: dep format
 2023-06-06 12:11:22 +00:00
Lei, HUANG
f63ddb57c3 fix: parquet time range predicate panic (#1735) 
fix: parquet reader should use store schema to build time range predicate
 2023-06-06 19:11:45 +08:00
Yingwen
1b4976b077 feat: Adds some metrics for write path and flush (#1726) 
* feat: more metrics

* feat: Add preprocess elapsed

* chore(storage): rename metric

* test: fix tests
 2023-06-05 21:35:44 +08:00
Lei, HUANG
166fb8871e chore: bump greptimedb version 0.4.0 (#1724) 2023-06-05 18:41:53 +08:00
JeremyHi
3d7185749d feat: insert with stream (#1703) 
* feat: insert with stream

* chore: by CR
 2023-06-03 03:58:00 +00:00
LFC
5004cf6d9a feat: make grpc insert requests in a batch (#1687) 
* feat: make Prometheus remote write in a batch

* rebase

* fix: resolve PR comments

* fix: resolve PR comments

* fix: resolve PR comments
 2023-06-02 09:06:48 +00:00
Ruihang Xia
8e69aef973 feat: serialize/deserialize support for PromQL plans (#1684) 
* implement serializer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy and CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* register registry

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable promql plan for dist planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-02 16:14:05 +08:00
Ruihang Xia
2615718999 feat: merge scan for distributed execution (#1660) 
* generate exec plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move DatanodeClients to client crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* wip MergeScanExec::to_stream

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix default catalog

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix expand order of new stage

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move sqlness cases contains plan out of common dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* refactor information schema to allow duplicated scan call

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: ignore two cases due to substrait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* reorganise sqlness common cases

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* redact round robin partition number

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

* skip tranforming projection

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* revert common/order

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/dist_plan/merge_scan.rs

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore region failover IT

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result again and again

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* unignore some tests about projection

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* enable failover tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-06-02 06:42:54 +00:00
fys
fe6e3daf81 fix: failed to insert data with u8 (#1701) 
* fix: failed to insert data with u8 field

* remove unused code

* fix cr
 2023-06-02 06:01:59 +00:00
Weny Xu
f811ae4665 fix: enable region failover test (#1699) 
fix: fix region failover test
 2023-06-02 12:05:37 +08:00
Ruihang Xia
e5b6f8654a feat: optimizer rule to pass expected output ordering hint (#1675) 
* move type convertsion rule into optimizer dir

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement order_hint rule

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* it works!

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use column name instead

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* accomplish test case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update lock file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-06-02 03:43:51 +00:00