Yingwen
e5663a075f
feat(mito): preparation to implementing write ( #2085 )
...
* refactor: move request mod
* feat: add mutation
* feat: add handle_write mod
* feat: one mutation at a time
* feat: handle write requests
* feat: validate schema
* refactor: move schema check to write request
* feat: add convert value
* feat: fill default values
* chore: remove comments
* feat: remove code
* feat: remove code
* feat: buf requests
* style: fix clippy
* refactor: rename check functions
* chore: fix compile error
* chore: Revert "feat: remove code"
This reverts commit 65165975405f2b790a01killme2008@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-08-04 02:53:02 +00:00
fys
dda922507f
feat: impl pubsub in metasrv ( #2045 )
...
* feat: impl pubsub
* add test_subscriber_disconnect unit test
* chore: cr
* cr
* cr
2023-08-03 03:56:43 +00:00
Ning Sun
7a27ef8d11
fix: remove openssl from reqwest and use rustls instead ( #2081 )
...
* fix: remove openssl from reqwest and use rustls instead
* fix: correct server url
* style: fix toml format
2023-08-02 10:23:21 +00:00
zyy17
be8f243c64
chore: update Cargo.lock ( #2068 )
2023-08-02 15:23:16 +08:00
JeremyHi
fd7eb87a52
refactor: common semantic-type ( #2057 )
2023-08-01 11:18:05 +00:00
Sunray Ley
090b7e61ca
feat: make the gRPC channel between Frontend and Datanode configurable ( #2044 )
...
* feat: expose frontend datanode_client_options
* chore: add configuration options to the configuration file
* refactor(frontend): extract DatanodeOptions to service_config
* refactor(frontend): extract DatanodeOptions to service_config
* style: remove unnecessary suffix in variable name
Co-authored-by: Yingwen <realevenyag@gmail.com >
* feat: use humantime_serde for readable duration
---------
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-08-01 10:49:41 +00:00
gongzhengyang
0eac56a442
chore: remove unused dependencies ( #2055 )
...
Co-authored-by: gongzhengyang <gongzhengyang@bolean.com.cn >
2023-08-01 07:43:03 +00:00
Ruihang Xia
44f3ed2f74
chore(deps): bump datafusion to the latest commit ( #1967 )
...
* bump deps
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile except pyo3 backend
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix promql-parser metric name matcher
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyarrow convert
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix pyo3 compiling
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix tests
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove deadcode
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update stream adapter display format
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix physical optimizer rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-08-01 02:10:49 +00:00
Ruihang Xia
5bd80a74ab
feat: prepare for implementing considering partition key in the distributed planner ( #2000 )
...
* basic impl
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix frontend logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add sqlness test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* check substrait compatibility before pushdown
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* going to revert some rules
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness result
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test and clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix compile error
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove println
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* Apply suggestions from code review
Co-authored-by: Yingwen <realevenyag@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
Co-authored-by: Yingwen <realevenyag@gmail.com >
2023-07-31 12:36:23 +00:00
localhost
7681864eb4
chore: add version reporter ( #2007 )
...
* chore: add version reporter
* chore: add uuid for version report
* chore: add file license
* chore: format code
* chore: fix by pr comment
* chore: change version report api url
* chore: change greptimedb opentelemetry crate name
* chore: minor code beautification
* chore: add keys only option when range etcd
* chore: fix by pr comment
* chore: fix by pr comment
* chore: change uuid file location
* chore: only run telemetry in meta leader
* chore: add more test and some minor fix
* chore: make clippy happy
* chore: fix by pr comment
* chore: fix by pr comment
* chore: add debug log for greptimedb telemetry
2023-07-31 06:58:00 +00:00
shuiyisong
9c1f0234de
refactor: query context ( #2022 )
...
* chore: refactor query_context
* chore: remove use statement
* chore: make query_context build return arc
* fix: sqlness test
* fix: cr issue
* fix: use unwrap or else
2023-07-25 06:11:34 +00:00
Ruihang Xia
f55bff51ac
feat: set and retrieve trace id in log macro ( #2016 )
...
* trace id passed by task local store
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* modify log macro
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* remove tokio::spawn
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* use real trace id
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-25 03:50:27 +00:00
Yingwen
5f65e3ff44
feat(mito): Port parquet writer and reader to mito2 ( #2018 )
...
* feat(mito): Port Batch and BufferedWriter
* feat: encode metadata to parquet
* feat: define BatchReader trait
* chore: ParquetWriter write_all takes `&mut self`
* feat(mito): port ParquetReader
* chore: fix typo
* chore: address CR comment
2023-07-24 09:35:21 +00:00
shuiyisong
632cb26430
feat: trace_id in query context ( #2014 )
...
* chore: unify once_cell version
* chore: update cargo lock
* chore: add gen_trace_id
* chore: add trace_id to query_ctx
* chore: add debug log
* Revert "chore: add debug log"
This reverts commit f52ab3bb300f1d73117cd6ebbd8e0162829b1aba.
* chore: add frontend node id option
* chore: add query ctx to query engine ctx
* chore: set trace_id to logical_plan api
* chore: add trace_id in grpc entrance
* chore: generate trace_id while creating query_ctx
* chore: fix typo
* chore: extract trace_id from grpc header
* chore: extract trace_id from grpc header
* chore: fix clippy
* chore: add `QueryContextBuilder`
* chore: change node_id in fe to string
2023-07-24 07:35:06 +00:00
LFC
953b8a0132
feat: benchmark table metadata managers ( #2008 )
...
* feat: benchmark table metadata managers
* feat: benchmark table metadata managers
2023-07-21 05:41:06 +00:00
Ning Sun
a7557b70f1
feat: Add more tags for OTLP metrics protocol ( #2003 )
...
* test: add integration tests for otlp
* feat: add resource and scope attributes as tag
2023-07-21 02:02:43 +00:00
Vanish
51fe074666
feat: truncate table execute ( #2002 )
...
* feat: implement truncate execute in standalone mode
* feat: implement truncate execute in distribute mode
* chore: update greptime-proto
* fix: license header
* chore: CR
* chore: update greptime-proto
2023-07-20 11:04:37 +00:00
LFC
172febb1af
refactor!: trying to replace TableGlobalValue, part 2 ( #1985 )
...
* refactor!: using the new table metadata values
* fix: resolve PR comments
* fix: resolve PR comments
* fix: resolve PR comments
2023-07-19 12:01:43 +00:00
Yingwen
bb8468437e
feat(mito): Define Version and metadata builders for mito2 ( #1989 )
...
* feat: define structs for version
* feat: Build region from metadata and memtable builder
* feat: impl validate for metadata
* feat: add more fields to RegionMetadata
* test: more tests
* test: more check and test
* feat: allow overwriting version
* style: fix clippy
2023-07-19 07:50:20 +00:00
Ning Sun
62a41d2280
feat: initial implementation for OpenTelemetry otlp/http ( #1974 )
...
* feat: initial implementation for otlp
* feat: implement more opentelemetry data types
* feat: add metrics
* feat: add support for parsing db name from headers
* feat: allow dbname authentication via header
* chore: disable histogram for now
* refactor: correct error name
* test: add tests for otlp encoders
* Update src/servers/src/error.rs
Co-authored-by: Eugene Tolbakov <ev.tolbakov@gmail.com >
* refactor: address review issues
---------
Co-authored-by: Eugene Tolbakov <ev.tolbakov@gmail.com >
2023-07-19 03:03:52 +00:00
Weny Xu
37dad206f4
fix: fix wait procedure watcher bug ( #1987 )
2023-07-18 09:07:31 +00:00
dennis zhuang
b81570b99a
feat: impl time type ( #1961 )
...
* chore: remove useless Option type in plugins (#1544 )
Co-authored-by: paomian <qtang@greptime.com >
* chore: remove useless Option type in plugins (#1544 )
Co-authored-by: paomian <qtang@greptime.com >
* chore: remove useless Option type in plugins (#1544 )
Co-authored-by: paomian <qtang@greptime.com >
* chore: remove useless Option type in plugins (#1544 )
Co-authored-by: paomian <qtang@greptime.com >
* feat: first commit for time type
* feat: impl time type
* fix: arrow vectors type conversion
* test: add time test
* test: adds more tests for time type
* chore: style
* fix: sqlness result
* Update src/common/time/src/time.rs
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* chore: CR comments
---------
Co-authored-by: localhost <xpaomian@gmail.com >
Co-authored-by: paomian <qtang@greptime.com >
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
2023-07-18 02:55:28 +00:00
LFC
dd62f4c407
feat: tool to migrate table metadata values ( #1971 )
...
feat: tool to migrate table metadata values when upgrading to version 0.4
2023-07-17 15:06:32 +08:00
LFC
7cf6c2bd5c
refactor: trying to replace TableGlobalValue, part 1 ( #1956 )
...
* refactor: trying to replace TableGlobalValue, part 1
* fix: resolve PR comments
2023-07-17 11:32:46 +08:00
Yingwen
ef7c5dd311
feat(mito): Implement WorkerGroup to handle requests ( #1950 )
...
* feat: engine worker framework
* feat: worder comments
* feat: divide worker requests by type
* feat: handlers for worker thread
* refactor: rename requests to ddl and dml requests
* feat: methods to stop and submit requests
* refactor: rename request queue to request buffer
* refactor: remove ddl and dml request
* feat: send request to worker
* test: test stop
* docs(mito): worker group docs
* style: fix clippy
* docs: update WorkerGroup comment
* chore: address CR comments
* chore: fix comment issues
* feat: use mpsc::channel
* feat: check is_running flag
* chore: Add stop request to notify a worker
* refactor: add join_dir to join paths
* feat: redefine region requests
* docs: more comments
* refactor: rename worker thread to worker loop
* chore: address CR comments
2023-07-14 08:06:44 +00:00
JeremyHi
8b4145b634
feat: simplify the usage of channel_manage ( #1949 )
...
* feat: simplify the usage of channelJ_manager by avoiding the external call of start_channel_recycle
* chore: fix unit test
* Update src/common/grpc/Cargo.toml
Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com >
---------
Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com >
2023-07-13 06:11:11 +00:00
Weny Xu
735c6390ca
feat: implement alter table procedure ( #1878 )
...
* feat: implement alter table procedure
* fix: fix uncaught error
* refactor: move fetch_table/s to table_routes.rs
* refactor: refactor error handling
* chore: apply suggestions from CR
* feat: switch to using alter table procedure
* feat: add table_version
* chore: apply suggestions from CR
* feat: introduce ddl_channel_manager
* chore: update greptime-proto
2023-07-13 10:41:46 +08:00
LFC
4fdb6d2f21
refactor: remote catalog uses memory ( #1926 )
...
* refactor: remote catalog uses memory
* rebase develop
* fix: resolve PR comments
2023-07-12 09:33:33 +00:00
shuiyisong
5422224530
chore: upgrade toml version ( #1945 )
2023-07-12 14:22:02 +08:00
Ruihang Xia
077785cf1e
refactor(mito): define manifest related API ( #1942 )
...
* refactor: port some manifest struct to mito2
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix clippy and nextest
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* revert lock file and resolve clippy warnings
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-12 03:42:55 +00:00
Weny Xu
264c5ea720
feat: meta procedure options ( #1937 )
...
* feat: meta procedure options
* chore: tune meta procedure options in tests
* Update src/common/procedure/Cargo.toml
Co-authored-by: dennis zhuang <killme2008@gmail.com >
---------
Co-authored-by: dennis zhuang <killme2008@gmail.com >
2023-07-12 02:22:08 +00:00
Weny Xu
2d30f4c373
fix: fix broken CI ( #1933 )
2023-07-11 14:48:41 +08:00
Ning Sun
f293126315
feat: add logical plan based prepare statement for postgresql ( #1813 )
...
* feat: add logical plan based prepare statement for postgresql
* refactor: correct more types
* Update src/servers/src/postgres/types.rs
Co-authored-by: LFC <bayinamine@gmail.com >
* fix: address review issues
* test: add datetime in integration tests
---------
Co-authored-by: LFC <bayinamine@gmail.com >
2023-07-11 11:07:18 +08:00
Yingwen
65f5349767
feat(mito2): Define basic structs for MitoEngine ( #1928 )
...
* chore: metadata wip
* docs(mito2): Add struct relationships
* feat(mito2): define basic structs
* feat: add version and refactor other metadata
* docs: remove generics param from MitoEngine
* chore: Update src/mito2/Cargo.toml
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
* chore: Apply suggestions from code review
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
---------
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-10 12:25:33 +00:00
Lei, HUANG
207d3d23a1
chore: bump latest greptime-proto version ( #1920 )
2023-07-10 16:28:22 +08:00
LFC
f19498f73e
refactor: unify KvBackend and KvStore ( #1890 )
...
* refactor: unify KvBackend and KvStore
2023-07-07 19:53:49 +08:00
LFC
03e30652c8
refactor: TableNameKey and DatanodeTableKey ( #1868 )
...
* refactor: TableNameKey and DatanodeTableKey
2023-07-07 13:27:43 +08:00
Weny Xu
dc085442d7
chore: bump orc-rust to 0.2.4 ( #1894 )
...
chore: bump orc-rust to 0.2
2023-07-06 08:18:24 +00:00
Weny Xu
28748edb0d
chore: bump proto to 917ead6 ( #1892 )
...
* feat: add table_id for ddl exprs
* chore: bump proto to 917ead6
2023-07-06 13:29:36 +08:00
Weny Xu
ee16262b45
feat: add create table procedure ( #1845 )
...
* feat: add create table procedure
* feat: change table_info type from vec u8 to RawTableInfo
* feat: return create table status
* fix: fix uncaught error
* refactor: use a notifier to respond to callers
* chore: apply suggestions from CR
* chore: apply suggestions from CR
* chore: add comment
* chore: apply suggestions from CR
* refacotr: make CreateMetadata step after DatanodeCreateTable step
2023-07-04 22:24:43 +08:00
Yingwen
f37b394f1a
fix: check table existence in create table procedure ( #1880 )
...
* fix: check table existence in table procedures
* fix: use correct error variant
* chore: address view comments
* chore: address comments
* test: change error code
2023-07-04 22:01:27 +08:00
Ruihang Xia
bee8323bae
chore: bump sqlness to 0.5.0 ( #1877 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-07-04 19:49:12 +08:00
Weny Xu
000df8cf1e
feat: add ddl client ( #1856 )
...
* feat: add ddl client
* chore: apply suggestions from CR
* chore: apply suggestions from CR
2023-07-04 19:32:02 +08:00
Yingwen
884731a2c8
chore: initialize mito2 crate ( #1875 )
2023-07-04 17:55:00 +08:00
LFC
5b42e15105
refactor: add TableInfoKey and TableRegionKey ( #1865 )
...
* refactor: add TableInfoKey and TableRegionKey
* refactor: move KvBackend to common-meta
* fix: resolve PR comments
2023-07-03 18:01:20 +08:00
Ruihang Xia
25cb667470
fix: sort unstable sqlness result ( #1858 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2023-06-30 09:25:24 +00:00
Weny Xu
acfa229641
chore: bump orc-rust to 0319acd ( #1847 )
2023-06-29 10:45:05 +08:00
Lei, HUANG
559d1f73a2
feat: push all possible filters down to parquet exec ( #1839 )
...
* feat: push all possible filters down to parquet exec
* fix: project
* test: add ut for DatafusionArrowPredicate
* fix: according to CR comments
2023-06-28 20:14:37 +08:00
fys
313121f2ae
fix: block when stream insert ( #1835 )
...
* fix: stream insert blocking
* fix: example link
* chore: Increase the default channel size "1024" -> "65536"
2023-06-27 16:57:03 +08:00
shuiyisong
03057cab6c
feat: physical plan wrapper ( #1837 )
...
* test: add physical plan wrapper trait
* test: add plugins to datanode initialization
* test: add plugins to datanode initialization
* chore: add metrics method
* chore: update meter-core version
* chore: remove unused code
* chore: impl metrics method on df execution plan adapter
* chore: minor comment fix
* chore: add retry in create table
* chore: shrink keep lease handler buffer
* chore: add etcd batch size warn
* chore: try shrink
* Revert "chore: try shrink"
This reverts commit 0361b51670waynestxia@gmail.com >
* calc elapsed time and rows
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* chore: remove timer in scan batch
* chore: add back stream metrics wrapper
* chore: add timer to ready poll
* chore: minor update
* chore: try using df_plan.metrics()
* chore: remove table scan timer
* chore: remove scan timer
* chore: add debug log
* Revert "chore: add debug log"
This reverts commit 672a0138fdwaynestxia@gmail.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2023-06-27 14:04:04 +08:00
