rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-20 15:00:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,538 Commits 133 Branches 253 Tags
711e27d9faa2e5925a5855fbd139bb00565db56a
Commit Graph

352 Commits

Author SHA1 Message Date
LFC
711e27d9fa feat: distributed alter table in region server (#2311) 
* feat: distributed alter table in region server

* rebase
 2023-09-12 07:57:15 -05:00
Weny Xu
922e342b63 refactor: refactor ddl manager (#2306) 
* refactor: refactor ddl manager

* chore: apply suggestions from CR
 2023-09-12 07:57:15 -05:00
Zhenchi
7dde9ce3ce feat(frontend): migrate insert to region server (#2318) 
* feat(frontend): migrate insert to region server

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: move converter to Inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: rename convert function

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: add span id

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: compilation

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* retrigger action

* retrigger action

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
dennis zhuang
a3d5931fca feat: unify all protocol options (#2316) 
* feat: unify all protocol options

* feat: adds enable to example configs

* chore: style

Co-authored-by: JeremyHi <jiachun_feng@proton.me>

---------

Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-09-12 07:57:15 -05:00
Niwaka
3504d8254e fix: unused table options (#2267) 
* fix: unused table options keys

* refactor: simplify validate table options

* chore: Add newlines

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-09-12 07:57:15 -05:00
Weny Xu
4818887e38 refactor: refactor DistInstance (#2305) 2023-09-12 07:57:15 -05:00
Zhenchi
b42d343ae6 feat(frontend): unify column inserter and row inserter (#2293) 
* refactor: unify inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(frontend): unify column inserter and row inserter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove redundant clone

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: move empty check ahead

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add more logs

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: leading license

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* adjust indent

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
718246ea1a feat: implement heartbeat for region server (#2279) 
* retrieve region stats from region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable non-unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
JeremyHi
58d07e0e62 feat: v04 rm unused exprs (#2285) 
* feat: rm compact and flush exprs

* refactor: continue to rm compact and flush
 2023-09-12 07:57:15 -05:00
shuiyisong
baa372520d fix: json compatibility to null (#2287) 
* fix: existing null value for schema name value

* chore: fix null check

* fix: change catalognamevalue and schemanamevalue to option

* fix: fix null case
 2023-08-31 14:21:10 +08:00
shuiyisong
5df4d44761 feat: schema level opts (#2283) 
* chore: update proto

* chore: add try from for schema name value

* chore: merge schema opts to table opts while creating table

* chore: use table ttl opts first

* chore: add unit test

* chore: update proto version
 2023-08-30 08:11:08 +00:00
Weny Xu
22dea02485 fix: use RegionId region number instead (#2273) 2023-08-29 02:52:24 +00:00
LFC
ef75e8f7c3 feat: create distributed Mito2 table (#2246) 
* feat: create distributed Mito2 table

* rebase develop
 2023-08-28 12:07:52 +00:00
JeremyHi
63b22b2403 feat: prometheus row inserter (#2263) 
* feat: prometheus row inserter

* chore: add unit test

* refactor: to row_insert_requests

* chore: typo

* chore: alloc row by TableData

* chore: by review comment
 2023-08-28 03:22:23 +00:00
shuiyisong
f16f58266e refactor: query_ctx from http middleware (#2253) 
* chore: change userinfo to query_ctx in http handler

* chore: minor change

* chore: move prometheus http to http mod

* chore: fix uni test:

* chore: add back schema check

* chore: minor change

* chore: remove clone
 2023-08-25 09:36:33 +00:00
Ruihang Xia
8d446ed741 fix: quote ident on rendered SQL (#2248) 
* fix: quote ident on rendered SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* read quote style from query context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 07:25:21 +00:00
Zhenchi
9d87c8b6de refactor(table): cleanup dist table (#2255) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-25 06:37:39 +00:00
Zhenchi
9ce73e7ca1 refactor(frontend): TableScan instead of scan_to_stream for COPY TO (#2244) 
* refactor(frontend): TableScan instead of `scan_to_stream` for `COPY TO`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-24 12:46:54 +00:00
JeremyHi
7746e5b172 feat: dist row inserter (#2231) 
* feat: fronend row inserter

* feat: row splitter

chore: row splitter's unit test

* feat: RowDistInserter

* feat: make influxdb line protocol using row-based protocol

* Update src/partition/src/row_splitter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by review comment

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* chore: by comment

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-08-24 06:58:05 +00:00
Weny Xu
a7e0e2330e fix: invalidate cache after altering (#2239) 2023-08-24 03:56:17 +00:00
Zhenchi
af95e46512 refactor(table): eliminate calls to DistTable.delete (#2225) 
* refactor(table): eliminate calls to DistTable.delete

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: clippy

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-23 02:33:48 +00:00
Ning Sun
88247e4284 fix!: resolve residual issues with removing prometheus port (#2227) 
* fix: resolve residual issues when removing prometheus port

* fix: remove prometheus from sample config as well
 2023-08-23 01:49:11 +00:00
Ruihang Xia
18250c4803 feat: implement Flight and gRPC services for RegionServer (#2226) 
* extract FlightCraft trait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split service handler in GrpcServer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* left grpc server implement

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start region server if configured

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-22 13:30:09 +00:00
Zhenchi
cb3561f3b3 refactor(table): eliminate calls to DistTable.insert (#2219) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-22 06:15:02 +00:00
Niwaka
b3b43fe1c3 fix: table options can't be found in distributed mode (#2209) 
* fix: table options can't be found in distributed mode

* refactor: use iterator for regions_numbers

* chore: remove TODO
 2023-08-22 03:53:56 +00:00
niebayes
e5f4ca2dab feat: streaming do_get (#2171) 
* feat: rewrite do_get for streaming get flight data

* feat: rewrite do_get call stack but leave the async stream adapter not modified yet

* feat: rewrite the async stream adapter to accept greptime record batch stream

* fix: resolve some PR comments

* feat: rewrite tests to adapt to the streaming do_get

* feat: add unit tests for streaming do_get

* feat: rewrite timer metric of merge scan

* remove unhelpful unit tests for streaming do_get

* add a new metric timer for merge scan and fix some test errors

* rewrite mysql writer to write query results in a streaming manner

* fix: fix fmt errors

* fix: rewrite sqlness runner to take into account the streaming do_get

* fix: fix toml format errors

* fix: resolve some PR comments

* fix: resolve some PR comments

* fix: refactor do_get to increase readability

* fix: refactor mysql try_write_one to increase readability
 2023-08-22 02:54:05 +00:00
JeremyHi
033b650d0d feat: row write protocol (#2189) 
* feat: datanode's row insrter

* refactor: ExprFactory

* feat: row inserter in standalon mode

* chore: minor refactor

* feat: influxdb line protocol's row protocol

* chore: minor refactor

* improve: avoid to use too many string

* no longer async

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: do not check empty data

* chore: by review comment

* chore: by comment

* chore: by review comment

* chore: by review comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-19 13:08:44 +00:00
dennis zhuang
272f649b22 fix: some TODO in sqlness cases and refactor meta-client error (#2207) 
* fix: some TODO in sqlness cases and refactor meta-client error

* fix: delete tests/cases/standalone/alter/drop_col_not_null_next.output
 2023-08-18 10:09:11 +00:00
Zhenchi
8d6a2d0b59 refactor: apply numbers to ThinTable (#2202) 
* refactor: apply numbers to `ThinTable`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: tiny polish

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: unused import

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-18 03:11:37 +00:00
Weny Xu
177036475a fix: support to copy from parquet with typecast (#2201) 2023-08-18 03:09:54 +00:00
Zhenchi
87a730658a refactor: add ThinTable to proxy tables from infoschema (#2193) 
* refactor: add thin table to proxy tables in info_schema

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix(catalog): fix typo in DataSourceAdapter struct name

* fix: remove redundant Send + Sync

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor(catalog): rename DataSourceAdapter to InformationTableDataSource

* feat(catalog): add ThinTableAdapter for adapting ThinTable to Table interface

* rebase develop

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: default impl for table_type of InformationTable

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: filter_pushdown as table field

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: remove explicit type declaration

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-08-17 15:19:14 +00:00
Ruihang Xia
4aaf6aa51b feat: implement query API for RegionServer (#2197) 
* some initial change

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl dummy structs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* decode and send logical plan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement table scan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add some comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-17 11:02:31 +00:00
Weny Xu
6e6ff5a606 refactor: update table metadata in single txn (#2172) 
* refactor: table-metadata-manager

* feat: remove comparing when deleting metadata

* fix: fix comment typos

* chore: apply suggestions from CR

* test: add tests for updating DatanodeTable

* fix: fix clippy

* chore: apply suggestions from CR

* refactor: improve update table route tests

* refactor: return Txn instead of TxnRequest

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: update table metadata in single txn

* feat: check table exists before drop table executing

* test: add tests for table metadata manager

* refactor: remove table region manager

* chore: apply suggestions from CR

* feat: add bench program

* chore: apply suggestions from CR
 2023-08-17 06:29:19 +00:00
JeremyHi
6cd7319d67 refactor: grpc insert (#2188) 
* feat: interval type for row protocol

* feat: minor refactor grpc insert

* Update src/common/grpc-expr/src/util.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: by comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-16 11:25:25 +00:00
sh2
202540823f refactor!: move prometheus routes to default http server (#2005) 
* move prometheus routes to default http server

Signed-off-by: sh2 <shawnhxh@outlook.com>

* fix ci test and remove the server logic of prometheus

* remove unused import and prometheus relevant code

* fix ci: rustfmt and test

* fix ci: silly fmt

* fix ci: silly silly fmt

* change `/prom_store` back to `/prometheus`

* remove unused variable

---------

Signed-off-by: sh2 <shawnhxh@outlook.com>
 2023-08-16 03:21:14 +00:00
shuiyisong
c8cde704cf chore: minor auth crate change (#2176) 
* chore: pub auth_mysql

* chore: pub all error

* chore: remove back to error

* chore: wrap failed permission check result to err

* chore: minor change
 2023-08-15 10:49:22 +00:00
Niwaka
a8f2e4468d feat: handle multiple grpc deletes (#2150) 
* feat: handle multiple grpc deletes

* fix: make DistDeleter::grpc_delete return usize

* fix: remove backtrace from MissingTimeIndexColumn

* fix: avoid using unwrap in PartitionRuleManager::split_delete_request

* fix: simplify MissingTimeIndexColumn
 2023-08-15 08:22:46 +00:00
Ruihang Xia
2168970814 feat: define region server and related requests (#2160) 
* define region server and related requests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fill request body

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change mito2's request type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: bump greptime-proto to d9167cab (row insert/delete)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove name_to_index

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* address cr comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finilise

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-15 06:27:27 +00:00
Weny Xu
69a2036cee feat!: add deserializer for Partition (#2169) 
* feat!: add deserializer for Partition

* fix: fix tests
 2023-08-15 03:36:58 +00:00
Lei, HUANG
e924b44e83 refactor: KeyValues return ValueRef (#2170) 
* refactor: KeyValues return ValueRef

* 1. Change KeyValues returned value from pb value to ValueRef
2. Replace OpType/SemanticType with pb's OpType and SemanticType to avoid duplicated conversions.

* feat: define min value of OpType as a const

* fix: toml format
 2023-08-14 14:51:13 +00:00
LFC
606b489d53 feat: redact secrets in sql when logging (#2141) 2023-08-14 06:40:00 +00:00
Weny Xu
d0b3607633 feat: add table route manager and upgrade tool (#2145) 
* feat: add table route manager and upgrade tool

* test: add table route manager tests

* feat: add new TableRouteValue struct

* chore: apply suggestions from CR

* refactor: change HashMap to BTreeMap

* feat: add version to TableRouteValue
 2023-08-14 04:19:44 +00:00
Weny Xu
5b012a1f67 feat!: switch to new catalog/schema key (#2140) 
* feat!: switch to new catalog/schema key

* chore: apply suggestions from CR
 2023-08-14 03:08:43 +00:00
shuiyisong
7f51141ed0 refactor: auth crate (#2148) 
* chore: move user_info to auth crate

* chore: temp commit before resolving tests compile error

* chore: fix compile issue

* chore: minor fix

* chore: tmp save

* chore: change user_info to trait

* chore: minor change & use auth result user info in pg session setup

* chore: add as_any to user_info

* chore: rename user_info

* chore: remove ice file

* chore: add permission checker

* chore: add grpc permission check

* chore: add session spawn user_info to query_ctx

* chore: minor update

* chore: add permission checker to sql handler & temp save

* chore: add permission checker to prometheus handler

* chore: add permission checker to opentsdb handler

* chore: add permission checker to other handlers

* chore: add test

* chore: add user_info setting on http entrance

* chore: fix toml

* chore: remove box in permission req

* chore: cr issue

* chore: cr issue
 2023-08-14 02:51:26 +00:00
Ruihang Xia
b9bac2b195 fix: let information_schema know itself (#2149) 
* rename show create table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* register information_schema on registering catalog

* fix tests in standalone

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix frontend catalog manager

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy & typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tweak sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename constructor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename method

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo (again)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove redundent clones

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-11 15:37:27 +08:00
WU Jingdi
b7319fe2b1 feat: Support RangeSelect LogicalPlan rewrite (#2058) 
* feat: Support RangeSelect LogicalPlan rewrite

* chore: fix code advice

* fix: change format of range_fn

* chore: optimize project plan rewrite

* chore: fix code advice
 2023-08-10 02:53:20 +00:00
LFC
ea3708b33d fix: deserialize TableInfoValue with missing field (#2134) 2023-08-10 02:43:24 +00:00
Zhenchi
7abe71f399 fix(table): return correct table types (#2131) 
* fix(table): return correct table types

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix: NumbersTable to be Temporary table

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(test): fix affected cases

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(test): fix affected cases

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix: fmt

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>

* fix(tests): fix instance_test expected result

* retrigger action

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: zhongzc <zhongzc@zhongzcs-MacBook-Pro.local>
 2023-08-09 11:07:00 +00:00
gongzhengyang
ce0f909cac perf: change current schema and catalog to borrow, clone only necessary (#2116) 
perf: change current schema and catalog to borrow, clone only when necessary

Co-authored-by: gongzhengyang <gongzhengyang@bolean.com.cn>
 2023-08-08 12:48:24 +00:00
Ning Sun
c39de9072f refactor: use workspace dependencies for internal modules (#2119) 
* refactor: use workspace dependencies for internal modules

* fix: resolve issue with mock module in datanode

* refactor: update test modules
 2023-08-08 11:02:34 +00:00