rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-31 04:10:38 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,578 Commits 139 Branches 254 Tags
47bf3008694ae8028c65d66f8e506f763a4e798a
Commit Graph

322 Commits

Author SHA1 Message Date
Yingwen
d4ae8a6fed feat(mito): Add writable flag to region (#2349) 
* feat: add writable flag to region.

* refactor: rename MitoEngine to MitoEngine::scanner

* feat: add set_writable() to RegionEngine

* feat: check whether region is writable

* feat: make set_writable sync

* test: test set_writable

* docs: update comments

* feat: send result on compaction failure

* refactor: wrap output sender in new type

* feat: on failure

* refactor: use get_region_or/writable_region_or

* refactor: remove send_result

* feat: notify waiters on flush scheduler drop

* test: fix tests

* fix: only alter writable region
 2023-09-14 02:45:30 +00:00
LFC
d08b05c963 fix: make test-integration able to compile (#2384) 
* fix: make test-integration able to compile

* chore: fmt toml

---------

Co-authored-by: WenyXu <wenymedia@gmail.com>
 2023-09-13 12:42:46 +00:00
dennis zhuang
3f97a0d285 fix: gRPC max mesage size limitation (#2375) 
* fix: gRPC max mesage size limitation

* chore: don't set max_encoding_message_size
 2023-09-13 08:13:49 +00:00
Ruihang Xia
eb7116ab56 feat: read/write works in distributed mode 🎉 (#2327) 
* add do_get method to RegionRequestHandler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move RegionRequestHandler to client crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use RegionRequestHandler in MergeScan

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor fix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* ignore tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
50fca2400e feat: adapt methods from RegionEngine for MitoEngine (#2315) 
* feat: adapt methods from RegionEngine for MitoEngine

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor fixes

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
LFC
ff3881f0e1 feat: drop distributed Mito2 table (#2260) 
* feat: drop distributed Mito2 table

* rebase develop

* fix: resolve PR comments

* fix: resolve PR comments
 2023-09-12 07:57:15 -05:00
Zhenchi
d6c82867d5 refactor: remove the most Table impls (#2274) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
Ruihang Xia
718246ea1a feat: implement heartbeat for region server (#2279) 
* retrieve region stats from region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat handler

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start datanode with region server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove comment

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable non-unit test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* implement heartbeat task

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-09-12 07:57:15 -05:00
JeremyHi
58d07e0e62 feat: v04 rm unused exprs (#2285) 
* feat: rm compact and flush exprs

* refactor: continue to rm compact and flush
 2023-09-12 07:57:15 -05:00
ZonaHe
de966af83b feat: update dashboard to v0.3.3 (#2339) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-09-06 19:19:56 +08:00
ZonaHe
a0d15b489a feat: update dashboard to v0.3.2 (#2295) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-08-31 22:05:00 +08:00
shuiyisong
5df4d44761 feat: schema level opts (#2283) 
* chore: update proto

* chore: add try from for schema name value

* chore: merge schema opts to table opts while creating table

* chore: use table ttl opts first

* chore: add unit test

* chore: update proto version
 2023-08-30 08:11:08 +00:00
LFC
ef75e8f7c3 feat: create distributed Mito2 table (#2246) 
* feat: create distributed Mito2 table

* rebase develop
 2023-08-28 12:07:52 +00:00
Ruihang Xia
6b8cf0bbf0 feat: impl region engine for mito (#2269) 
* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* convert request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* import result convertor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename symbols

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-28 09:24:12 +00:00
JeremyHi
63b22b2403 feat: prometheus row inserter (#2263) 
* feat: prometheus row inserter

* chore: add unit test

* refactor: to row_insert_requests

* chore: typo

* chore: alloc row by TableData

* chore: by review comment
 2023-08-28 03:22:23 +00:00
dennis zhuang
5160838d04 chore: change version to 0.4.0-nightly (#2258) 
* chore: change version to 0.4.0-nightly

* fix: test
 2023-08-25 09:44:39 +00:00
shuiyisong
f16f58266e refactor: query_ctx from http middleware (#2253) 
* chore: change userinfo to query_ctx in http handler

* chore: minor change

* chore: move prometheus http to http mod

* chore: fix uni test:

* chore: add back schema check

* chore: minor change

* chore: remove clone
 2023-08-25 09:36:33 +00:00
Ruihang Xia
8d446ed741 fix: quote ident on rendered SQL (#2248) 
* fix: quote ident on rendered SQL

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* read quote style from query context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-25 07:25:21 +00:00
JeremyHi
7746e5b172 feat: dist row inserter (#2231) 
* feat: fronend row inserter

* feat: row splitter

chore: row splitter's unit test

* feat: RowDistInserter

* feat: make influxdb line protocol using row-based protocol

* Update src/partition/src/row_splitter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by review comment

* Update src/frontend/src/instance/distributed/row_inserter.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* chore: by comment

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-08-24 06:58:05 +00:00
Weny Xu
d81ddd8879 chore: fix clippy (#2232) 2023-08-23 02:24:29 +00:00
Ning Sun
88247e4284 fix!: resolve residual issues with removing prometheus port (#2227) 
* fix: resolve residual issues when removing prometheus port

* fix: remove prometheus from sample config as well
 2023-08-23 01:49:11 +00:00
Ruihang Xia
18250c4803 feat: implement Flight and gRPC services for RegionServer (#2226) 
* extract FlightCraft trait

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split service handler in GrpcServer

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* left grpc server implement

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* start region server if configured

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-22 13:30:09 +00:00
niebayes
e5f4ca2dab feat: streaming do_get (#2171) 
* feat: rewrite do_get for streaming get flight data

* feat: rewrite do_get call stack but leave the async stream adapter not modified yet

* feat: rewrite the async stream adapter to accept greptime record batch stream

* fix: resolve some PR comments

* feat: rewrite tests to adapt to the streaming do_get

* feat: add unit tests for streaming do_get

* feat: rewrite timer metric of merge scan

* remove unhelpful unit tests for streaming do_get

* add a new metric timer for merge scan and fix some test errors

* rewrite mysql writer to write query results in a streaming manner

* fix: fix fmt errors

* fix: rewrite sqlness runner to take into account the streaming do_get

* fix: fix toml format errors

* fix: resolve some PR comments

* fix: resolve some PR comments

* fix: refactor do_get to increase readability

* fix: refactor mysql try_write_one to increase readability
 2023-08-22 02:54:05 +00:00
shuiyisong
c5f507c20e fix: add user_info extension to prom_store handler (#2212) 
chore: add user_info extention to prom_store auth
 2023-08-21 04:55:34 +00:00
JeremyHi
033b650d0d feat: row write protocol (#2189) 
* feat: datanode's row insrter

* refactor: ExprFactory

* feat: row inserter in standalon mode

* chore: minor refactor

* feat: influxdb line protocol's row protocol

* chore: minor refactor

* improve: avoid to use too many string

* no longer async

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* chore: do not check empty data

* chore: by review comment

* chore: by comment

* chore: by review comment

* chore: by review comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-19 13:08:44 +00:00
dennis zhuang
272f649b22 fix: some TODO in sqlness cases and refactor meta-client error (#2207) 
* fix: some TODO in sqlness cases and refactor meta-client error

* fix: delete tests/cases/standalone/alter/drop_col_not_null_next.output
 2023-08-18 10:09:11 +00:00
ZonaHe
b8595e1960 feat: update dashboard to v0.3.1 (#2192) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-08-18 09:42:18 +00:00
shuiyisong
61e6656fea fix: auth in prometheus gateway service (#2206) 
* fix: auth in prometheus gateway service

* chore: remove unused code

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-18 09:41:38 +00:00
JeremyHi
6cd7319d67 refactor: grpc insert (#2188) 
* feat: interval type for row protocol

* feat: minor refactor grpc insert

* Update src/common/grpc-expr/src/util.rs

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

* fix: by comment

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-16 11:25:25 +00:00
sh2
202540823f refactor!: move prometheus routes to default http server (#2005) 
* move prometheus routes to default http server

Signed-off-by: sh2 <shawnhxh@outlook.com>

* fix ci test and remove the server logic of prometheus

* remove unused import and prometheus relevant code

* fix ci: rustfmt and test

* fix ci: silly fmt

* fix ci: silly silly fmt

* change `/prom_store` back to `/prometheus`

* remove unused variable

---------

Signed-off-by: sh2 <shawnhxh@outlook.com>
 2023-08-16 03:21:14 +00:00
shuiyisong
c8cde704cf chore: minor auth crate change (#2176) 
* chore: pub auth_mysql

* chore: pub all error

* chore: remove back to error

* chore: wrap failed permission check result to err

* chore: minor change
 2023-08-15 10:49:22 +00:00
JeremyHi
24dc827ff9 feat: grpc handler result (#2107) 
* feat: grpc handler inner result

* feat: ext header, x-greptime-err-code, x-greptime-err-msg

* fix: sqlness case

* chore: by comment

* fix: convert status to Error
 2023-08-15 09:34:00 +00:00
Niwaka
a8f2e4468d feat: handle multiple grpc deletes (#2150) 
* feat: handle multiple grpc deletes

* fix: make DistDeleter::grpc_delete return usize

* fix: remove backtrace from MissingTimeIndexColumn

* fix: avoid using unwrap in PartitionRuleManager::split_delete_request

* fix: simplify MissingTimeIndexColumn
 2023-08-15 08:22:46 +00:00
Ruihang Xia
2168970814 feat: define region server and related requests (#2160) 
* define region server and related requests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fill request body

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change mito2's request type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* chore: bump greptime-proto to d9167cab (row insert/delete)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix test compile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove name_to_index

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* address cr comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* finilise

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-15 06:27:27 +00:00
Lei, HUANG
e924b44e83 refactor: KeyValues return ValueRef (#2170) 
* refactor: KeyValues return ValueRef

* 1. Change KeyValues returned value from pb value to ValueRef
2. Replace OpType/SemanticType with pb's OpType and SemanticType to avoid duplicated conversions.

* feat: define min value of OpType as a const

* fix: toml format
 2023-08-14 14:51:13 +00:00
Ning Sun
f3157df190 fix: normalize otlp string keys (#2168) 2023-08-14 09:39:54 +00:00
LFC
606b489d53 feat: redact secrets in sql when logging (#2141) 2023-08-14 06:40:00 +00:00
shuiyisong
7f51141ed0 refactor: auth crate (#2148) 
* chore: move user_info to auth crate

* chore: temp commit before resolving tests compile error

* chore: fix compile issue

* chore: minor fix

* chore: tmp save

* chore: change user_info to trait

* chore: minor change & use auth result user info in pg session setup

* chore: add as_any to user_info

* chore: rename user_info

* chore: remove ice file

* chore: add permission checker

* chore: add grpc permission check

* chore: add session spawn user_info to query_ctx

* chore: minor update

* chore: add permission checker to sql handler & temp save

* chore: add permission checker to prometheus handler

* chore: add permission checker to opentsdb handler

* chore: add permission checker to other handlers

* chore: add test

* chore: add user_info setting on http entrance

* chore: fix toml

* chore: remove box in permission req

* chore: cr issue

* chore: cr issue
 2023-08-14 02:51:26 +00:00
Ruihang Xia
4fd1057764 fix: several clippy error/warnings after upgrading toolchain (#2156) 
* fix pyscripts mod

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy::needless-pass-by-ref-mut

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add pyo3 feature gate in Makefile

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-11 20:20:21 +08:00
Zou Wei
6877d082f6 feat: compatible with postgres interval type (#2146) 
* feat: impl ToSql/FromSql/ToSqlText for PgInterval.

* chore: remove useless code.

* feat: compatible with postgres interval type.

* chore: cr comment.
 2023-08-11 20:19:57 +08:00
Ruihang Xia
b9bac2b195 fix: let information_schema know itself (#2149) 
* rename show create table

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* register information_schema on registering catalog

* fix tests in standalone

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix frontend catalog manager

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy & typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tweak sqlness test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename constructor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* rename method

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo (again)

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove redundent clones

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-08-11 15:37:27 +08:00
LFC
5c19913a91 build: on windows (#2054) 
* build on windows

* rebase develop

* fix: resolve PR comments
 2023-08-10 08:08:37 +00:00
Ning Sun
0270708d6d fix: correct grpc metric labels (#2136) 2023-08-10 03:59:41 +00:00
Ning Sun
d18eb18b32 feat: use server inferenced types on statement describe (#2032) 
* feat: use server inferenced types on statement describe

* feat: add support for server inferenced type

* feat: allow parameter type inferencing

* chore: update comments

* fix: lint issue

* style: comfort rustfmt

* Update src/servers/src/postgres/types.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-08-09 02:57:56 +00:00
gongzhengyang
ce0f909cac perf: change current schema and catalog to borrow, clone only necessary (#2116) 
perf: change current schema and catalog to borrow, clone only when necessary

Co-authored-by: gongzhengyang <gongzhengyang@bolean.com.cn>
 2023-08-08 12:48:24 +00:00
Ruihang Xia
4c693799d8 fix: bugs related to merge scan (#2118) 
* fix: prevent optimize merge scan, mark distinct as unsupported

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix some other problems

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix unit tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove deadcode

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add some comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update src/query/src/optimizer/type_conversion.rs

Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Lei, HUANG <6406592+v0y4g3r@users.noreply.github.com>
 2023-08-08 11:42:57 +00:00
Ning Sun
c39de9072f refactor: use workspace dependencies for internal modules (#2119) 
* refactor: use workspace dependencies for internal modules

* fix: resolve issue with mock module in datanode

* refactor: update test modules
 2023-08-08 11:02:34 +00:00
LFC
46fa3eb629 chore: upgrade rust toolchain to latest nightly (#2049) 
* chore: upgrade rust toolchain to latest nightly

* rebase develop

* update rust toolchain in ci
 2023-08-08 07:17:51 +00:00
shuiyisong
7efff2d704 fix: introduce taplo.toml and sort Cargo.toml (#2096) 
* fix: add taplo.toml

* fix: introduce taplo.toml & sort cargo.toml

* chore: remove option in ci too
 2023-08-04 06:44:45 +00:00
shuiyisong
6f1094db0a fix: arc() usage in non-test code (#2091) 
* chore: try fix arc issue

* chore: move `parse_catalog_and_schema_from_client_database_name` to catalog crate

* fix: arc issue

* fix: arc issue

* fix: arc issue

* fix: arc issue

* fix: minor change
 2023-08-03 10:16:02 +00:00