rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2025-12-27 00:19:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,979 Commits 103 Branches 223 Tags
chore/manual_pre_release_docs
Commit Graph

711 Commits

Author SHA1 Message Date
Yingwen
eb50cee601 feat: Switch to the procedure framework (#1448) 
* feat: Remove create_mock_sql_handler()

create_to_request() and alter_to_request() don't need `&self`, so
we don't need to mock the sql handler to test them

* feat: Enable procedure manager by default

* docs: Update config example

* test: Enable procedure framework in all tests

* refactor(datanode): rename methods using procedure

* test(catalog): Fix temp dir drops before test finishes

* tests: Enable procedure framework in sqlness

* test: Fix sqlness standalone rename test

* fix: Drop procedure allows table not in engine

* test: Change rename table test

* fix: add options to table meta when creating table by procedure

* test: adjust error message in schema test case

* test: Fix test_sql_api error message
 2023-04-25 12:04:02 +08:00
Ruihang Xia
f9ea6b63bf feat: impl instant query and add tests (#1452) 
* feat: impl instant query and add tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-25 11:08:14 +08:00
Ruihang Xia
2cf828da3c feat: implement Prometheus-compatible API in gRPC (#1449) 
* update greptime-proto

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove duplicate delete enum

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl handler and service

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-24 18:03:48 +08:00
Ruihang Xia
b6647af2e3 test: add integration case to check dashboard path (#1422) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-20 11:17:01 +08:00
Ruihang Xia
e8bb00f0be feat: impl instant query interface (#1410) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-04-18 23:25:14 +08:00
Weny Xu
ef134479ef feat: support multi table engines in distributed mode (#1316) 
* chore: bump greptime-proto to 59afacd

* feat: support multi table engines in distributed mode
 2023-04-04 10:27:08 +08:00
Weny Xu
451f9d2d4e feat: support multi table engines (#1277) 
* feat: support multi table engines

* refactor: adapt SqlHandler to support multiple table engines

* refactor: refactor TableEngineManager

* chore: apply review suggestions

* chore: apply review suggestions

* chore: apply review suggestions

* chore: snafu context styling
 2023-04-03 14:49:12 +00:00
LFC
48c2841e4d feat: execute python script in distributed mode (#1264) 
* feat: execute python script in distributed mode

* fix: rebase develop
 2023-04-02 20:36:48 +08:00
localhost
a2b262ebc0 chore: add http metrics server in datanode node when greptime start in distributed mode (#1256) 
* chore: add http metrics server in datanode node when greptime start in distributed mode

* chore: add some docs and license

* chore: change metrics_addr to resolve address already in use error

* chore add metrics for meta service

* chore: replace metrics exporter http server from hyper to axum

* chore: format

* fix: datanode mode branching error

* fix: sqlness test address already in use and start metrics in defualt config

* chore: change metrics location

* chore: use builder pattern to builder httpserver

* chore: remove useless debug_assert macro in httpserver builder

* chore: resolve conflicting build error

* chore: format code
 2023-03-31 18:37:52 +08:00
dennis zhuang
563adbabe9 feat!: improve region manifest service (#1268) 
* feat: try to use batch delete in ManifestLogStorage

* feat: clean temp dir when startup with file backend

* refactor: export region manifest checkpoint actions magin and refactor storage options

* feat: purge unused manifest and checkpoint files by repeat gc task

* chore: debug deleted logs

* feat: adds RepeatedTask and refactor all gc tasks

* chore: clean code

* feat: export gc_duration to manifest config

* test: assert gc works

* fix: typo

* Update src/common/runtime/src/error.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* Update src/common/runtime/src/repeated_task.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* Update src/common/runtime/src/repeated_task.rs

Co-authored-by: LFC <bayinamine@gmail.com>

* fix: format

* Update src/common/runtime/src/repeated_task.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: by CR comments

* chore: by CR comments

* fix: serde default for StorageConfig

* chore: remove compaction config in StandaloneOptions

---------

Co-authored-by: LFC <bayinamine@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-03-31 10:42:00 +08:00
Lei, HUANG
5edd2a3dbe feat: upgrade opendal (#1245) 
* chore: upgrade opendal

* chore: finish upgrading opendal

* fix: clippy complaints

* fix some tests

* fix: all unit tests

* chore: rebase develop

* fix: sqlness tests

* optimize imports

* chore: rebase develop

* doc: add todo
 2023-03-28 09:47:33 +08:00
Ning Sun
e63b28bff1 feat: add dbname and health check for grpc api (#1220) 
* feat: add dbname and health check for grpc api

* refactor: move health check to dedicated service

* chore: switch to merged proto rev

* feat: implement healthcheck on server-side
 2023-03-28 09:46:30 +08:00
LFC
b2a09c888a feat: phi accrual failure detector (#1200) 2023-03-21 11:47:47 +08:00
LFC
f6669a8201 feat: add GRPC unary call service to our GreptimeDB (#1196) 
* feat: add GRPC unary call service to our GreptimeDB
 2023-03-20 14:27:32 +08:00
Lei, HUANG
17eb99bc52 feat: allow manual table flush through HTTP API (#1184) 2023-03-15 20:15:34 +08:00
LFC
604c20a83d refactor: remove the SQL execution interfaces in Datanode (#1135) 
* refactor: remove the SQL execution interfaces in Datanode
 2023-03-13 18:45:30 +08:00
Eugene Tolbakov
b31a6cb506 refactor: replace tempdir with tempfile (#1123) 
* refactor: replace tempdir with tempfile

* refactor(query): move tempfile dependency under the workspace's Cargo.toml

* refactor(tempfile): create common-test-util

* refactor(tempfile): fix toml format

* refactor(tempfile): remove tempfile out of dependencies

* refactor(tempfile): fix incorrect toml
 2023-03-08 11:15:56 +08:00
discord9
379f581780 test: add Integrated Test for Coprocessor& fix minor bugs (#1122) 
* feat: cache `Runtime`

* fix: coprstream schema not set

* test: integrated tests for Coprocessor

* fix: UDF fixed

* style: remove unused import

* chore: remove more unused import

* feat: `filter`, (r)floordiv for Vector

* chore: CR advices

* feat: auto convert to `lit`

* chore: fix typo

* feat: from&to `pyarrow.array`

* feat: allow `pyarrow.array` as args to builtins

* chore: cargo fmt

* test: CI add `pyarrow`

* test: install Python&PyArrow in CI

* test: not cache depend for now

* chore: CR advices

* test: fix name

* style: rename
 2023-03-06 19:20:59 +08:00
yuanbohan
d4e0dc3685 feat: specify prom server start addr (#1111) 
* feat: specify promql server start addr

* refactor: rename promql to prom in Prometheus API server scenario
 2023-03-06 11:07:21 +08:00
LFC
11d45e2918 refactor: upgrade DataFusion, Arrow and Sqlparser (#1074) 
* refactor: upgrade DataFusion, Arrow and Sqlparser

* fix: resolve PR comments
 2023-02-27 22:20:08 +08:00
Ning Sun
81eab74b90 refactor: remove grpc client constructor with default catalog/schema (#1060) 
* refactor: remove grpc client with default catalog/schema

* refactor: re-export consts in client module
 2023-02-24 11:06:14 +08:00
Xuanwo
fdd17c6eeb refactor: Clean up re-export of opendal services (#1067) 
Signed-off-by: Xuanwo <github@xuanwo.io>
 2023-02-23 14:12:34 +08:00
Xuanwo
98ef74bff4 chore: Bump OpenDAL to v0.27 (#1057) 
* Bump OpenDAL to v0.27

Signed-off-by: Xuanwo <github@xuanwo.io>

* Make cargo check happy

Signed-off-by: Xuanwo <github@xuanwo.io>

* Address comments

Signed-off-by: Xuanwo <github@xuanwo.io>

* Address comments

Signed-off-by: Xuanwo <github@xuanwo.io>

* Format toml

Signed-off-by: Xuanwo <github@xuanwo.io>

* Make taplo happy

Signed-off-by: Xuanwo <github@xuanwo.io>

---------

Signed-off-by: Xuanwo <github@xuanwo.io>
 2023-02-23 11:20:45 +08:00
shuiyisong
fb2e0c7cf3 feat: add auth to grpc handler (#1051) 
* chore: get header in grpc & temp save

* chore: change authscheme to include data str

* chore: add auth to grpc flight handler

* chore: add unit test & hold for now since grpc api doesnt accept req input

* chore: minor change

* chore: minor change

* chore: add flight context to database interface

* chore: add test

* chore: update proto version & fix cr issue

* chore: add test

* chore: minor update
 2023-02-22 15:20:10 +08:00
Lei, HUANG
e17d5a1c41 feat: support table options (#1044) 
* feat: change table options from string map to a struct, add ttl and write_buffer_size

* fix: also pass table options to table meta

* feat: pass table options when opening/creating regions

* fix: CR comments
 2023-02-21 08:10:23 +00:00
Ruihang Xia
68b231987c feat: improve Prometheus compliance (#1022) 
* initial impl

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* minor (useless) refactor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* retrieve metric name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add time index column to group by columns
filter out NaN in normalize
remove NULL in instant manipulator
accept form data as HTTP params
correct API URL
accept second literal as step param

* happy clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-20 07:29:43 +00:00
Yingwen
ddbc97befb refactor: changes CreateTableRequest::schema to RawSchema (#1018) 
* refactor: changes CreateTableRequest::schema to RawSchema

* refactor(grpc-expr): create_table_schema returns RawSchema
 2023-02-16 16:04:17 +08:00
Ruihang Xia
dfe7bfb07f feat: handle PromQL HTTP API parameters (#985) 
* feat: impl EvalStmt parser

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile errors

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add integration test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* impl From<PromqlQuery> for PromQuery

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move format into with_context

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update test result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* shorthand compound error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use rfc3339 error to report float parsing error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove CompoundError

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-15 17:15:44 +08:00
shuiyisong
301656d568 fix: rename schema to db in http param (#1008) 
chore: rename schema to db in http script handler
 2023-02-15 15:59:00 +08:00
elijah
926022e14c feat: enable caching when using object store (#928) 
* feat: enable caching when using object store

* feat: support file cache for object store

* feat: maintaining the cached files with lru

* fix: improve the code

* empty commit

* improve the code
 2023-02-07 15:46:37 +08:00
ShenJunkun
afac885c10 refactor: add schema column to the scripts table (#868) 2023-02-07 11:07:32 +08:00
Ruihang Xia
fc9276c79d feat: export promql service in server (#924) 
* chore: some tiny typo/style fix

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* feat: add promql server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* works for mocked query

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* integration test case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* expose promql api to our http server

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* resolve CR comments

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* adjust router structure

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-02-03 08:28:56 +00:00
LFC
af935671b2 feat: support "use" in GRPC requests (#922) 
* feat: support "use catalog and schema"(behave like the "use" in MySQL) in GRPC requests

* fix: rebase develop
 2023-02-02 20:02:56 +08:00
Yingwen
b2ad0e972b feat: Define procedure related traits (#904) 
* chore: Move uuid to workspace.dependencies

* feat: Define procedure related traits

* test: Add tests

* chore: Update imports

* feat: Submit ProcedureWithId to manager

* chore: pub ProcedureId::parse_str

* refactor: ProcedureId::parse_str returns Result

* chore: Address CR comments

Also implements FromStr for ProcedureId
 2023-01-31 14:17:28 +08:00
Yun Chen
a7dc86ffe5 feat: oss storage support (#911) 
* feat: add oss storage support

* fix: ci build format check

* fix: align OSS to Oss

* fix: cr comments

* fix: rename OSS to Oss in integration tests

* fix: clippy fix
 2023-01-29 20:09:38 +08:00
shuiyisong
6960739b3d feat: add authorize to UserProvider trait (#879) 
* feat: add SchemaValidator

* feat: add schema validator to mysql shim

* chore: pass schema validator to http auth layer

* feat: add schema validator to http

* feat: add schema validator to pg

* feat: add schema validator to pg

* feat: add schema validator test

* chore: remove println in test

* chore: use !matches

* refactor: refac authenticate and authorize in http auth

* refactor: refac authenticate and authorize in http auth

* chore: typo

* chore: minor change

* refactor: merge schema_validator into user_providier

* chore: fix license issue

* refactor: change http query param from database to db

* chore: fix cr issue
 2023-01-18 12:42:08 +08:00
LFC
2664436194 feat: handle "USE <catalog>-<schema>" in MySQL (#857) 
* feat: handle "USE <catalog>-<schema>" in MySQL

* fix: resolve PR comments
 2023-01-12 11:12:11 +08:00
Lei, HUANG
90fcaa8487 feat: expose wal config (#852) 
* feat: wal config

* fix: use human-readable string in wal config

* feat: copy ReadableSize and humanize size config items in toml files

* fix: clippy
 2023-01-10 16:07:26 +08:00
LFC
72f05a3137 feat: flight aboard (#840) 
feat: replace old GRPC interface with Arrow Flight
 2023-01-09 17:06:24 +08:00
Ning Sun
3988770266 feat: add catalog name resolution for postgres and http interface (#810) 
* feat: add catalog name resolution for postgres and http interface

* test: add tests for catalog resolution on http and postgres

* feat: assign custom catalog for query

* chore: order code for better readability
 2023-01-09 11:43:25 +08:00
LFC
50cc0e9b51 feat: Impl Insert functionality of Arrow Flight service for Frontend Instance (#821) 
* feat: Implement Insert functionality of Arrow Flight service for Frontend Instance

* fix: update license content

* Update src/common/grpc-expr/src/alter.rs

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>

* fix: resolve PR comments

* fix: resolve PR comments

Co-authored-by: fys <40801205+Fengys123@users.noreply.github.com>
 2023-01-04 17:48:59 +08:00
dennis zhuang
7762873842 feat: endpoint and region config for s3 storage (#820) 
* feat: adds serde default attribute to options

* feat: adds endpoint and region for s3 config
 2023-01-04 11:24:24 +08:00
Lei, HUANG
8ffc078f88 fix: license header (#815) 2023-01-03 15:09:49 +08:00
LFC
d13de0aeba refactor: remove AdminExpr, make DDL expressions as normal GRPC requests (#808) 
* refactor: remove AdminExpr, make DDL expressions as normal GRPC requests
 2022-12-30 16:47:45 +08:00
LFC
de6803d253 feat: handle InsertRequest(formerly InsertExpr) in new Arrow Flight (#800) 
feat: handle InsertRequest(formerly InsertExpr) in new Arrow Flight interface
 2022-12-30 10:24:09 +08:00
LFC
04df80e640 fix: further ease the restriction of executing SQLs in new GRPC interface (#797) 
* fix: carry not recordbatch result in FlightData, to allow executing SQLs other than selection in new GRPC interface

* Update src/datanode/src/instance/flight/stream.rs

Co-authored-by: Jiachun Feng <jiachun_feng@proton.me>
 2022-12-28 16:43:21 +08:00
Ruihang Xia
26a3e93ca7 chore: util workspace deps in more places (#792) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2022-12-27 16:26:59 +08:00
LFC
7829e4a219 feat: Implement Arrow Flight Service (except gRPC server) for selection (#768) 
* feat: Implement Arrow Flight Service (but not the GRPC server) for selection

Co-authored-by: luofucong <luofucong@greptime.com>
 2022-12-26 16:41:10 +08:00
LFC
dc52a51576 chore: upgrade to Arrow 29.0 and use workspace package and dependencies (#782) 
* chore: upgrade to Arrow 29.0 and use workspace package and dependencies

* fix: resolve PR comments

Co-authored-by: luofucong <luofucong@greptime.com>
 2022-12-23 14:28:37 +08:00
LFC
ea9af42091 chore: upgrade Rust to nightly 2022-12-20 (#772) 
* chore: upgrade Rust to nightly 2022-12-20

* chore: upgrade Rust to nightly 2022-12-20

Co-authored-by: luofucong <luofucong@greptime.com>
 2022-12-21 19:32:30 +08:00