rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-08 06:12:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,001 Commits 103 Branches 226 Tags
122bd5f0ab465537c878fbab50e8072db4adcce9
Commit Graph

1001 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eugene Tolbakov
122bd5f0ab feat(tql): add initial implementation for explain & analyze (#1427) 
* feat(tql): resolve conflicts after merge,formatting and clippy issues, add sqlness tests, adjust explain with start, end, step

* feat(tql): adjust sqlness assertions
 2023-05-16 07:28:24 +08:00
Ruihang Xia
2fd1075c4f fix: uses nextest in the Release CI (#1582) 
* fix: uses nextest in the Release CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* install nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update Makefile

Co-authored-by: zyy17 <zyylsxm@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: zyy17 <zyylsxm@gmail.com>
 2023-05-15 21:09:09 +08:00
fys
027707d969 feat: support frontend-meta heartbeat (#1555) 
* feat: support frontend heartbeat

* fix: typo "reponse" -> "response"

* add ut

* enable start heartbeat task

* chore: frontend id is specified by metasrv, not in the frontend startup parameter

* fix typo

* self-cr

* cr

* cr

* cr

* remove unnecessary headers

* use the member id in the header as the node id
 2023-05-15 09:54:45 +00:00
Yingwen
8d54d40b21 feat: Add FlushPicker to flush regions periodically (#1559) 
* feat: Add FlushPicker

* feat(storage): Add close to StorageEngine

* style(storage): fix clippy

* feat(storage): Close regions in StorageEngine::close

* chore(storage): Clear requests on scheduler stop

* test(storage): Test flush picker

* feat(storage): Add metrics for auto flush

* feat(storage): Add flush reason and record it in metrics

* feat: Expose flush config

docs(config): Update config example

* refactor(storage): Run auto flush task in FlushScheduler

* refactor(storage): Add FlushItem trait to make FlushPicker easy to test
 2023-05-15 17:29:28 +08:00
Ning Sun
497b1f9dc9 feat: metrics for storage engine (#1574) 
* feat: add storage engine region count gauge

* test: remove catalog metrics because we can't get a correct number

* feat: add metrics for log store write and compaction

* fix: address review issues
 2023-05-15 15:22:00 +08:00
LFC
4ae0b5e185 test: move instances tests to "tests-integration" (#1573) 
* test: move standalone and distributed instances tests from "frontend" crate to "tests-integration"

* fix: resolve PR comments
 2023-05-15 12:00:43 +08:00
Lei, HUANG
cfcfc72681 refactor: remove version column (#1576) 2023-05-15 11:03:37 +08:00
Weny Xu
66903d42e1 feat: implement OpenTableHandler (#1567) 
* feat: implement OpenTableHandler

* chore: apply suggestion from CR

* chore: apply suggestion from CR
 2023-05-15 10:47:28 +08:00
zyy17
4fc173acf0 refactor: support layered configuration (#1535) 
* refactor: add a layered configuration by using config-rs

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add 'env_var_prefix' for 'load_options()' and remove duplicate default construction in frontend

* refactor: add test_config_precedence_order in standalone

* refactor: add 'test_config_precedence_order()' test case in metasrv

* refactor: add 'test_config_precedence_order()' test case in datanode

* refactor: refine the naming '*_env_var_*' -> '*_env_vars_*'

* refactor: fix clippy error

* refactor: refine error naming 'LoadConfig' -> 'LoadLayeredConfig' and add Location

* refactor: move 'env_vars_prefix' to clap options

* fix: use '__' as envrionment variables separator and simplify load_layered_options()

* refactor: derive 'Default' for StartCommand and use default function to simplify the test cases

* fix: clippy error

* chore: update comments

* chore(deps): update deps info

* refactor(naming): 'env_vars_prefix' -> 'env_prefix'

* refactor: simplify the code

* refactor: change some argument type of 'load_layered_options()'

* refactor: simplify the code

* refactor: remove unnecessary 'clone()'

* refactor: add 'GREPTIMEDB_*' prefix for env_prefix

* refactor: modify configuration precedence order: cli > config file > environment variables > default values

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
v0.3.0-nightly-20230515 		2023-05-13 22:37:47 +08:00
Huaijin
f9a4326461 fix: unwrap() None in NULL value exist multi-field table during prometheus query_range (#1571) 
* fix: NULL value in multi-field table meet error in prometheus query_range

* fix: suggestion

* chore: change style
 2023-05-12 17:36:03 +08:00
Ning Sun
4151d7a8ea fix: allow cross-schema query on information_schema (#1568) 2023-05-11 16:54:28 +08:00
LFC
a4e106380b fix: refreshing Dashboard returns 404 (#1562) 
* fix: refreshing Dashboard returns 404

* fix: refreshing Dashboard returns 404
 2023-05-11 15:08:20 +08:00
Ruihang Xia
7a310cb056 docs: rfc of distributed planner (#1554) 
* docs: rfc of distributed planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update docs/rfcs/2023-05-09-distributed-planner.md

Co-authored-by: LFC <bayinamine@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-05-11 14:45:32 +08:00
LFC
8fef32f8ef feat: enable tokio console in cluster mode (#1512) 
* feat: enable tokio console subscriber

* fix: resolve PR comments

* fix: resolve PR comments

* fix: resolve PR comments
 2023-05-11 14:35:06 +08:00
Ning Sun
8c85fdec29 fix: correct schema/table count in catalog metrics (#1565) 2023-05-11 14:20:42 +08:00
ZonaHe
84f6b46437 feat: update dashboard to v0.2.5 (#1563) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-05-11 13:55:42 +08:00
Weny Xu
44aef6fcbd feat(datanode): iImplement the heartbeat response handler (#1547) 
* feat(datanode): implement instruction handler

* chore: apply suggestion from CR

* refactor: refactor heartbeat response handler
 2023-05-11 09:27:13 +08:00
JeremyHi
7a9dd5f0c8 feat: ignore mailbox message into stat (#1560) 2023-05-10 18:06:04 +08:00
WU Jingdi
486bb2ee8e feat: Compress manifest and checkpoint (#1497) 
* feat: Compress manifest and checkpoint

* refactor: use file extention infer compression type

* chore: apply suggestions from CR

* Update src/storage/src/manifest/storage.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: CR advices

* chore: Fix bugs, strengthen test

* chore: Fix CR, strengthen test

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-05-10 07:53:06 +00:00
Weny Xu
020c55e260 refactor: change mailbox_messages to mailbox_message (#1557) 2023-05-10 07:17:11 +00:00
Yingwen
ee3e1dbdaa feat: Use LocalScheduler framework to implement FlushScheduler (#1531) 
* test: simplify countdownlatch

* feat: impl Drop for LocalScheduler

* feat(storage): Impl FlushRequest and FlushHandler

* feat(storage): Use scheduler to handle flush job

* chore(storage): remove unused code

* feat(storage): Use new type pattern for RegionMap

* feat(storage): Remove on_success callback

* feat(storage): Address CR comments and add some metrics to flush
 2023-05-10 07:16:51 +00:00
dennis zhuang
aa0c5b888c docs: update readme (#1549) 
* docs: update readme

* Update README.md

Co-authored-by: Ning Sun <classicning@gmail.com>

* chore: cr comments

* chore: cr comments

---------

Co-authored-by: Ning Sun <classicning@gmail.com>
 2023-05-10 14:36:07 +08:00
Weny Xu
fbb7db42aa chore: unify code styling (#1523) v0.3.0-nightly-20230510 2023-05-10 11:10:39 +08:00
Ning Sun
a1587595d9 feat: add information_schema as exception of cross schema check (#1551) 
* feat: add information_schema as a cross-schema query exception

* fix: resolve lint issue
 2023-05-10 10:55:00 +08:00
Weny Xu
abd5a8ecbb chore(datasource): make CompressionType follow the style of the guide (#1522) 2023-05-10 10:50:24 +08:00
Ruihang Xia
4ddab8e982 build: change release CI to only run test on linux (#1548) 
* disable all linux release

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split linux and macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* correct job name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add missing build job

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* run build-macos first

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable unstable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable test on macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* re-enable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* do not dependent on build-macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-10 10:49:14 +08:00
Yingwen
1833e487a4 refactor: remove unnecessary async from RepeatedTask::start (#1545) 
* refactor: relax RepeatedTask requirements

Some refactor:
- Remove async from start()
- Cancel task in drop
- Allow TaskFunction::call taking &mut self
- Make start/stop concurrent safe

* test(log-store): Fix log store tests (start multiple times)
 2023-05-09 21:03:15 +08:00
ZonaHe
c93b5743e8 feat: update dashboard to v0.2.4 (#1553) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-05-09 20:56:20 +08:00
Weny Xu
550c494d25 fix: Copy from must follow the order of table fields issue (#1521) 
* fix: Copy from must follow the order of table fields issue

* chore: apply suggestion from CR
 2023-05-09 17:46:16 +08:00
Yingwen
2ab0e42d6f feat: clean procedure's state after it is done (#1543) 
* feat(common-procedure): pub(crate) use proc_path

* feat(common-procedure): Implement delete_procedure

* feat(common-procedure): Clean procedure after it is finished

* chore(common-procedure): put path_string in front of try_stream

* test(common-procedure): Test cleaning up procedures

* feat(common-procedure): Clean procedure states in recover()

* feat(common-procedure): Use VecDeque for finished procedures
 2023-05-09 11:44:50 +08:00
JeremyHi
05e6ca1e14 fix: the latest number of regions (#1546) 
* fix: the latest number of regions

* fix: unit test
 2023-05-09 10:11:26 +08:00
localhost
b9661818f2 chore: remove useless Option type in plugins (#1544) 
Co-authored-by: paomian <qtang@greptime.com>
 2023-05-08 21:54:24 +08:00
localhost
f86390345c chore: remove useless Option type in plugins (#1544) 
Co-authored-by: paomian <qtang@greptime.com>
 2023-05-08 21:53:45 +08:00
localhost
7191bb9652 chore: remove useless Option type in plugins (#1544) 
Co-authored-by: paomian <qtang@greptime.com>
 2023-05-08 21:52:12 +08:00
localhost
34c7f78861 chore: add configurator to http server (#1488) 
* chore: add configurator params to start server fun

* chore: update plugins type

---------

Co-authored-by: paomian <qtang@greptime.com>
 2023-05-08 10:55:03 +00:00
JeremyHi
610651fa8f feat: meta metrics (#1538) 
* chore: from_etcd_kv (better name)

* feat: kv request metric

* feat: router metric

* feat: connections metric
 2023-05-08 17:50:21 +08:00
fys
c48067f88d fix: no active datanode when frontend start (#1533) 
* fix: no active datanode when frontend start

* chore: add log when can not get stat_val
 2023-05-08 15:02:07 +08:00
Ning Sun
ec1b95c250 docs: add play section (#1528) 
* docs: add play section

* Update README.md

Co-authored-by: xiaomin tang <xtang@users.noreply.github.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: xiaomin tang <xtang@users.noreply.github.com>
 2023-05-08 14:26:22 +08:00
gitccl
fbf1ddd006 feat: open catalogs and schemas in parallel (#1527) 
* feat: open catalogs and schemas in parallel

* fix: code review
 2023-05-08 10:34:30 +08:00
Ning Sun
d679cfcb53 feat: add semantic_type to information_schema.columns (#1530) 2023-05-06 15:48:37 +08:00
discord9
2c82ded975 feat: table metrics (#1469) 
* feat: Statistic

* add todo

* fmt: cargo fmt

* feat: some simple impl for MemTable

* chore: a try on adding statistics

* Update src/table/src/stats.rs

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

* docs: fix typo

* newlines unnecessary

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-05-06 14:59:49 +08:00
Ruihang Xia
d4f3f617e4 chore(toolchain): update rust-toolchain to 2023-05-03 (#1524) 
* chore(toolchain): update rust-toolchain to 2023-05-03

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-06 11:34:09 +08:00
Ruihang Xia
6fe117d7d5 fix: vector and matrix in Prometheus use different field (#1520) 
* fix empty tag

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix result type

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* make it work

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-05 15:54:26 +08:00
Ning Sun
b0ab641602 feat: add catalog/schema/table count as catalog metrics (#1499) 
* feat: add catalog/schema/table count as catalog metrics

* test: add integration tests for catalog metrics
 2023-05-05 05:54:12 +00:00
Huaijin
224ec9bd25 fix: wrong max_table_id log in remote catalog manager (#1516) 
* fix: wrong max_table_id log in remote catalog manager

* chore: update link in CONTRIBUTING.md

* chore: add a new const MAX_SYS_TABLE_ID
 2023-05-05 03:39:45 +00:00
Niwaka
d86b3386dc fix: incorrect show create table output (#1514) 
* fix: incorrect show create table output

* feat: change CreateTable's Display if table is external

* feat: change CreateTable's Display if table is external
 2023-05-05 11:29:09 +08:00
Lei, HUANG
c8301feed7 fix: respect MySQL timestamp format (#1510) 2023-05-04 18:57:38 +08:00
dennis zhuang
b1920c41a4 fix: object store cache bug (#1482) 
* feat: use streaming read instead of reading whole file

* feat: enable atomic writing for object store file caching

* fix: recover existing keys from local cache

* test: recovering keys from local file cache for LruCachePolicy

* Update src/datanode/src/instance.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: cr comments

* feat: md5 hash caching path

* fix: test

* fix: read cache

* Update src/object-store/src/cache_policy.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-05-04 18:25:40 +08:00
Weny Xu
c471007edd feat: support to copy table from/to CSV and JSON format file (#1475) 
* refactor: refactor copy from executor

* feat: support to copy from CSV and JSON format files

* feat: support to copy table to the CSV and JSON format file

* test: add tests copy from/to

* chore: apply suggestions from CR
 2023-05-04 17:20:28 +08:00
Yingwen
2818f466d3 feat: Log error in GreptimeRequestHandler (#1507) 
* feat(common-error): Add should_log_error

* feat(servers): log error in grpc handler
 2023-05-04 15:48:38 +08:00