rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-05 21:02:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,020 Commits 105 Branches 225 Tags
e5a215de465234fbff2558ea4ea78e1efe7b9904
Commit Graph

1020 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
JeremyHi
e5a215de46 chore: truncate route-table (#1619) 2023-05-22 14:54:40 +08:00
Ruihang Xia
e5aad0f607 feat: distributed planner basic (#1599) 
* basic skeleton

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change QueryEngineState's constructor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* install extension planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tidy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-22 11:48:03 +08:00
QuenKar
edf6c0bf48 refactor: add "table engine" to datanode heartbeat. (#1616) 
refactor:add "table engine" to datanode heartbeat.
 2023-05-22 10:09:32 +08:00
Ruihang Xia
c3eeda7d84 refactor(frontend): adjust code structure (#1615) 
* move  to expr_factory

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move configs into service_config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move GrpcQueryHandler into distributed.rs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
v0.3.0-nightly-20230522 		2023-05-20 02:09:20 +08:00
Lei, HUANG
82f2b34f4d fix: wal replay ignore manifest entries (#1612) 
* fix: wal replay ignore manifest entries

* test: add ut
 2023-05-19 18:12:44 +08:00
Vanish
8764ce7845 feat: add delete WAL in drop_region (#1577) 
* feat: add delete WAL in drop_region

* chore: fix typo err.

* feat: mark all SSTs deleted and remove the region from StorageEngine's region map.

* test: add test_drop_region for StorageEngine.

* chore: make clippy happy

* fix: fix conflict

* chore: CR.

* chore: CR

* chore: fix clippy

* fix: temp file life time
 2023-05-18 18:02:34 +08:00
localhost
d76ddc575f fix: meta admin API get catalog table name error (#1603) 2023-05-18 14:27:40 +08:00
Weny Xu
68dfea0cfd fix: fix datanode cannot start while failing to open tables (#1601) 2023-05-17 20:56:13 +08:00
fys
57c02af55b feat: change default selector in meta from "LeaseBased" to "LoadBased" (#1598) 
* feat: change default selector from "LeaseBased" to "LoadBased"

* fix: ut
 2023-05-17 17:48:13 +08:00
Lei, HUANG
e8c2222a76 feat: add WindowedReader (#1532) 
* feat: add WindowedReader

* fix: some cr comments

* feat: filter memtable by timestamp range

* fix: add source in error variants

* fix: some CR comments

* refactor: filter memtable in MapIterWrapper

* fix: clippy
 2023-05-17 17:34:29 +08:00
JeremyHi
eb95a9e78b fix: sequence out of range (#1597) 2023-05-17 14:43:54 +08:00
zyy17
4920836021 refactor: support parsing env list (#1595) 
* refactor: support parse env list

* refactor: set 'multiple = true' for metasrv_addr cli option and remove duplicated parsing
 2023-05-17 14:37:08 +08:00
Huaijin
715e1a321f feat: implement /api/v1/labels for prometheus (#1580) 
* feat: implement /api/v1/labels for prometheus

* fix: only gather match[]

* chore: fix typo

* chore: fix typo

* chore: change style

* fix: suggestion

* fix: suggestion

* chore: typo

* fix: fmt

* fix: add more test
 2023-05-17 03:56:22 +00:00
localhost
a6ec79ee30 chore: add a uniform prefix to the metrics using the official recommendation of (#1590) 2023-05-17 11:08:49 +08:00
Lei, HUANG
e70d49b9cf feat: memtable stats (#1591) 
* feat: memtable stats

* chore: add tests for timestamp subtraction

* feat: add `Value:as_timestamp` method
 2023-05-17 11:07:07 +08:00
Weny Xu
ca75a7b744 fix: remove region number validation (#1593) 
* fix: remove region number validation

* Update src/mito/src/engine.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-05-17 09:23:56 +08:00
localhost
3330957896 chore: add fmt for statement query (#1588) 
* chore: add fmt for statement query

* chore: add test for query display
 2023-05-16 16:14:11 +08:00
WU Jingdi
fb1ac0cb9c feat: support user config manifest compression (#1579) 
* feat: support user config manifest compression

* chore: change style

* chore: enhance test
 2023-05-16 11:02:59 +08:00
Niwaka
856ab5bea7 feat: make RepeatedTask invoke remove_outdated_meta method (#1578) 
* feat: make RepeatedTask invoke remove_outdated_meta method

* fix: typo

* chore: improve error message
 2023-05-16 10:21:35 +08:00
Eugene Tolbakov
122bd5f0ab feat(tql): add initial implementation for explain & analyze (#1427) 
* feat(tql): resolve conflicts after merge,formatting and clippy issues, add sqlness tests, adjust explain with start, end, step

* feat(tql): adjust sqlness assertions
 2023-05-16 07:28:24 +08:00
Ruihang Xia
2fd1075c4f fix: uses nextest in the Release CI (#1582) 
* fix: uses nextest in the Release CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* install nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update Makefile

Co-authored-by: zyy17 <zyylsxm@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: zyy17 <zyylsxm@gmail.com>
 2023-05-15 21:09:09 +08:00
fys
027707d969 feat: support frontend-meta heartbeat (#1555) 
* feat: support frontend heartbeat

* fix: typo "reponse" -> "response"

* add ut

* enable start heartbeat task

* chore: frontend id is specified by metasrv, not in the frontend startup parameter

* fix typo

* self-cr

* cr

* cr

* cr

* remove unnecessary headers

* use the member id in the header as the node id
 2023-05-15 09:54:45 +00:00
Yingwen
8d54d40b21 feat: Add FlushPicker to flush regions periodically (#1559) 
* feat: Add FlushPicker

* feat(storage): Add close to StorageEngine

* style(storage): fix clippy

* feat(storage): Close regions in StorageEngine::close

* chore(storage): Clear requests on scheduler stop

* test(storage): Test flush picker

* feat(storage): Add metrics for auto flush

* feat(storage): Add flush reason and record it in metrics

* feat: Expose flush config

docs(config): Update config example

* refactor(storage): Run auto flush task in FlushScheduler

* refactor(storage): Add FlushItem trait to make FlushPicker easy to test
 2023-05-15 17:29:28 +08:00
Ning Sun
497b1f9dc9 feat: metrics for storage engine (#1574) 
* feat: add storage engine region count gauge

* test: remove catalog metrics because we can't get a correct number

* feat: add metrics for log store write and compaction

* fix: address review issues
 2023-05-15 15:22:00 +08:00
LFC
4ae0b5e185 test: move instances tests to "tests-integration" (#1573) 
* test: move standalone and distributed instances tests from "frontend" crate to "tests-integration"

* fix: resolve PR comments
 2023-05-15 12:00:43 +08:00
Lei, HUANG
cfcfc72681 refactor: remove version column (#1576) 2023-05-15 11:03:37 +08:00
Weny Xu
66903d42e1 feat: implement OpenTableHandler (#1567) 
* feat: implement OpenTableHandler

* chore: apply suggestion from CR

* chore: apply suggestion from CR
 2023-05-15 10:47:28 +08:00
zyy17
4fc173acf0 refactor: support layered configuration (#1535) 
* refactor: add a layered configuration by using config-rs

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add 'env_var_prefix' for 'load_options()' and remove duplicate default construction in frontend

* refactor: add test_config_precedence_order in standalone

* refactor: add 'test_config_precedence_order()' test case in metasrv

* refactor: add 'test_config_precedence_order()' test case in datanode

* refactor: refine the naming '*_env_var_*' -> '*_env_vars_*'

* refactor: fix clippy error

* refactor: refine error naming 'LoadConfig' -> 'LoadLayeredConfig' and add Location

* refactor: move 'env_vars_prefix' to clap options

* fix: use '__' as envrionment variables separator and simplify load_layered_options()

* refactor: derive 'Default' for StartCommand and use default function to simplify the test cases

* fix: clippy error

* chore: update comments

* chore(deps): update deps info

* refactor(naming): 'env_vars_prefix' -> 'env_prefix'

* refactor: simplify the code

* refactor: change some argument type of 'load_layered_options()'

* refactor: simplify the code

* refactor: remove unnecessary 'clone()'

* refactor: add 'GREPTIMEDB_*' prefix for env_prefix

* refactor: modify configuration precedence order: cli > config file > environment variables > default values

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
v0.3.0-nightly-20230515 		2023-05-13 22:37:47 +08:00
Huaijin
f9a4326461 fix: unwrap() None in NULL value exist multi-field table during prometheus query_range (#1571) 
* fix: NULL value in multi-field table meet error in prometheus query_range

* fix: suggestion

* chore: change style
 2023-05-12 17:36:03 +08:00
Ning Sun
4151d7a8ea fix: allow cross-schema query on information_schema (#1568) 2023-05-11 16:54:28 +08:00
LFC
a4e106380b fix: refreshing Dashboard returns 404 (#1562) 
* fix: refreshing Dashboard returns 404

* fix: refreshing Dashboard returns 404
 2023-05-11 15:08:20 +08:00
Ruihang Xia
7a310cb056 docs: rfc of distributed planner (#1554) 
* docs: rfc of distributed planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update docs/rfcs/2023-05-09-distributed-planner.md

Co-authored-by: LFC <bayinamine@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-05-11 14:45:32 +08:00
LFC
8fef32f8ef feat: enable tokio console in cluster mode (#1512) 
* feat: enable tokio console subscriber

* fix: resolve PR comments

* fix: resolve PR comments

* fix: resolve PR comments
 2023-05-11 14:35:06 +08:00
Ning Sun
8c85fdec29 fix: correct schema/table count in catalog metrics (#1565) 2023-05-11 14:20:42 +08:00
ZonaHe
84f6b46437 feat: update dashboard to v0.2.5 (#1563) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-05-11 13:55:42 +08:00
Weny Xu
44aef6fcbd feat(datanode): iImplement the heartbeat response handler (#1547) 
* feat(datanode): implement instruction handler

* chore: apply suggestion from CR

* refactor: refactor heartbeat response handler
 2023-05-11 09:27:13 +08:00
JeremyHi
7a9dd5f0c8 feat: ignore mailbox message into stat (#1560) 2023-05-10 18:06:04 +08:00
WU Jingdi
486bb2ee8e feat: Compress manifest and checkpoint (#1497) 
* feat: Compress manifest and checkpoint

* refactor: use file extention infer compression type

* chore: apply suggestions from CR

* Update src/storage/src/manifest/storage.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: CR advices

* chore: Fix bugs, strengthen test

* chore: Fix CR, strengthen test

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-05-10 07:53:06 +00:00
Weny Xu
020c55e260 refactor: change mailbox_messages to mailbox_message (#1557) 2023-05-10 07:17:11 +00:00
Yingwen
ee3e1dbdaa feat: Use LocalScheduler framework to implement FlushScheduler (#1531) 
* test: simplify countdownlatch

* feat: impl Drop for LocalScheduler

* feat(storage): Impl FlushRequest and FlushHandler

* feat(storage): Use scheduler to handle flush job

* chore(storage): remove unused code

* feat(storage): Use new type pattern for RegionMap

* feat(storage): Remove on_success callback

* feat(storage): Address CR comments and add some metrics to flush
 2023-05-10 07:16:51 +00:00
dennis zhuang
aa0c5b888c docs: update readme (#1549) 
* docs: update readme

* Update README.md

Co-authored-by: Ning Sun <classicning@gmail.com>

* chore: cr comments

* chore: cr comments

---------

Co-authored-by: Ning Sun <classicning@gmail.com>
 2023-05-10 14:36:07 +08:00
Weny Xu
fbb7db42aa chore: unify code styling (#1523) v0.3.0-nightly-20230510 2023-05-10 11:10:39 +08:00
Ning Sun
a1587595d9 feat: add information_schema as exception of cross schema check (#1551) 
* feat: add information_schema as a cross-schema query exception

* fix: resolve lint issue
 2023-05-10 10:55:00 +08:00
Weny Xu
abd5a8ecbb chore(datasource): make CompressionType follow the style of the guide (#1522) 2023-05-10 10:50:24 +08:00
Ruihang Xia
4ddab8e982 build: change release CI to only run test on linux (#1548) 
* disable all linux release

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split linux and macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* correct job name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add missing build job

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* run build-macos first

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable unstable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* disable test on macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* re-enable test

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* do not dependent on build-macos

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-10 10:49:14 +08:00
Yingwen
1833e487a4 refactor: remove unnecessary async from RepeatedTask::start (#1545) 
* refactor: relax RepeatedTask requirements

Some refactor:
- Remove async from start()
- Cancel task in drop
- Allow TaskFunction::call taking &mut self
- Make start/stop concurrent safe

* test(log-store): Fix log store tests (start multiple times)
 2023-05-09 21:03:15 +08:00
ZonaHe
c93b5743e8 feat: update dashboard to v0.2.4 (#1553) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-05-09 20:56:20 +08:00
Weny Xu
550c494d25 fix: Copy from must follow the order of table fields issue (#1521) 
* fix: Copy from must follow the order of table fields issue

* chore: apply suggestion from CR
 2023-05-09 17:46:16 +08:00
Yingwen
2ab0e42d6f feat: clean procedure's state after it is done (#1543) 
* feat(common-procedure): pub(crate) use proc_path

* feat(common-procedure): Implement delete_procedure

* feat(common-procedure): Clean procedure after it is finished

* chore(common-procedure): put path_string in front of try_stream

* test(common-procedure): Test cleaning up procedures

* feat(common-procedure): Clean procedure states in recover()

* feat(common-procedure): Use VecDeque for finished procedures
 2023-05-09 11:44:50 +08:00
JeremyHi
05e6ca1e14 fix: the latest number of regions (#1546) 
* fix: the latest number of regions

* fix: unit test
 2023-05-09 10:11:26 +08:00