rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-07 22:02:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,038 Commits 103 Branches 226 Tags
eb24bab5df14046ecc93aa308c562200c02c4fa5
Commit Graph

1038 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
LFC
eb24bab5df refactor: set the filters for testing logs (#1637) 
minor: set the filters for testing logs
 2023-05-25 11:07:57 +08:00
fys
8f9e9686fe chore: add metrics for table route getting (#1636) 
chore: add metrics for getting table_route
 2023-05-25 10:02:59 +08:00
shuiyisong
61a32d1b9c chore: add boxed error for custom error map (#1635) 
* chore: add boxed error for custom error map

* chore: fix typo

* chore: add comment & update display msg

* chore: change name to other error
 2023-05-24 12:54:52 +00:00
Weny Xu
74a6517bd0 refactor: move the common part of the heartbeat response handler to common (#1627) 
* refactor: move heartbeat response handler to common

* chore: apply suggestions from CR
 2023-05-24 07:55:06 +00:00
fys
fa4a497d75 feat: add cache for catalog kv backend (#1592) 
* feat: add kvbackend cache

* fix: cargo fmt
 2023-05-24 15:07:29 +08:00
Ning Sun
ddca0307d1 feat: more configurable logging levels (#1630) 
* feat: make logging level more configurable

* chore: resolve lint warnings

* fix: correct default level for h2

* chore: update text copy
 2023-05-24 14:47:41 +08:00
Weny Xu
3dc45f1c13 feat: implement CloseRegionHandler (#1569) 
* feat: implement CloseRegionHandler

* feat: register heartbeat response handlers

* test: add tests for heartbeat response handlers

* fix: drop table does not release regions

* chore: apply suggestion from CR

* fix: fix close region issue

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: modify method name and add log

* refactor: refactor HeartbeatResponseHandler

* chore: apply suggestion from CR

* refactor: remove close method from Region trait

* chore: apply suggestion from CR

* chore: remove PartialEq from CloseTableResult

* chore: apply suggestion from CR
 2023-05-23 15:44:27 +08:00
dennis zhuang
7c55783e53 feat!: reorganize the storage layout (#1609) 
* feat: adds data_home to DataOptions

* refactor: split out object store stuffs from datanode instance

* feat: move data_home into FileConfig

* refactor: object storage layers

* feat: adds datanode path to procedure paths

* feat: temp commit

* refactor: clean code

* fix: forgot files

* fix: forgot files

* Update src/common/test-util/src/ports.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update tests/runner/src/env.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: compile error

* chore: cr comments

* fix: dependencies order in cargo

* fix: data path in test

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-05-23 13:58:26 +08:00
shuiyisong
5b304fa692 chore: add grpc query interceptor (#1626) 2023-05-23 13:57:54 +08:00
Weny Xu
9f67ad8bce fix: fix doesn't release closed regions issue (#1596) 
* fix: fix close region issue

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* chore: apply suggestion from CR

* refactor: remove close method from Region trait

* chore: remove PartialEq from CloseTableResult
 2023-05-23 11:40:12 +08:00
Weny Xu
e646490d16 chore: fix code styling (#1623) 2023-05-23 10:09:34 +08:00
JeremyHi
1225edb065 refactor: move rpc's commons to common-meta (#1625) 2023-05-23 10:07:24 +08:00
Lei, HUANG
8e7ec4626b refactor: remove useless error (#1624) 
* refactor: remove useless

* fix: remove useless error variant
 2023-05-22 22:55:27 +08:00
LFC
f64527da22 feat: region failover procedure (#1558) 
* feat: region failover procedure
 2023-05-22 19:54:52 +08:00
Yingwen
6dbceb1ad5 feat: Trigger flush based on global write buffer size (#1585) 
* feat(storage): Add AllocTracker

* feat(storage): flush request wip

* feat(storage): support global write buffer size

* fix(storage): Test and fix size based strategy

* test(storage): Test AllocTracker

* test(storage): Test pick_by_write_buffer_full

* docs: Add flush config example

* test(storage): Test schedule_engine_flush

* feat(storage): Add metrics for write buffer size

* chore(flush): Add log when triggering flush by global buffer

* chore(storage): track allocation in update_stats
 2023-05-22 19:00:30 +08:00
Ning Sun
067c5ee7ce feat: time_zone variable for mysql connections (#1607) 
* feat: add timezone info to query context

* feat: parse mysql compatible time zone string

* feat: add method to timestamp for rendering timezone aware string

* feat: use timezone from session for time string rendering

* refactor: use querycontectref

* feat: implement session/timezone variable read/write

* style: resolve toml format

* test: update tests

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update src/session/src/context.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* refactor: address review issues

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-05-22 18:30:23 +08:00
Yingwen
32ad358323 fix(table-procedure): Open table in RegisterCatalog state (#1617) 
* fix(table-procedure): on_register_catalog should use open_table

* test: Test recover RegisterCatalog state

* test: Fix subprocedure does not execute in test

* feat(mito): adjust procedure log level

* refactor: rename execute_parent_procedure

execute_parent_procedure -> execute_until_suspended_or_done
 2023-05-22 17:54:02 +08:00
Chuanle Chen
77497ca46a feat: support /api/v1/label/<label_name>/values from Prometheus (#1604) 
* feat: support `/api/v1/label/<label_name>/values` from Prometheus

* chore: apply CR

* chore: apply CR
 2023-05-22 07:24:12 +00:00
JeremyHi
e5a215de46 chore: truncate route-table (#1619) 2023-05-22 14:54:40 +08:00
Ruihang Xia
e5aad0f607 feat: distributed planner basic (#1599) 
* basic skeleton

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* clean up

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* change QueryEngineState's constructor

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* install extension planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tidy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-05-22 11:48:03 +08:00
QuenKar
edf6c0bf48 refactor: add "table engine" to datanode heartbeat. (#1616) 
refactor:add "table engine" to datanode heartbeat.
 2023-05-22 10:09:32 +08:00
Ruihang Xia
c3eeda7d84 refactor(frontend): adjust code structure (#1615) 
* move  to expr_factory

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move configs into service_config

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move GrpcQueryHandler into distributed.rs

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
v0.3.0-nightly-20230522 		2023-05-20 02:09:20 +08:00
Lei, HUANG
82f2b34f4d fix: wal replay ignore manifest entries (#1612) 
* fix: wal replay ignore manifest entries

* test: add ut
 2023-05-19 18:12:44 +08:00
Vanish
8764ce7845 feat: add delete WAL in drop_region (#1577) 
* feat: add delete WAL in drop_region

* chore: fix typo err.

* feat: mark all SSTs deleted and remove the region from StorageEngine's region map.

* test: add test_drop_region for StorageEngine.

* chore: make clippy happy

* fix: fix conflict

* chore: CR.

* chore: CR

* chore: fix clippy

* fix: temp file life time
 2023-05-18 18:02:34 +08:00
localhost
d76ddc575f fix: meta admin API get catalog table name error (#1603) 2023-05-18 14:27:40 +08:00
Weny Xu
68dfea0cfd fix: fix datanode cannot start while failing to open tables (#1601) 2023-05-17 20:56:13 +08:00
fys
57c02af55b feat: change default selector in meta from "LeaseBased" to "LoadBased" (#1598) 
* feat: change default selector from "LeaseBased" to "LoadBased"

* fix: ut
 2023-05-17 17:48:13 +08:00
Lei, HUANG
e8c2222a76 feat: add WindowedReader (#1532) 
* feat: add WindowedReader

* fix: some cr comments

* feat: filter memtable by timestamp range

* fix: add source in error variants

* fix: some CR comments

* refactor: filter memtable in MapIterWrapper

* fix: clippy
 2023-05-17 17:34:29 +08:00
JeremyHi
eb95a9e78b fix: sequence out of range (#1597) 2023-05-17 14:43:54 +08:00
zyy17
4920836021 refactor: support parsing env list (#1595) 
* refactor: support parse env list

* refactor: set 'multiple = true' for metasrv_addr cli option and remove duplicated parsing
 2023-05-17 14:37:08 +08:00
Huaijin
715e1a321f feat: implement /api/v1/labels for prometheus (#1580) 
* feat: implement /api/v1/labels for prometheus

* fix: only gather match[]

* chore: fix typo

* chore: fix typo

* chore: change style

* fix: suggestion

* fix: suggestion

* chore: typo

* fix: fmt

* fix: add more test
 2023-05-17 03:56:22 +00:00
localhost
a6ec79ee30 chore: add a uniform prefix to the metrics using the official recommendation of (#1590) 2023-05-17 11:08:49 +08:00
Lei, HUANG
e70d49b9cf feat: memtable stats (#1591) 
* feat: memtable stats

* chore: add tests for timestamp subtraction

* feat: add `Value:as_timestamp` method
 2023-05-17 11:07:07 +08:00
Weny Xu
ca75a7b744 fix: remove region number validation (#1593) 
* fix: remove region number validation

* Update src/mito/src/engine.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-05-17 09:23:56 +08:00
localhost
3330957896 chore: add fmt for statement query (#1588) 
* chore: add fmt for statement query

* chore: add test for query display
 2023-05-16 16:14:11 +08:00
WU Jingdi
fb1ac0cb9c feat: support user config manifest compression (#1579) 
* feat: support user config manifest compression

* chore: change style

* chore: enhance test
 2023-05-16 11:02:59 +08:00
Niwaka
856ab5bea7 feat: make RepeatedTask invoke remove_outdated_meta method (#1578) 
* feat: make RepeatedTask invoke remove_outdated_meta method

* fix: typo

* chore: improve error message
 2023-05-16 10:21:35 +08:00
Eugene Tolbakov
122bd5f0ab feat(tql): add initial implementation for explain & analyze (#1427) 
* feat(tql): resolve conflicts after merge,formatting and clippy issues, add sqlness tests, adjust explain with start, end, step

* feat(tql): adjust sqlness assertions
 2023-05-16 07:28:24 +08:00
Ruihang Xia
2fd1075c4f fix: uses nextest in the Release CI (#1582) 
* fix: uses nextest in the Release CI

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* install nextest

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update Makefile

Co-authored-by: zyy17 <zyylsxm@gmail.com>

* update workflow yaml

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: zyy17 <zyylsxm@gmail.com>
 2023-05-15 21:09:09 +08:00
fys
027707d969 feat: support frontend-meta heartbeat (#1555) 
* feat: support frontend heartbeat

* fix: typo "reponse" -> "response"

* add ut

* enable start heartbeat task

* chore: frontend id is specified by metasrv, not in the frontend startup parameter

* fix typo

* self-cr

* cr

* cr

* cr

* remove unnecessary headers

* use the member id in the header as the node id
 2023-05-15 09:54:45 +00:00
Yingwen
8d54d40b21 feat: Add FlushPicker to flush regions periodically (#1559) 
* feat: Add FlushPicker

* feat(storage): Add close to StorageEngine

* style(storage): fix clippy

* feat(storage): Close regions in StorageEngine::close

* chore(storage): Clear requests on scheduler stop

* test(storage): Test flush picker

* feat(storage): Add metrics for auto flush

* feat(storage): Add flush reason and record it in metrics

* feat: Expose flush config

docs(config): Update config example

* refactor(storage): Run auto flush task in FlushScheduler

* refactor(storage): Add FlushItem trait to make FlushPicker easy to test
 2023-05-15 17:29:28 +08:00
Ning Sun
497b1f9dc9 feat: metrics for storage engine (#1574) 
* feat: add storage engine region count gauge

* test: remove catalog metrics because we can't get a correct number

* feat: add metrics for log store write and compaction

* fix: address review issues
 2023-05-15 15:22:00 +08:00
LFC
4ae0b5e185 test: move instances tests to "tests-integration" (#1573) 
* test: move standalone and distributed instances tests from "frontend" crate to "tests-integration"

* fix: resolve PR comments
 2023-05-15 12:00:43 +08:00
Lei, HUANG
cfcfc72681 refactor: remove version column (#1576) 2023-05-15 11:03:37 +08:00
Weny Xu
66903d42e1 feat: implement OpenTableHandler (#1567) 
* feat: implement OpenTableHandler

* chore: apply suggestion from CR

* chore: apply suggestion from CR
 2023-05-15 10:47:28 +08:00
zyy17
4fc173acf0 refactor: support layered configuration (#1535) 
* refactor: add a layered configuration by using config-rs

Signed-off-by: zyy17 <zyylsxm@gmail.com>

* refactor: add 'env_var_prefix' for 'load_options()' and remove duplicate default construction in frontend

* refactor: add test_config_precedence_order in standalone

* refactor: add 'test_config_precedence_order()' test case in metasrv

* refactor: add 'test_config_precedence_order()' test case in datanode

* refactor: refine the naming '*_env_var_*' -> '*_env_vars_*'

* refactor: fix clippy error

* refactor: refine error naming 'LoadConfig' -> 'LoadLayeredConfig' and add Location

* refactor: move 'env_vars_prefix' to clap options

* fix: use '__' as envrionment variables separator and simplify load_layered_options()

* refactor: derive 'Default' for StartCommand and use default function to simplify the test cases

* fix: clippy error

* chore: update comments

* chore(deps): update deps info

* refactor(naming): 'env_vars_prefix' -> 'env_prefix'

* refactor: simplify the code

* refactor: change some argument type of 'load_layered_options()'

* refactor: simplify the code

* refactor: remove unnecessary 'clone()'

* refactor: add 'GREPTIMEDB_*' prefix for env_prefix

* refactor: modify configuration precedence order: cli > config file > environment variables > default values

---------

Signed-off-by: zyy17 <zyylsxm@gmail.com>
v0.3.0-nightly-20230515 		2023-05-13 22:37:47 +08:00
Huaijin
f9a4326461 fix: unwrap() None in NULL value exist multi-field table during prometheus query_range (#1571) 
* fix: NULL value in multi-field table meet error in prometheus query_range

* fix: suggestion

* chore: change style
 2023-05-12 17:36:03 +08:00
Ning Sun
4151d7a8ea fix: allow cross-schema query on information_schema (#1568) 2023-05-11 16:54:28 +08:00
LFC
a4e106380b fix: refreshing Dashboard returns 404 (#1562) 
* fix: refreshing Dashboard returns 404

* fix: refreshing Dashboard returns 404
 2023-05-11 15:08:20 +08:00
Ruihang Xia
7a310cb056 docs: rfc of distributed planner (#1554) 
* docs: rfc of distributed planner

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Update docs/rfcs/2023-05-09-distributed-planner.md

Co-authored-by: LFC <bayinamine@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: LFC <bayinamine@gmail.com>
 2023-05-11 14:45:32 +08:00