rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-06 21:32:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,007 Commits 105 Branches 226 Tags
0d4265104725bf58f3c2f8615740068dea737259
Commit Graph

2007 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
zyy17
0d42651047 feat: add '--store-key-prefix' to support multiple meta instances can use the same etcd backend (#2988) 
* feat: add '--store-key-prefix' to support multiple meta instances can use the same etcd backend

* refactor: refine lock_key
 2023-12-25 09:33:25 +00:00
niebayes
bab198ae68 feat(remote_wal): impl kafka log store (#2971) 
* feat: introduce client manager

* chore: add errors for client manager

* chore: add record utils

* chore: impl kafka log store

* chore: build kafka log store upon starting datanode

* chore: update comments for kafka log store

* chore: add a todo for getting entry offset

* fix: typo

* chore: remove unused

* chore: update comments

* fix: typo

* fix: resolve some review conversations

* chore: move commonly referenced crates to workspace Cargo.toml

* fix: style

* fix: style

* chore: unify topic name prefix

* chore: make backoff config configurable by users

* chore: properly use backoff config in wal config

* refactor: read/write of kafka log store

* fix: typo

* fix: typo

* fix: resolve review conversations
 2023-12-25 09:21:52 +00:00
niebayes
d4ac8734bc refactor(remote_wal): entry id usage (#2986) 
* chore: update comments for log store stuff

* refactor: entry id usage

* tmp update

* Revert "tmp update"

This reverts commit fcfcda2851.

* fix: resolve review conversations

* fix: resolve review conversations

* chore: remove entry offset
 2023-12-25 07:30:27 +00:00
Lei, HUANG
4664cc601c fix: install nextest bin (#2990) 
* use binstall to install nextest

* fix: change all docker files

* Update docker/dev-builder/centos/Dockerfile

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-12-25 06:01:47 +00:00
Weny Xu
06fd7fd210 feat: add skip_wal_replay to OpenRegion instruction (#2977) 
feat: add skip_wal_replay to OpenRegion instruction
 2023-12-23 06:42:21 +00:00
dennis zhuang
d7b2e791b9 fix: duplicate information_schema (#2979) 
* fix: duplicate information_schema

* chore: style

* fix: comment in sqlness
 2023-12-22 14:02:11 +00:00
niebayes
7d509e97f6 chore: move some commonly referenced crates to workspace Cargo.toml (#2981) 
fix: resolve conflicts
 2023-12-22 09:13:18 +00:00
Wei
a7349b573b feat: support fetch ranges in concurrent (#2959) 
* feat: concurrent fetch ranges

* chore: cr comment

* chore: cr comment

* chore: clippy
 2023-12-22 08:47:55 +00:00
niebayes
830a91c548 feat(remote_wal): implement topic allocation (#2970) 
* chore: implement wal options allocator

* chore: implement round-robin topic selector

* feat: add shuffle to round-robin topic selector

* chore: implement kafka topic manager

* test: add tests for wal options allocator

* test: add wal provider to test config files

* test: leave todos for adding tests for remote wal

* fix: resolve review conversations

* fix: typo
 2023-12-22 08:26:48 +00:00
ZonaHe
6221e5b105 feat: update dashboard to v0.4.4 (#2978) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-12-22 06:37:39 +00:00
Ning Sun
43f01cc594 feat: add a default internal schema (#2974) 2023-12-22 06:25:19 +00:00
WU Jingdi
675767c023 feat: Support export metric in remote write format (#2928) 
* feat: remote write metric task

* chore: pass stanalone task to frontend

* chore: change name to system metric

* fix: add header and rename to export metrics
 2023-12-22 02:09:12 +00:00
dennis zhuang
054bca359e feat: adds build_info table (#2969) 2023-12-21 12:28:09 +00:00
Weny Xu
ff8c10eae7 feat: add CatchupRequest to engine (#2939) 
* chore: remove redundant code

* feat(mito): add CatchupRequest

feat: reopen region before replay if need

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* Apply suggestions from code review

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-12-21 08:27:53 +00:00
Zhenchi
b5c5458798 refactor: remove unused code for pruning row groups (#2973) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-12-21 07:54:29 +00:00
Ruihang Xia
6a1f5751c6 feat(metric-engine): open and close metric regions (#2961) 
* implementation

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* tests

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add metrics

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* define consts

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix compile error

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* print region id with display

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* only ignore region not found

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-12-21 07:33:22 +00:00
LFC
8776b1204b feat: use explicitly set table id (#2945) 
* feat: make standalone table metadata allocator able to use explicitly set table id

* rebase

* fix: resolve PR comments
 2023-12-21 05:51:43 +00:00
Yiran
bad89185c2 ci: add user-doc label checker (#2967) 2023-12-21 03:04:03 +00:00
Weny Xu
6c1c7d8d24 feat: allow initializing regions in background (#2930) 
* refactor: open regions in background

* feat: add status code of RegionNotReady

* feat: use RegionNotReady instead of RegionNotFound for a registering region

* chore: apply suggestions from CR

* feat: add status code of RegionBusy

* feat: return RegionBusy for mutually exclusive operations
 2023-12-20 11:37:46 +00:00
niebayes
9da1f236d9 feat(remote_wal): add skeleton for remote wal related to datanode (#2941) 
* refactor: refactor wal config

* test: update tests related to wal

* feat: introduce kafka wal config

* chore: augment proto with wal options

* feat: augment region open request with wal options

* feat: augment mito region with wal options

* feat: augment region create request with wal options

* refactor: refactor log store trait

* feat: add skeleton for kafka log store

* feat: generalize building log store when starting datanode

* feat: integrate wal options to region write

* chore: minor update

* refactor: remove wal options from region create/open requests

* fix: compliation issues

* chore: insert wal options into region options upon initializing region server

* chore: integrate wal options into region options

* chore: fill in kafka wal config

* chore: reuse namespaces while writing to wal

* chore: minor update

* chore: fetch wal options from region while handling truncate/flush

* fix: region options test

* fix: resolve some review conversations

* refactor: serde with wal options

* fix: resolve some review conversations
 2023-12-20 09:01:17 +00:00
LFC
6ac47e939c feat: MetricsEngine table route (part 1) (#2952) 
* refactor: make `TableRouteValue` an enum to add the variant of MetricsEngine table route

* fix: resolve PR comments

* Update src/common/meta/src/key/table_route.rs

Co-authored-by: Weny Xu <wenymedia@gmail.com>

---------

Co-authored-by: Weny Xu <wenymedia@gmail.com>
 2023-12-20 07:31:59 +00:00
Zhenchi
7d1724f832 feat(inverted_index.create): add index creator (#2960) 
* feat(inverted_index.create): add read/write for external intermediate files

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.create): add external sorter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: drop the stream as early as possible to avoid recursive calls to poll

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: project merge sorted stream

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add total_row_count to SortOutput

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: remove change of format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.create): add index creator

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add check for total_row_count

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: lazy set meta of writer

This reverts commit 63cb5bdb5c3a08406d978357d8167ca18ed1b83b.

* feat: lazyily provide inverted index writer

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish readability

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add push_with_name_n

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-12-20 07:02:13 +00:00
Weny Xu
d2f49cbc2e chore: remove debug = 1 of dev profile (#2966) 2023-12-20 04:37:46 +00:00
Weny Xu
97c3755ab6 feat(mito): add skip_wal_replay option to OpenRegionRequest (#2955) 
* feat(mito): add skip_replay_wal option to OpenRegionRequest

* test: add tests for skip replay wal

* chore: rename `skip_replay_wal` to `skip_wal_replay`
 2023-12-20 03:31:48 +00:00
JeremyHi
5f8c17514f chore: SelectorType use snake_case (#2962) 
chore: let SelectorType use snake_case
 2023-12-20 02:28:29 +00:00
niebayes
839e653e0d feat(remote_wal): add skeleton for remote wal related to meta srv (#2933) 
* feat: introduce wal config and kafka config

* feat: introduce kafka topic manager and selector

* feat: introduce region wal options

* chore: build region wal options upon starting meta srv

* feat: integrate region wal options allocator into table meta allocator

* chore: add wal config to metasrv.example.toml

* chore: add region wal options map to create table procedure

* feat: augment region create request with wal options

* feat: augment DatanodeTableValue with region wal options map

* chore: encode region wal options upon constructing table creator

* feat: persist region wal options when creating table meta

* fix: sqlness test

* chore: set default wal provider to raft-engine

* refactor: refactor wal options

* chore: update wal options allocator

* refactor: rename region wal options to wal options

* chore: update usages of region wal options

* chore: add some comments to kafka

* chore: fill in kafka config

* test: add tests for serde wal config

* test: add tests for wal options

* refactor: refactor wal options allocator to enum

* refactor: store wal options into the request options instead

* fix: typo

* fix: typo

* refactor: move wal options map to region info

* refactor: refacto serialization and deserialization of wal options

* refactor: use serde_json to encode wal options

* chore: rename wal_options_map to region_wal_options

* chore: resolve some review comments

* fix: typo

* refactor: replace kecab-case with snake_case

* fix: sqlness and converage tests

* fix: typo

* fix: coverage test

* fix: coverage test

* chore: resolve some review conversations

* fix: resolve some review conversations

* chore: format comments in metasrv.example.toml

* chore: update import style

* feat: integrate wal options allocator to standalone mode

* test: add compatible test for OpenRegion

* test: add compatible test for UpdateRegionMetadata

* chore: remove based suffix from topic selector type
 2023-12-19 12:43:47 +00:00
LFC
c7b36779c1 fix: correctly decode mysql timestamp binary value (#2946) 
* fix: correctly decode mysql timestamp binary value

* fix: ut

* fix: resolve PR comments
 2023-12-19 11:05:28 +00:00
Wei
bbcac3a541 fix: auto create datatype_extension missing (#2953) 2023-12-19 10:53:31 +00:00
dennis zhuang
600cde1ff2 fix: wrong link for selector (#2958) 2023-12-19 10:50:48 +00:00
Zhenchi
83de399bef feat(inverted_index.create): add external sorter (#2950) 
* feat(inverted_index.create): add read/write for external intermediate files

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat(inverted_index.create): add external sorter

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: drop the stream as early as possible to avoid recursive calls to poll

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: project merge sorted stream

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add total_row_count to SortOutput

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: remove change of format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: rename segment null bitmap

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: test type alias

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: allow `memory_usage_threshold` to be None to turn off dumping

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: change segment_row_count type to NonZeroUsize

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: accept BytesRef instead

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: add `push_n` to adapt mito2

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: add k-way merge TODO

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: more sorter cases

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* refactor: make the merge tree balance

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update src/index/src/inverted_index/create/sort/external_sort.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: address comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: stable feature

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-12-19 08:14:37 +00:00
Ruihang Xia
6b8dbcfb54 chore: update toolchain to 20231219 (#2932) 
* update toolchain file, remove unused feature gates

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix format

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update action file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update to 12-19

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-12-19 07:24:08 +00:00
Wei
3e6a564f8e fix: blocking read timer lack of parameter (#2954) 2023-12-19 06:16:55 +00:00
Zhenchi
ccbd49777d fix: correct the error message for snappy compress (#2956) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-12-19 06:14:40 +00:00
Weny Xu
29fc2ea9d8 feat: fetch manifests in concurrent (#2951) 
* feat: fetch manifests in concurrent

* chore: set fetching manifest concurrency limit to 16
 2023-12-19 06:12:30 +00:00
WU Jingdi
d180e41230 fix: change range query time slot to [align_ts, align_ts + range) (#2938) 2023-12-19 02:35:14 +00:00
liyang
62d5fcbd76 refactor: greptimedb cluster sqlness test scripts (#2947) 2023-12-18 10:59:14 +00:00
Zhenchi
d339191e29 refactor: remove reduntant .compat() (#2949) 
* refactor: remove reduntant `.compat()`

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* remove dep

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-12-18 10:36:45 +00:00
Zhenchi
029ff2f1e3 feat(inverted_index.create): add read/write for external intermediate files (#2942) 
* feat(inverted_index.create): add read/write for external intermediate files

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: MAGIC_CODEC_V1 -> CODEC_V1_MAGIC

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: fix typos intermedia -> intermediate

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: typos

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: futures_code -> asynchronous_codec

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: bump bytes to 1.5

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-12-18 09:44:48 +00:00
Ruihang Xia
9af9c0229a feat: create table procedure for metric engine, part 1 (#2943) 
* implementation

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* initialize

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove empty file

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* apply review sugg

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-12-18 08:03:28 +00:00
WU Jingdi
4383a69876 fix!: remove range calendar as To option (#2940) 
fix: remove range calendar as `To` option
 2023-12-18 07:48:49 +00:00
LFC
033a065359 refactor: make sequence bounded with max value (#2937) 
* refactor: make sequence bounded with max value

(cherry picked from commit 3a8eba6f863327a96b617cd86ee2d39fac30abb2)

* fix: resolve PR comments
 2023-12-18 07:05:28 +00:00
dennis zhuang
262a79a170 feat: adds some tables to information_schema (#2935) 
* feat: adds engines table to information_schema

* feat: adds COLUMN_PRIVILEGES and COLUMN_STATISTICS

* feat: refactor memory tables

* chore: rename memory_tables

* test: adds unit tests

* chore: format

* chore: style

* fix: by cr comments

* refactor: tables
 2023-12-18 06:10:22 +00:00
Wei
5dc7ce1791 fix: typos and bit operations (#2944) 
* fix: typos and bit operation

* fix: helper

* chore: add tests in decimal128.rs and interval.rs

* chore: test

* chore: change proto version

* chore: clippy
 2023-12-18 03:06:11 +00:00
discord9
e35a494a3f test: fix a wronged test script (#2934) 2023-12-14 14:27:36 +00:00
shuiyisong
5dba373ede chore: return json body under http status 401 (#2924) 
* chore: change auth_fn to function and return response with json body

* chore: move unsupported to debug level

* chore: add docs and tests

* chore: rebase and update test
 2023-12-14 10:01:12 +00:00
Weny Xu
518bac35bc feat: add log and tracing layers for Copy From statement (#2929) 
feat: add log and tracing layers
 2023-12-14 06:15:30 +00:00
Ning Sun
39f80876cd feat: upgrade rustls library family, opensrv-mysql and pgwire (#2927) 
* feat: deps up

* fmt: toml format
 2023-12-14 05:56:39 +00:00
LFC
181e16a11a refactor: make instance started separately (#2911) 
* refactor: make instance started separately, to support further integrated into other binaries

* fix: resolve PR comments

* fix: resolve PR comments
 2023-12-14 03:44:29 +00:00
JeremyHi
99dda93f0e feat: sql with influxdb v1 result format (#2917) 
* feat: sql with influxdb v1 result format

* chore: add unit tests

* feat: minor refactor

* chore: by comment

* chore; u128 to u64 since serde can't deser u128 in enum

* chore: by comment

* chore: apply suggestion

* chore: revert suggestion

* chore: try again

---------

Co-authored-by: dennis zhuang <killme2008@gmail.com>
 2023-12-13 16:15:37 +00:00
ZonaHe
d3da128d66 feat: update dashboard to v0.4.3 (#2926) 
Co-authored-by: ZonaHex <ZonaHex@users.noreply.github.com>
 2023-12-13 11:12:17 +00:00