rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-01-14 01:02:55 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,879 Commits 105 Branches 227 Tags
ff8ab6763bfe8fc647ab0bd4cbafcc4bfbd52281
Commit Graph

1564 Commits

Author SHA1 Message Date
tison
ff8ab6763b chore: internal ChrootKvBackend refactor and test (#2799) 
* try avoid rate limit

Signed-off-by: tison <wander4096@gmail.com>

* chroot utilities as method

Signed-off-by: tison <wander4096@gmail.com>

* add test

Signed-off-by: tison <wander4096@gmail.com>

* make clippy happy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2023-11-24 06:45:09 +00:00
Yingwen
00e4bd45f0 feat: add put_only field to skip filtering deletion (#2801) 
* feat: add put_only field to skip filtering deletion

* docs: fix typo
 2023-11-24 06:33:17 +00:00
Ning Sun
85eebcb16f fix: correct mysql timestamp encoding for binary protocol (#2797) 
* fix: correct mysql timestamp encoding for binary protocol

* test: add tests for mysql timestamp encoding
 2023-11-23 18:54:59 +00:00
tison
102e43aace test: use EtcdStore in IT cases (#2734) 
* test: use EtcdStore in IT cases

Signed-off-by: tison <wander4096@gmail.com>

* retrigger CI

Signed-off-by: tison <wander4096@gmail.com>

* refactor: KvPair can take etcd KeyValue

Signed-off-by: tison <wander4096@gmail.com>

* temporary use fork

Signed-off-by: tison <wander4096@gmail.com>

* drop cloned

Signed-off-by: tison <wander4096@gmail.com>

* chroot_key_value

Signed-off-by: tison <wander4096@gmail.com>

* chroot and prepend in each point

Signed-off-by: tison <wander4096@gmail.com>

* adjust call points

Signed-off-by: tison <wander4096@gmail.com>

* cargo clippy

Signed-off-by: tison <wander4096@gmail.com>

* point to upstream etcd-client

Signed-off-by: tison <wander4096@gmail.com>

* test etcd chroot

Signed-off-by: tison <wander4096@gmail.com>

* add NO_CHROOT constant

Signed-off-by: tison <wander4096@gmail.com>

* check

Signed-off-by: tison <wander4096@gmail.com>

* handle range end

Signed-off-by: tison <wander4096@gmail.com>

* handle special encoded key or range_end

Signed-off-by: tison <wander4096@gmail.com>

* fixup implementation

Signed-off-by: tison <wander4096@gmail.com>

* clippy

Signed-off-by: tison <wander4096@gmail.com>

* avoid test name conflict

Signed-off-by: tison <wander4096@gmail.com>

* chroot to kvbackend level

Signed-off-by: tison <wander4096@gmail.com>

* fixup all occurances

Signed-off-by: tison <wander4096@gmail.com>

* fix type

Signed-off-by: tison <wander4096@gmail.com>

* Update src/common/meta/src/kv_backend/txn.rs

* make github happy

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: LFC <990479+MichaelScofield@users.noreply.github.com>
 2023-11-23 05:47:00 +00:00
Ruihang Xia
56fc77e573 fix: add missing error display message (#2791) 
* fix: add missing error display message

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* update sqlness result

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-23 02:59:49 +00:00
Weny Xu
4c76d4d97e feat: add update metadata step for downgrading leader region (#2771) 2023-11-21 12:01:28 +00:00
Zhenchi
9e5cdf47d9 chore(puffin): re-add tests file (#2790) 
* chore(puffin): re-add tests file

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-11-21 11:02:18 +00:00
Zhenchi
bdb677dc52 chore(puffin): remove tests (#2789) 
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-11-21 10:02:34 +00:00
Ruihang Xia
99dbb7401c refactor: remove sequence number from ScanRequest (#2785) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-21 07:38:30 +00:00
Zhenchi
a7bbd61f28 feat(puffin): add file writer (#2776) 
* feat(puffin): add file writer

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update src/puffin/src/file_format/writer/file.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* Update src/puffin/src/file_format/writer/file.rs

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* feat: footer bytes with capacity

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: footer bytes with capacity

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update src/puffin/src/file_format/writer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* feat: add flush

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: specify default flags

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* feat: close async writer

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-11-21 06:35:48 +00:00
tison
efc5abfc02 build: upgrade etcd-client to 0.12.2 (#2781) 
* build: upgrade etcd-client to 0.12.2

Signed-off-by: tison <wander4096@gmail.com>

* upgrade nightly toolchain

Signed-off-by: tison <wander4096@gmail.com>

* chore: run clippy

Signed-off-by: tison <wander4096@gmail.com>

* Update Cargo.toml

Co-authored-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
Co-authored-by: Ning Sun <classicning@gmail.com>
 2023-11-21 06:33:41 +00:00
Weny Xu
43a7457e15 fix: bring back inactive_region_ids (#2783) 2023-11-21 06:24:33 +00:00
Ruihang Xia
20f01219e9 refactor: adjust metric engine structure (#2773) 
* extract consts into a separate mod

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add documents

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split state

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split create

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split alter

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* split put

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix metadata clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix engine clippy

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-21 05:21:22 +00:00
JeremyHi
dc351a6de9 feat: heartbeat handler control (#2780) 2023-11-21 02:48:11 +00:00
Yingwen
b9146c88ff refactor: Remove usages of the old storage crate (#2777) 
* chore: remove storage from some crate

* feat: remove storage config

* feat: remove storage from cmd

* feat: impl stream_to_parquet

* feat: remove storage from operator

* feat: remove stream writer from mito2

* feat: remove storage from project toml

* test: fix config api test

* docs: remove outdated configs

* refactor: remove storage directory
 2023-11-20 20:29:41 +08:00
Bruce Chen
9558b3c201 build(cmd): upgrade clap to 4.x (#2775) 
* build(cmd): upgrade clap to 4.4.8

* build(cmd): upgrade clap to 4.4
 2023-11-20 10:43:31 +00:00
fys
da68d8ce4b feat: add random weighted choose in load_based selector (#2234) 
* feat: add random weigted choose in load_based selector

* fix: meta cannot save heartbeats when cluster have no region

* chore: print some log

* chore: remove unused code

* cr

* add some logs when filter result is empty
 2023-11-20 06:47:42 +00:00
Wei
01867adaa7 feat: unit test for mutable vector (#2768) 
feat: test for mutable trait datatype method
 2023-11-20 06:20:13 +00:00
Zhenchi
d9eeeee06e feat(puffin): add file reader (#2751) 
* feat(puffin): add file reader

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: toml format

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: rename PuffinParser to PuffinFileReader

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* chore: polish comments

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* Update src/puffin/src/file_format/reader/footer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/puffin/src/file_format/reader/file.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/puffin/src/file_format/reader/footer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* Update src/puffin/src/file_format/reader/footer.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

* fix: check file size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: redundant type cast

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: reuse read buffer

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: check payload size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: check payload size

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: validate blob offset

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: validate blob offset

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-11-20 04:29:41 +00:00
Weny Xu
4fcda272fb feat: add open candidate region step (#2757) 
* feat: add open candidate region state

* feat: register the opening region

* chore: apply suggestions from CR
 2023-11-20 03:36:00 +00:00
Yingwen
ce959ddd3f feat(mito): implements row group level page cache (#2688) 
* feat: add page cache

* docs: update mito config toml

* feat: impl CachedPageReader

* feat: use cache reader to read row group

* feat: do not fetch data if we have pages in cache

* chore: return if nothing to fetch

* feat: enlarge page cache size

* test: test write read parquet

* test: test cache

* docs: update comments

* test: fix config api test

* feat: cache metrics

* feat: change default page cache size

* test: fix config api test
 2023-11-20 02:55:50 +00:00
Weny Xu
730a3faa02 fix: fix exits typos (#2772) 2023-11-20 02:07:01 +00:00
WU Jingdi
91820a8006 fix: empty by in range query (#2770) 
* fix: empty by in range query

* Apply suggestions from code review

---------

Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-17 11:18:14 +00:00
WU Jingdi
500e299e40 feat: Enable distributed tracing in greptimedb (#2755) 
* feat: implement distributed tracing

* fix: change usage of span

* fix: use otlp as exporter

* chore: update dependence

* chore: add span info

* chore: add alias

* chore: use instrument instead of trace
 2023-11-17 08:51:57 +00:00
Ruihang Xia
ac4b6cd7f0 feat: write logical region to metric engine (#2759) 
* transform write request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add tests for put request

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* use table_id instead of metric_name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix typo

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* CR sugg.

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* define random state as const

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-17 07:44:11 +00:00
Weny Xu
3ab494764f feat: add migration start step (#2756) 
* feat: add migration start state

* refactor: move PersistentContext and VolatileContext into Context

* chore: apply suggestions from CR
 2023-11-17 07:05:04 +00:00
Lanqing Yang
5608035074 fix!: improve user experience on setting compression type (#2765) 
* fixes: https://github.com/GreptimeTeam/greptimedb/issues/2758
Chore: improve user experience on setting compression type

* Update src/common/datasource/src/compression.rs

Co-authored-by: Yingwen <realevenyag@gmail.com>

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-11-17 06:27:13 +00:00
Yingwen
06327fba1e feat: improve http error message (#2767) 
* feat: improve http error message

* feat: log http error

* style: fix clippy

* test: fix test
 2023-11-17 04:03:30 +00:00
Yingwen
06da33b1ed fix: mysql writer does not print error message in some cases (#2763) 
* fix: mysql writer does not print error in some cases

* feat: http error msg
 2023-11-17 03:07:54 +00:00
Yingwen
2aa6ac5731 fix: correct memtable cost (#2762) 2023-11-17 02:58:51 +00:00
Weny Xu
b28af9443b feat: grant lease to opening regions (#2752) 
* feat: add OpeningRegionKeeper

* feat: grant lease to opening regions
 2023-11-17 02:19:50 +00:00
Yingwen
142035340d fix: compiler errors under non-default features (#2761) 
* feat: bump prost and fix pprof feature compiler errors

* feat: fix compiler errors on tokio-console

* chore: fix compiler errors

* ci: add all features check to ci
 2023-11-16 19:43:44 +00:00
Wei
d2cf72e0f1 fix: iterator bad implement (#2760) 
* fix: iter_data() doesn't stop

* chore: Update src/datatypes/src/vectors/decimal.rs

---------

Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-11-16 12:26:05 +00:00
Wei
ae27fbc7f2 feat: impl Vector, DataType traits for decimal128 (#2727) 
* feat: decimal128 vector

* feat: Decimal128Type in datatypes

* refactor: cast function

* feat: decimal128 type in ConcreteDataType

* chore: decimal

* chore: sql_data_type_to_concrete_data_type

* chore: cr comment

Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>

* chore: fix finish()

* chore: add debug_assert to new decimal128

---------

Co-authored-by: Dennis Zhuang <killme2008@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
 2023-11-16 06:44:26 +00:00
Weny Xu
9bd10134dd feat: add region migration procedure skeleton (#2743) 
* feat: add region migration procedure skeleton

* chore: apply suggestions from CR

* chore: apply suggestions from CR

* refactor: unify the lock key
 2023-11-15 09:07:23 +00:00
Zhenchi
3329da5b72 feat(puffin): add partial reader (#2741) 
* feat(puffin): add partial reader

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* address comment

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-11-15 06:28:20 +00:00
Lei, HUANG
a24f8c96b3 perf: disable dictionary of ts columns (#2749) 
* perf: disable dictionary of ts columns

* fix: unit test
 2023-11-15 06:14:28 +00:00
Ruihang Xia
a691cff0c4 feat: alter logical region in metric region (#2726) 
* add test for add and alter logical region

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove table id

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>

* CR sugg.

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* extract internal states into a struct

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* remove unused deps

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* Apply suggestions from code review

Co-authored-by: dennis zhuang <killme2008@gmail.com>

* tweak fn name

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
Co-authored-by: JeremyHi <jiachun_feng@proton.me>
 2023-11-15 03:52:52 +00:00
Yingwen
f92b55c745 feat: Ignore file not found error temporarily while compacting files (#2745) 
* feat: support ignoring file not found error

* feat: ignore not found during compaction
 2023-11-15 03:48:51 +00:00
Ruihang Xia
5b978269cc feat: simplify log file output by removing bunyan layer (#2747) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-14 14:18:34 +00:00
Ruihang Xia
e2a770f8de fix: handle if num_cpus / 2 == 0 (#2740) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-14 06:44:10 +00:00
Weny Xu
dc46e96879 refactor: replace InactiveRegionManager with RegionLeaseKeeper (#2729) 
* refactor: replace InactiveRegionManager with RegionLeaseKeeper

* refactor: remove InactiveRegionManager

* chore: remove remark_inactive_region field

* chore: apply suggestions from CR

* refactor: reduce meta server cache scope

* chore: remove redundant code
 2023-11-14 05:16:13 +00:00
Ruihang Xia
8f3b299a45 fix: Normalize default constrain fn name (#2737) 
* fix: normalize current_timestamp to current_timestamp()

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* add sqlness case

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix clippy lints

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

---------

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-14 03:22:23 +00:00
Ruihang Xia
506e6887f3 chore: add rerun-if-changed to dashboard feature (#2728) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-13 18:20:59 +00:00
Zhenchi
1757061272 feat: introduce metadata struct of puffin (#2725) 
* feat: introduce metadata struct of puffin

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: fmt

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

* fix: address suggestions

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>

---------

Signed-off-by: Zhenchi <zhongzc_arch@outlook.com>
 2023-11-13 10:02:40 +00:00
Ruihang Xia
6599bb5a46 feat: do not require worker's number to be power of 2 (#2732) 
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
 2023-11-13 09:04:44 +00:00
tison
f5eede4ce1 feat: support prometheus format_query endpoint (#2731) 
* feat: support prometheus format_query endpoint

Signed-off-by: tison <wander4096@gmail.com>

* add test

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2023-11-10 10:33:28 +00:00
Weny Xu
22ee45f3df feat: downgrade failed region before closing (#2715) 
* feat: add update_region_status method

* feat: downgrade failed region before closing

* chore: apply suggestions from CR
 2023-11-10 05:05:05 +00:00
tison
8fd0766754 fix: store raw objname internally (#2719) 
* fix: store raw objname internally

Signed-off-by: tison <wander4096@gmail.com>

* add a utility

Signed-off-by: tison <wander4096@gmail.com>

* add a sqlness test case

Signed-off-by: tison <wander4096@gmail.com>

* cargo clippy

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2023-11-10 02:56:30 +00:00
tison
af7107565a refactor: use humantime for user-facing options (#2724) 
* refactor: use humantime for user-facing options

Signed-off-by: tison <wander4096@gmail.com>

* unify quote style

Signed-off-by: tison <wander4096@gmail.com>

* taplo fmt

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
 2023-11-09 13:29:47 +00:00