dependabot[bot]
97cbfcfe23
build(deps): bump mio from 0.8.10 to 0.8.11 ( #3434 )
...
Bumps [mio](https://github.com/tokio-rs/mio ) from 0.8.10 to 0.8.11.
- [Release notes](https://github.com/tokio-rs/mio/releases )
- [Changelog](https://github.com/tokio-rs/mio/blob/master/CHANGELOG.md )
- [Commits](https://github.com/tokio-rs/mio/compare/v0.8.10...v0.8.11 )
---
updated-dependencies:
- dependency-name: mio
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-05 11:04:14 +00:00
Lei, HUANG
7183fa198c
refactor: make MergeTreeMemtable the default choice ( #3430 )
...
* refactor: make MergeTreeMemtable the default choice
* refactor: reformat
* chore: add doc to config
2024-03-05 10:00:08 +00:00
Lei, HUANG
02b18fbca1
feat: decode prom requests to grpc ( #3425 )
...
* hack: inline decode
* move to servers
* fix: samples lost
* add bench
* remove useless functions
* wip
* feat: remove object pools
* fix: minor issues
* fix: remove useless dep
* chore: rebase main
* format
* finish
* fix: format
* feat: introduce request pool
* try to fix license issue
* fix: clippy
* resolve comments
* fix:typo
* remove useless comments
2024-03-05 09:47:32 +00:00
shuiyisong
7b1c3503d0
fix: complete interceptors for all frontend entry ( #3428 )
2024-03-05 09:38:47 +00:00
liyang
6fd2ff49d5
ci: refine windows output env ( #3431 )
2024-03-05 08:38:28 +00:00
WU Jingdi
53f2a5846c
feat: support tracing rule sampler ( #3405 )
...
* feat: support tracing rule sampler
* chore: simplify code
2024-03-05 15:40:02 +08:00
Yingwen
49157868f9
feat: Correct server metrics and add more metrics for scan ( #3426 )
...
* feat: drop timer on stream terminated
* refactor: combine metrics into a histogram vec
* refactor: frontend grpc metrics
* feat: add metrics middleware layer to grpc server
* refactor: move http metrics layer to metrics mod
* feat: bucket for grpc/http elapsed
* feat: remove duplicate metrics
* style: fix cilppy
* fix: incorrect bucket of promql series
* feat: more metrics for mito
* feat: convert cost
* test: fix metrics test
2024-03-04 10:15:10 +00:00
Ruihang Xia
ae2c18e1cf
docs(rfcs): multi-dimension partition rule ( #3350 )
...
* docs(rfcs): multi-dimension partition rule
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* change math block type
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update tracking issue
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update discussion
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix typo
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-03-04 08:10:54 +00:00
dennis zhuang
e6819412c5
refactor: show tables and show databases ( #3423 )
...
* refactor: show tables and show databases
* chore: clean code
2024-03-04 06:15:17 +00:00
tison
2a675e0794
docs: update pull_request_template.md ( #3421 )
2024-03-03 09:51:44 +00:00
JeremyHi
0edf1bbacc
feat: reduce a clone of string ( #3422 )
2024-03-03 08:09:17 +00:00
Eugene Tolbakov
8609977b52
feat: add verbose support for tql explain/analyze ( #3390 )
...
* feat: add verbose support for tql explain/analyze
* chore: apply clippy suggestions
* feat: add sqlness tests
* fix: adjust sqlness replace rules
* fix: address CR (move tql explain/analyze inside common folder)
* fix: address CR(improve comments to indicate that verbose is optional)
2024-03-02 11:18:22 +00:00
JeremyHi
2d975e4f22
feat: tableref cache ( #3420 )
...
* feat: tableref cache
* chore: minor refactor
* chore: avoid to string
* chore: change log level
* feat: add metrics for prometheus remote write decode
2024-03-02 07:37:31 +00:00
Kould
00cbbc97ae
feat: support Create Table ... Like ( #3372 )
...
* feat: support `Create Table ... Like`
* fix: `check_permission` for `Create Table ... Like`
* style: renaming `name` -> `table_name` & `target` -> `source_name` and make `Create Table ... Like` testcase more complicated
* rebase
* avoid _ fn
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
Co-authored-by: tison <wander4096@gmail.com >
2024-03-02 06:34:13 +00:00
niebayes
7d30c2484b
fix: mitigate memory spike during startup ( #3418 )
...
* fix: fix memory spike during startup
* fix: allocate a region write ctx for each wal entry
2024-03-01 07:46:05 +00:00
Lei, HUANG
376409b857
feat: employ sparse key encoding for shard lookup ( #3410 )
...
* feat: employ short key encoding for shard lookup
* fix: license
* chore: simplify code
* refactor: only enable sparse encoding to speed lookup on metric engine
* fix: names
2024-03-01 06:22:15 +00:00
Ning Sun
d4a54a085b
feat: add configuration for tls watch option ( #3395 )
...
* feat: add configuration for tls watch option
* test: sleep longer to ensure async task run
* test: update config api integration test
* refactor: rename function
2024-03-01 03:49:54 +00:00
dennis zhuang
c1a370649e
fix: show table names not complete from information_schema ( #3417 )
2024-03-01 02:51:46 +00:00
JeremyHi
3cad9d989d
fix: partition region id ( #3414 )
2024-02-29 09:09:59 +00:00
JohnsonLee
a50025269f
feat: Support automatic DNS lookup for kafka bootstrap servers ( #3379 )
...
* feat: Support automatic DNS lookup for kafka bootstrap servers
* Revert "feat: Support automatic DNS lookup for kafka bootstrap servers"
This reverts commit 5baed7b01dniebayes@gmail.com >
* move more common code
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
Co-authored-by: tison <wander4096@gmail.com >
Co-authored-by: niebayes <niebayes@gmail.com >
2024-02-29 07:29:20 +00:00
JeremyHi
a3533c4ea0
feat: zero copy on split rows ( #3407 )
2024-02-28 13:27:52 +00:00
Lei, HUANG
3413fc0781
refactor: move some costly methods in DataBuffer::read out of read lock ( #3406 )
...
* refactor: move some costly methods in DataBuffer::read out of read lock
* refactor: also replace ShardReader with ShardReaderBuilder
2024-02-28 12:22:44 +00:00
tison
dc205a2c5d
feat: enable ArrowFlight compression ( #3403 )
...
* feat: enable ArrowFlight compression
Signed-off-by: tison <wander4096@gmail.com >
* turn on features
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-02-28 08:55:44 +00:00
Lei, HUANG
a0a8e8c587
fix: some read metrics ( #3404 )
...
* fix: some read metrics
* chore: fix some metrics
* fix
2024-02-28 08:47:49 +00:00
Zhenchi
c3c80b92c8
feat(index): measure memory usage in global instead of single-column and add metrics ( #3383 )
...
* feat(index): measure memory usage in global instead of single-column and add metrics
* feat: add leading zeros to streamline memory usage
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: fmt
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: remove println
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
---------
Signed-off-by: Zhenchi <zhongzc_arch@outlook.com >
2024-02-28 06:49:24 +00:00
Weny Xu
a8cbec824c
refactor: refactor TableRouteManager ( #3392 )
...
* feat: introduce TableRouteStorage
* refactor: remove get & batch_get in TableRouteManager
* refactor: move txn related fn to TableRouteStorage
* chore: apply suggestions from CR
* chore(codecov): ingore tests-integration dir
2024-02-28 06:18:09 +00:00
tison
33d894c1f0
build: do not retry for connrefused ( #3402 )
...
* build: do not retry for connrefused
Signed-off-by: tison <wander4096@gmail.com >
* simplify layout
Signed-off-by: tison <wander4096@gmail.com >
---------
Signed-off-by: tison <wander4096@gmail.com >
2024-02-28 06:15:23 +00:00
Lei, HUANG
7942b8fae9
chore: add metris for memtable read path ( #3397 )
...
* chore: add metris for read path
* chore: add more metrics
2024-02-28 03:37:19 +00:00
Yingwen
b97f957489
feat: Use a partition level map to look up pk index ( #3400 )
...
* feat: partition level map
* test: test shard and builder
* fix: do not use pk index from shard builder
* feat: add multi key test
* fix: freeze shard before finding pk in shards
2024-02-28 03:17:09 +00:00
tison
f3d69e9563
chore: retry fetch dashboard assets ( #3394 )
...
Signed-off-by: tison <wander4096@gmail.com >
2024-02-27 10:07:21 +00:00
dennis zhuang
4b36c285f1
feat: flush or compact table and region functions ( #3363 )
...
* feat: adds Requester to process table flush and compaction request
* feat: admin_fn macros for administration functions
* test: add query result
* feat: impl flush_region, flush_table, compact_region, and flush_region functions
* docs: add Arguments to admin_fn macro
* chore: apply suggestion
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* chore: apply suggestion
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
* fix: group_requests_by_peer and adds log
* Update src/common/macro/src/admin_fn.rs
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
* feat: adds todo for spawan thread
* feat: rebase with main
---------
Co-authored-by: Zhenchi <zhongzc_arch@outlook.com >
Co-authored-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-27 08:57:38 +00:00
discord9
dbb1ce1a9b
feat(flow): impl for MapFilterProject ( #3359 )
...
* feat: mfp impls
* fix: after rebase
* test: temporal filter mfp
* refactor: more comments&test
* test: permute
* fix: check input len when eval
* refactor: err handle&docs: more explain graph
* docs: better flowchart map,filter,project
* refactor: visit_* falliable
* chore: better temp lint allow
* fix: permute partially
* chore: remove duplicated checks
* docs: more explain&tests for clarity
* refactor: use ensure! instead
2024-02-27 08:13:55 +00:00
Ruihang Xia
3544c9334c
feat!: new partition grammar - parser part ( #3347 )
...
* parser part
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* fix test in sql
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* comment out and ignore some logic
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update sqlness cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* temporary disable region migration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* allow dead code
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* update integration test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-27 07:20:16 +00:00
Lei, HUANG
492a00969d
feat: enable zstd compression and encodings in merge tree data part ( #3380 )
...
* feat: enable zstd compression in merge tree data part to save memory
* feat: also enable customized column encoding in DataPartEncoder
2024-02-27 06:54:56 +00:00
Yingwen
206666bff6
feat: Implement partition eviction and only add value size to write buffer size ( #3393 )
...
* feat: track key bytes in dict
* chore: done allocating on finish
* feat: evict keys
* chore: do not add to write buffer
* chore: only count value bytes
* fix: reset key bytes
* feat: remove write buffer manager from shards
* feat: change dict size compute method
* chore: adjust dictionary size by os memory
2024-02-27 06:28:57 +00:00
Weny Xu
7453d9779d
fix: throw errors instead of panic ( #3391 )
...
* fix: throw errors instead of panic
* chore: apply suggestions from CR
2024-02-27 03:46:12 +00:00
liyang
8e3e0fd528
ci: add builder result outputs in release action ( #3381 )
2024-02-27 03:43:16 +00:00
dimbtp
b1e290f959
fix: range fix in modulo function tests ( #3389 )
...
fix: range fix for modulo tests
2024-02-26 15:50:23 +00:00
Ruihang Xia
d8dc93fccc
feat(grafana): enable shared tooltip, add raft engine throughput ( #3387 )
...
feat: enable shared tooltip, add raft engine throughput
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-26 11:31:15 +00:00
Ning Sun
3887d207b6
feat: make tls certificates/keys reloadable (part 1) ( #3335 )
...
* feat: make tls certificates/keys reloadable (part 1)
* feat: add notify watcher for cert/key files
* test: add unit test for watcher
* fix: correct usage of watcher
* fix: skip watch when tls disabled
2024-02-26 09:37:54 +00:00
Ruihang Xia
e859f0e67d
chore: skip reorder workspace tables in taplo ( #3388 )
...
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-26 08:57:49 +00:00
Ruihang Xia
ce397ebcc6
feat: change how region id maps to region worker ( #3384 )
...
* feat: change how region id maps to region worker
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
* add overflow test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com >
2024-02-26 08:42:29 +00:00
Yingwen
26011ed0b6
fix: resets dict builder keys counter and avoid unnecessary pruning ( #3386 )
...
* fix: dict builder resets num_keys on finish
* feat: skip empty shard and builder
* feat: avoid pruning if possible
Implementations:
- Apply all filters on the partition column
- If no filter to prune, skip decoding keys
2024-02-26 08:24:46 +00:00
Lei, HUANG
8087822ab2
refactor: change the receivers of merge tree components ( #3378 )
...
* refactor: change the receivers of Shard::read/DataBuffer::read/DataParts::read to &self instead of &mut self
* refactor: remove allow(dead_code) in merge tree
2024-02-26 06:50:55 +00:00
Yingwen
e481f073f5
feat: Implement dedup for the new memtable and expose the config ( #3377 )
...
* fix: KeyValues num_fields() is incorrect
* chore: fix warnings
* feat: support dedup
* feat: allow using the new memtable
* feat: serde default for config
* fix: resets pk index after finishing a dict
2024-02-25 13:06:01 +00:00
Lei, HUANG
606309f49a
fix: remove unused imports in memtable_util.rs ( #3376 )
2024-02-25 09:23:28 +00:00
Yingwen
8059b95e37
feat: Implement iter for the new memtable ( #3373 )
...
* chore: read shard builder
* chore: reuse pk weights
* chore: prune key
* chore: shard reader wip
* refactor: shard builder DataBatch
* feat: merge shard readers
* feat: return shard id in shard readers
* feat: impl partition reader
* chore: impl partition read
* feat: impl iter tree
* chore: save last yield pk id
* style: fix clippy
* refactor: rename ShardReaderImpl to ShardReader
* chore: address CR comment
2024-02-25 07:42:16 +00:00
Lei, HUANG
afe4633320
feat: merge tree dedup reader ( #3375 )
...
* feat: add dedup option to merge tree component
* feat: impl dedup reader for shard reader
* refactor: DedupReader::new to DedupReader::try_new
* refactor: remove DedupReader::current_key field
* fix: some cr comments
* fix: fmt
* fix: remove shard_id method from DedupSource
2024-02-24 13:50:49 +00:00
Yingwen
abbfd23d4b
feat: Add freeze and fork method to the memtable ( #3374 )
...
* feat: add fork method to the memtable
* feat: allow mark immutable returns result
* feat: use fork to create the mutable memtable
* feat: remove memtable builder from freeze
* chore: warninigs
* fix: inspect error
* feat: iter returns result
* chore: maintains memtable id in region
* chore: update comment
* fix: remove region status if failed to freeze a memtable
* chroe: update comment
* chore: iter should not require sync
* chore: implement freeze and fork for the new memtable
2024-02-24 12:11:16 +00:00
Yingwen
1df64f294b
refactor: Remove Item from merger's Node trait ( #3371 )
...
* refactor: data reader returns reference to data batch
* refactor: use range to create merger
* chore: Reference RecordBatch in DataBatch
* fix: top node not read if no next node
* refactor: move timestamp_array_to_i64_slice to data mod
* style: fix cilppy
* chore: derive copy for DataBatch
* chore: address CR comments
2024-02-24 07:19:48 +00:00
