rust/greptimedb
1
0
Fork 0
mirror of https://github.com/GreptimeTeam/greptimedb.git synced 2026-05-25 01:10:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat!: reduce sorted runs during compaction (#3702)

Browse Source 
* feat: add functions to find and merge sorted runs

* chore: refactor code

* chore: remove some duplicates

* chore: remove one clone

* refactor: change max_active_window_files to max_active_window_runs

* feat: integrate with sorted runs

* fix: unit tests

* feat: limit num of sorted runs during compaction

* fix: some test

* fix: some cr comments

* feat: use smallvec

* chore: rebase main

* feat/reduce-sorted-runs:
 Refactor compaction logic and update test configurations

 - Refactored `merge_all_runs` function to use `sort_ranged_items` for sorting.
 - Improved item merging logic by iterating with `into_iter` and handling overlaps.
 - Updated test configurations to use `max_active_window_runs` instead of `max_active_window_files` for consistency.

---------

Co-authored-by: tison <wander4096@gmail.com>
This commit is contained in:
Lei, HUANG
2024-06-28 16:17:30 +08:00
committed by GitHub
parent 352cc9ddde
commit ef935a1de6
12 changed files with 797 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
src/store-api/src/mito_engine_options.rs
View File

@@ -22,8 +22,8 @@ pub fn is_mito_engine_option_key(key: &str) -> bool {
[
"ttl",
"compaction.type",
"compaction.twcs.max_active_window_files",
"compaction.twcs.max_inactive_window_files",
"compaction.twcs.max_active_window_runs",
"compaction.twcs.max_inactive_window_runs",
"compaction.twcs.time_window",
"storage",
"index.inverted_index.ignore_column_ids",
@@ -47,10 +47,10 @@ mod tests {
assert!(is_mito_engine_option_key("ttl"));
assert!(is_mito_engine_option_key("compaction.type"));
assert!(is_mito_engine_option_key(
"compaction.twcs.max_active_window_files"
"compaction.twcs.max_active_window_runs"
));
assert!(is_mito_engine_option_key(
"compaction.twcs.max_inactive_window_files"
"compaction.twcs.max_inactive_window_runs"
));
assert!(is_mito_engine_option_key("compaction.twcs.time_window"));
assert!(is_mito_engine_option_key("storage"));