Commit Graph

5 Commits

Author SHA1 Message Date
Lei, HUANG
ef935a1de6 feat!: reduce sorted runs during compaction (#3702)
* feat: add functions to find and merge sorted runs

* chore: refactor code

* chore: remove some duplicates

* chore: remove one clone

* refactor: change max_active_window_files to max_active_window_runs

* feat: integrate with sorted runs

* fix: unit tests

* feat: limit num of sorted runs during compaction

* fix: some test

* fix: some cr comments

* feat: use smallvec

* chore: rebase main

* feat/reduce-sorted-runs:
 Refactor compaction logic and update test configurations

 - Refactored `merge_all_runs` function to use `sort_ranged_items` for sorting.
 - Improved item merging logic by iterating with `into_iter` and handling overlaps.
 - Updated test configurations to use `max_active_window_runs` instead of `max_active_window_files` for consistency.

---------

Co-authored-by: tison <wander4096@gmail.com>
2024-06-28 08:17:30 +00:00
tison
bba3108e0d refactor!: unify sql options into OptionMap (#3792)
* unify sql options into OptionMap

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* Update src/sql/src/util.rs

* drop legacy regions option

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

* fixup

Signed-off-by: tison <wander4096@gmail.com>

---------

Signed-off-by: tison <wander4096@gmail.com>
2024-04-25 04:06:05 +00:00
Yingwen
39b69f1e3b refactor!: Renames the new memtable to PartitionTreeMemtable (#3547)
* refactor: rename mod merge_tree to partition_tree

* refactor: rename merge_tree

* refactor: change merge tree comment

* refactor: rename merge tree struct

* refactor: memtable options
2024-03-20 06:40:41 +00:00
Yingwen
641592644d feat: support per table memtable options (#3524)
* feat: add memtable builder to region

* refactor: rename memtable_builder in worker to default_memtable_builder

* fix: return error instead of using default compaction options

Support deserializing memtable and compaction options from the option
map

* feat: optional memtable options

* feat: add MemtableBuilderProvider to create builders

* feat: change default memtable and skip deserializing dedup

* chore: update test and comment

* chore: test invalid type

* feat: metric engine use new memtable manually

* feat: expose more memtable configs

* feat: add memtable options to valid option list

* test: add test

* test: sqlness test

* chore: serde workspace

* chore: remove comments
2024-03-19 08:50:10 +00:00
Yingwen
9aa8f756ab fix: allow passing extra table options (#3484)
* fix: do not check options in parser

* test: fix tests

* test: fix sqlness

* test: add sqlness test

* chore: log options

* chore: must specify compaction type

* feat: validate option key

* feat: add option key validation back
2024-03-12 07:03:52 +00:00