mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-05-25 09:20:40 +00:00
feat(mito): limit compaction output file size (#4754)
* Commit Message Clarify documentation for CompactionOutput struct Updated the documentation for the `CompactionOutput` struct to specify that the output time range is only relevant for windowed compaction. * Add max_output_file_size to TwcsPicker and TwcsOptions - Introduced `max_output_file_size` to `TwcsPicker` struct and its logic to enforce output file size limits during compaction. - Updated `TwcsOptions` to include `max_output_file_size` and adjusted related tests. - Modified `new_picker` function to initialize `TwcsPicker` with the new `max_output_file_size` field. * feat/limit-compaction-output-size: Refactor compaction picker and TWCS to support append mode and improve options handling - Update compaction picker to accept a reference to options and append mode flag - Modify TWCS picker logic to consider append mode when filtering deleted rows - Remove VersionControl usage in compactor and simplify return type - Adjust enforce_max_output_size logic in TWCS picker to handle max output file size - Add append mode flag to TwcsPicker struct - Fix incorrect condition in TWCS picker for enforcing max output size - Update region options tests to reflect new max output file size format (1GB and 7MB) - Simplify InvalidTableOptionSnafu error handling in create_parser - Add `compaction.twcs.max_output_file_size` to mito engine option keys * resolve some comments
This commit is contained in:
@@ -370,12 +370,7 @@ impl<'a> ParserContext<'a> {
|
||||
.map(parse_option_string)
|
||||
.collect::<Result<HashMap<String, String>>>()?;
|
||||
for key in options.keys() {
|
||||
ensure!(
|
||||
validate_table_option(key),
|
||||
InvalidTableOptionSnafu {
|
||||
key: key.to_string()
|
||||
}
|
||||
);
|
||||
ensure!(validate_table_option(key), InvalidTableOptionSnafu { key });
|
||||
}
|
||||
Ok(options.into())
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user