mirror of
https://github.com/GreptimeTeam/greptimedb.git
synced 2026-05-17 05:20:37 +00:00
f6851cf8d7
* feat(mito2): extract and cache primary key range for SST files Extracts primary key ranges from SST files during flush and compaction, and caches them in FileHandle for future use (e.g., overlapping checks). Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/compaction-overlapping-check: - **Enhance Primary Key Range Logic**: Updated the `primary_key_ranges_overlap` function in `run.rs` to use `chunk()` for comparing byte ranges, improving accuracy in overlap detection. - **Refactor Run Assignment Logic**: Simplified the logic for assigning items to runs in `run.rs` by removing redundant match statements and using `is_empty()` and `iter().any()` for cleaner checks. - **Add Test for Transitivity Break**: Introduced a new test `test_find_sorted_runs_handles_2d_transitivity_break` in `run.rs` to ensure correct handling of transitivity breaks in sorted runs. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/compaction-overlapping-check: - **Remove PK-disjoint detection logic**: Simplified the compaction logic in `twcs.rs` by removing the `has_time_overlapping_pairs` function and related logic for PK-disjoint detection. This includes the removal of the `append_mode_force_compact` condition and associated tests. - **Update compaction trigger settings**: Modified `append_mode_test.rs` to set `compaction.twcs.trigger_file_num` to "2" and adjusted the expected number of files in the scanner assertion from 1 to 2. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * chore: rebase main Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/compaction-overlapping-check: ### Remove Unused Import in `compactor.rs` - Removed the unused import `compact_request` from `compactor.rs` to clean up the codebase. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * fix: tighten `mito2` file lifecycle handling Refine compaction, flush, and SST/version bookkeeping across `src/mito2/src/compaction/*`, `src/mito2/src/flush.rs`, `src/mito2/src/region/*`, `src/mito2/src/sst/*`, and related tests/utilities. * fix: reuse primary key range merge in twcs compaction Centralize primary key range merging so can call the shared helper from . Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * refactor(mito2): simplify `FileHandle` initialization and internalize primary key range extraction - Updated `FileHandle::new` to automatically compute the primary key range directly from `FileMeta`. - Restricted `FileHandle::new_with_primary_key_range` to be test-only by adding the `#[cfg(test)]` attribute. - Simplified `SstVersion::add_files` by adopting the updated `FileHandle::new` instead of manually providing the primary key range. Modified files: - `src/mito2/src/sst/file.rs` - `src/mito2/src/sst/version.rs` Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/compaction-overlapping-check: ### Improve File Management and Documentation - **`twcs.rs`**: Added a comment to clarify the merging of small files when there are no overlapping files. - **`version.rs` (in `region` and `sst`)**: Enhanced documentation for `add_files` method, explaining its functionality and panic conditions. Simplified the file handle creation logic in `sst/version.rs`. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * feat/compaction-overlapping-check: ### Enhance Primary Key Range Handling in `opener.rs` - Updated logic in `opener.rs` to set the primary key range for file handles when it is not already defined. This change ensures that the primary key range is extracted and set using `extract_primary_key_range` when necessary. Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> * docs: add docs for the necessity of checking pk and timesmaps while find overlapping files. * chore: address review comments Signed-off-by: Lei, HUANG <mrsatangel@gmail.com> --------- Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>