* refactor(meta-srv): move group rollback ownership to parent repartition procedure
- Parent procedure now owns partial rollback based on failed/unknown subprocedures
- rollback order: group metadata first, then allocated-region cleanup
- original_target_routes captured during build-plan, persisted in RepartitionPlanEntry
- rollback_group_metadata_routes moved to utils as parent-owned helper
- Group subprocedure no longer supports rollback (rollback_supported = false)
- Removed UpdateMetadata::RollbackStaging from group state machine
- Deleted redundant group rollback tests and helpers
BREAKING CHANGE: group Procedure no longer handles rollback; parent procedure
is responsible for crash recovery and selecting which plans to roll back.
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: update comments
Signed-off-by: WenyXu <wenymedia@gmail.com>
* chore: apply suggestions from CR
Signed-off-by: WenyXu <wenymedia@gmail.com>
---------
Signed-off-by: WenyXu <wenymedia@gmail.com>
refactor/remove-compactor-compact:
### Remove Unused Compaction Functionality
- **Removed `compact` Method**: Eliminated the `compact` method from the `Compactor` trait and its default implementation, which was primarily used for local compaction in testing. This change affects `compactor.rs`.
- **Code Cleanup**: Removed associated code and comments related to the `compact` method, streamlining the `Compactor` trait interface.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore: fix git cliff errors in latest version
- Fix errors in v2.12.0
- Do not generate logs for beta/rc tags between the compared commits
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: preserve blank line before release date in changelog
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor(mito2): improve compaction error handling and file removal
Refactor compaction task execution to enhance error handling and robustness.
- Implemented parallel execution of compaction tasks with proper error capture and logging for individual task failures.
- Ensured JoinSnafu is no longer directly used in error propagation, instead handling errors within the task processing loop.
- Adjusted file removal logic to correctly include expired SSTs after compaction merges.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor(mito2): extract SstMerger trait for testability in compaction
Extract SstMerger trait and DefaultSstMerger implementation to improve the testability of DefaultCompactor.
The DefaultCompactor is now generic over SstMerger, allowing mock implementations to be injected for unit testing without relying on the full object storage access layer. This refactoring separates the concerns of SST file merging from the overall compaction orchestration logic.
Additionally:
- Updated CompactionScheduler to use DefaultCompactor::default().
- Added unit tests for DefaultCompactor using a MockMerger.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix(compaction): propagate join error during sst flush
Correctly propagates the error when joining SST flush handles during compaction. Previously, the error was logged but not returned, leading to potential silent failures.
Also reorders some imports for consistency.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* perf(compaction): pre-allocate capacity for compacted_inputs
Pre-allocates capacity for the compacted_inputs vector based on the estimated total size of inputs and expired SSTs. This optimization aims to reduce vector reallocations during the compaction process.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/allow-partial-compaction:
### Commit Message
Enhance `DefaultCompactor` and `MockMerger` for Improved Flexibility
- **`compactor.rs`**:
- Added `Clone` trait to `DefaultSstMerger` and `MockMerger` to allow cloning.
- Removed `Arc` wrapping from `DefaultCompactor`'s `merger` field for direct usage.
- Updated `merge_ssts` method to require `Clone` trait for `SstMerger`.
- Modified `MockMerger` to use `Arc<Mutex>` for `results` and `call_idx` to ensure thread safety.
- Adjusted error handling to use `error::InvalidMetaSnafu` directly.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
fix/flaky-test:
### Add Dynamic Port Selection for Standalone Tests
- **`cli.rs`**: Implemented functions `random_standalone_addrs` and `choose_random_unused_port_offset` to dynamically select unused ports for standalone tests, enhancing test reliability.
- Updated `test_export_create_table_with_quoted_names` to use dynamically assigned ports for HTTP, RPC, MySQL, and PostgreSQL addresses.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix(index): intersect bitmaps before early exit in predicates applier
The loop skipped intersecting when the next bitmap was empty, which left
the accumulator unchanged instead of zeroing it. Intersect first, then
break when the result is empty.
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
* per gemini
* style(index): format predicates applier loop
* fix(index): remove unused mut in predicates applier
---------
Signed-off-by: Weixie Cui <cuiweixie@gmail.com>
Co-authored-by: discord9 <55937128+discord9@users.noreply.github.com>
Co-authored-by: discord9 <discord9@163.com>
* feat: add support for decimal parameter type, remove string replacement fallback
* chore: format
* fix: add support for using unsigned bigint in postgres
* chore: format toml
* refactor: cleanup duplicated code
* fix: rescale decimal
perf: move Tantivy fulltext search to blocking thread pool
Wrap the synchronous Tantivy search (query parsing, posting list
traversal, stored field reads) in spawn_blocking_global to avoid
starving the tokio async runtime with CPU-bound work.
Signed-off-by: lyang24 <lanqingy93@gmail.com>
* fix: add overflow check before interleave()
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: pass batches and column index to check_interleave_bytes_overflow
Refactor check_interleave_bytes_overflow to accept batches and a column
index directly, avoiding the intermediate Vec collection of arrays.
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: support alter from primary_key to flat
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: alter flat to primary_key
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: change default_experimental_flat_format to true
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: compute channel size from splitted batch size
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: add tests for split and channel size
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: always set sst_format from manifest on region open
sanitize_region_options did not set options.sst_format when the
default (PrimaryKey) matched the manifest value, leaving it as None
after reopen. This caused the alter format change to appear lost.
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: fix tests
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: show create table after alteration
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor!: rename default_experimental_flat_format to default_flat_format
The flat format is no longer experimental. Remove "experimental" from
the config field name, doc comments, and all references.
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: fix clippy
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
