refactor: extract the common codes of creating proto ColumnSchema and Row to helper functions
fix: explicitly set the follower max sequence when finding extension ranges to avoid potential concurrency hazard
Signed-off-by: luofucong <luofc@foxmail.com>
* perf: cached reader do not get page concurrently
Otherwise they will all fetch the same pages in parallel
Signed-off-by: evenyag <realevenyag@gmail.com>
* perf: always disable zstd for bloom
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: not mark all deleted when partial trunc¬ update manifest when partial file range is empty
Signed-off-by: discord9 <discord9@163.com>
* docs: note
Signed-off-by: discord9 <discord9@163.com>
---------
Signed-off-by: discord9 <discord9@163.com>
fix/compaction-concurrency:
Add delay before compaction in `compaction_test.rs`
- Introduced a 2-millisecond delay using `tokio::time::sleep` before the `compact` function call in `test_compaction_region_with_overlapping_delete_all` to ensure proper timing and synchronization during the test execution.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
**Enhance Memtable Iteration and Flushing Logic**
- **`flush.rs`**: Updated `RegionFlushTask` to handle multiple ranges using `MergeReaderBuilder` for improved source management during flush operations.
- **`memtable.rs`**: Introduced `build_prune_iter` and `build_iter` methods in `MemtableRange` for flexible iteration. Added `MemtableRanges` struct to manage multiple contexts.
- **`simple_bulk_memtable.rs`**: Refactored to use `BatchIterBuilder` and `BatchIterBuilderDeprecated` for iteration, supporting new `read_to_values` method in `Series`.
- **`time_series.rs`**: Added `read_to_values` and `finish_cloned` methods in `Series` and `ValueBuilder` for efficient data handling.
- **`scan_util.rs`**: Replaced `build_iter` with `build_prune_iter` for range iteration, enhancing scan utility.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
- **Add Rayon for Parallel Processing**: Introduced `rayon` for parallel processing in `simple_bulk_memtable.rs` and updated `Cargo.toml` and `Cargo.lock` to include `rayon` dependency.
- **Enhance Benchmarking**: Added new benchmarks in `simple_bulk_memtable.rs` to compare parallel vs sequential processing, projection, sequence filtering, and write performance.
- **Make Structs and Methods Public**: Changed visibility of several structs and methods to `pub` in `simple_bulk_memtable.rs`, `memtable.rs`, `time_series.rs`, and `test_util.rs` to facilitate testing and benchmarking.
- **Update Criterion Features**: Modified `Cargo.toml` to include `html_reports` feature for `criterion`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
### Commit Summary
- **Refactor `SimpleBulkMemtable`**:
- Moved `ranges_sequential` function to a new `test_only` module and made it a method of `SimpleBulkMemtable`.
- Made several fields in `SimpleBulkMemtable` private and added a `region_metadata` getter.
- Affected files: `simple_bulk_memtable.rs`, `test_only.rs`.
- **Benchmark Adjustments**:
- Updated benchmark functions to use the new `ranges_sequential` method.
- Affected file: `simple_bulk_memtable.rs`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
### Add Test Configuration for `iter` Method in Memtable Implementations
- **Enhancements**:
- Added `#[cfg(any(test, feature = "test"))]` attribute to the `iter` method in various `Memtable` implementations to enable conditional compilation for testing purposes.
- Affected files:
- `src/mito2/src/memtable.rs`
- `src/mito2/src/memtable/bulk.rs`
- `src/mito2/src/memtable/partition_tree.rs`
- `src/mito2/src/memtable/simple_bulk_memtable.rs`
- `src/mito2/src/memtable/time_series.rs`
- `src/mito2/src/test_util/memtable_util.rs`
- **Benchmark Adjustments**:
- Removed `black_box` usage in `bench_memtable_write_performance` function to streamline benchmarking.
- Affected file: `src/mito2/benches/simple_bulk_memtable.rs`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
**Enhance Async Support and Refactor Iteration in `mito2`**
- **Add Async Features**: Updated `Cargo.toml` to include `async` and `async_tokio` features for `criterion`.
- **Async Iteration**: Introduced async functions `flush` and `flush_original` in `simple_bulk_memtable.rs` to handle memtable flushing using async iterators.
- **Refactor Iteration Logic**: Moved `create_iter` and `BatchIterBuilderDeprecated` to `test_only.rs` for better separation of concerns.
- **Public API Change**: Made `next_batch` in `read.rs` public to support async batch processing.
- **Benchmark Updates**: Modified benchmarks in `simple_bulk_memtable.rs` to use async runtime for performance testing.
Files affected: `Cargo.toml`, `simple_bulk_memtable.rs`, `test_only.rs`, `read.rs`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
**Enhance Benchmarking for Memtable**
- Refactored `create_large_memtable` to `create_memtable_with_rows` in `simple_bulk_memtable.rs` to allow dynamic row count configuration.
- Introduced parameterized benchmarking in `bench_ranges_parallel_vs_sequential` to test various row counts, improving the flexibility and coverage of performance tests.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
### Enhance Memory Management and Public API
- **`builder.rs`**: Made `next_offset` method public to allow external access to offset calculations.
- **`simple_bulk_memtable.rs`**: Simplified the `series.extend` method by removing the iterator conversion for `fields`.
- **`time_series.rs`**:
- Added `can_accommodate` method to `ValueBuilder` to check if fields can be accommodated without offset overflow.
- Modified `extend` method to use a `Vec` for `fields` instead of an iterator, improving memory management and error handling.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
Add License and Enhance Testing in `simple_bulk_memtable.rs`
- Added Apache License header to `simple_bulk_memtable.rs`.
- Modified test configuration in `simple_bulk_memtable.rs` to include `any(test, feature = "test")`.
- Introduced a new test `test_write_read_large_string` in `simple_bulk_memtable.rs` to verify handling of large strings.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
Update `Cargo.toml` dependencies
- Adjust features for `common-meta` and `mito-codec` to include "testing".
- Maintain `criterion` version and features for async support.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
### Update Predicate Type in Memtable Iterators
- **Files Modified**:
- `src/mito2/src/memtable.rs`
- `src/mito2/src/memtable/bulk.rs`
- `src/mito2/src/memtable/simple_bulk_memtable.rs`
- **Key Changes**:
- Updated the `iter` method in `Memtable` trait and its implementations to use `Option<table::predicate::Predicate>` instead of `Option<Predicate>`.
- Adjusted return type in `BulkMemtable`'s `iter` method to `Result<crate::memtable::BoxedBatchIterator>`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
**Enhance Memtable Functionality**
- **`memtable.rs`**:
- Added `Clone` trait to `MemtableStats` and made `num_ranges` public.
- Introduced `num_rows` field in `MemtableRange` and updated its constructor.
- Added `num_rows` method to `MemtableRange`.
- **`partition_tree.rs`, `simple_bulk_memtable.rs`, `time_series.rs`**:
- Updated `MemtableRange` instantiation to include `num_rows`.
- **`range.rs`**:
- Refactored `MemRangeBuilder` to handle a single `MemtableRange` and `MemtableStats`.
- **`scan_region.rs`**:
- Enhanced memtable filtering based on time range and updated `MemRangeBuilder` usage.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
**Enhancements and Bug Fixes**
- **Deduplication Enhancements**:
- Introduced `DedupReader` and `LastRow` as public structs in `dedup.rs` to enhance deduplication capabilities.
- Added `LastNonNull` deduplication strategy in `flush.rs` and `simple_bulk_memtable.rs`.
- **Memtable Improvements**:
- Updated `SimpleBulkMemtable` to support batch size configuration and deduplication strategies.
- Modified `Series` struct in `time_series.rs` to include a configurable capacity.
- **Testing Enhancements**:
- Added new test `test_write_dedup` in `simple_bulk_memtable.rs` to verify deduplication functionality.
- Updated existing tests to include `OpType` parameter for better operation type handling.
- **Refactoring**:
- Renamed `BatchIterBuilder` to `BatchRangeBuilder` in `simple_bulk_memtable.rs` for clarity.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* bulk-multiparts-merge-reader:
- **Refactor `flush.rs`:** Removed `LastNonNullIter` usage and adjusted `DedupReader` instantiation to use `LastRow::new(false)` and `LastNonNull::new(false)`.
- **Enhance `simple_bulk_memtable.rs`:** Added logic to handle `LastNonNull` merge mode in `IterBuilder`. Introduced new tests: `test_delete_only` and `test_single_range` to verify delete operations and single range handling.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix: tests
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor: remove staled manifest structures
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* add RegionId to FileId
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* rename method
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* fix test cases
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* fix test
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* refactor: introduce RegionFileId
- FileId still only consist of an uuid
- PathProvider accepts RegionFileId and doesn't need to keep a region id
in it
- All Index applier takes RegionFileId and respects the region id in the RegionFileId
- FileMeta can still derive Serialize/Deserialize
- Refactor the CacheManager to accept RegionFileId
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: define PathType
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: adding PathType WIP
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: fix compiler errors
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: add path_type to region_dir_from_table_dir
Move region_dir_from_table_dir to mito and use join_dir internally
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: set path type to ApplierBuilder
Signed-off-by: evenyag <realevenyag@gmail.com>
* style: fmt code
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: fix passing incorrect dir to access layer
Signed-off-by: evenyag <realevenyag@gmail.com>
* refactor: remove region_dir from CompactionRegion
We can get table_dir and path_type from the access layer
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: fix unit tests
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: fix typo
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: update comment
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: correct marker path
Signed-off-by: evenyag <realevenyag@gmail.com>
* feat: use AccessLayer::build_region_dir to get region dir
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: log entries in test
Signed-off-by: evenyag <realevenyag@gmail.com>
* fix: set path type in catchup
Signed-off-by: evenyag <realevenyag@gmail.com>
* test: fix test_open_region_failure test
Signed-off-by: evenyag <realevenyag@gmail.com>
* chore: fix compiler errors
Signed-off-by: evenyag <realevenyag@gmail.com>
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Signed-off-by: evenyag <realevenyag@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
* feat/add-sst-file-num-in-region-stat:
### Add SST File Count to Region Statistics
- **Enhancements**:
- Added `sst_num` to track the number of SST files in region statistics across multiple modules.
- Updated `RegionStat` and `RegionStatistic` structs in `datanode.rs` and `region_engine.rs` to include `sst_num`.
- Modified `MitoRegion` and `SstVersion` in `region.rs` and `version.rs` to compute and return the number of SST files.
- Adjusted test cases in `collect_leader_region_handler.rs`, `failure_handler.rs`, `region_lease_handler.rs`, and `weight_compute.rs` to initialize `sst_num`.
- Updated `get_region_statistic` in `utils.rs` to sum `sst_num` from metadata and data statistics.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* feat/add-sst-file-num-in-region-stat:
Add `sst_num` to `region_statistics`
- Updated `region_statistics.rs` to include a new constant `SST_NUM` and added it to the schema and builder structures.
- Modified `information_schema.result` to reflect the addition of `sst_num` in the `region_statistics` table.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* wip
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Commit Message
Enhance DDL Module Accessibility and Refactor `verify_alter` Function
- **`statement.rs`**: Made the `ddl` module public to enhance accessibility.
- **`ddl.rs`**:
- Made `NAME_PATTERN_REG` public for broader usage.
- Refactored `verify_alter` function to be a standalone public function, improving modularity and reusability.
- Made `parse_partitions` function public to allow external access.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Add Parquet Writer and Enhance Row Modifier
- **Add Parquet Writer Module**: Introduced a new module `parquet_writer.rs` to bridge `opendal` `Writer` with `parquet` `AsyncFileWriter`.
- **Enhance Row Modifier**: Updated `RowModifier` to use `Default` trait and made `fill_internal_columns` a public static method in `row_modifier.rs`.
- **Expose Internal Structures**: Made `RowsIter`, `RowIter`, `TablesBuilder`, and `TableBuilder` structs public in `row_modifier.rs` and `prom_row_builder.rs`.
- **Update Metric Engine**: Changed `RowModifier` instantiation to use `default()` in `engine.rs`.
- **Modify Table Options Handling**: Added `fill_table_options_for_create` function in `insert.rs` to handle table options based on `AutoCreateTableType`.
- **Make Constants Public**: Changed `DEFAULT_ROW_GROUP_SIZE` to public in `parquet.rs`.
- **Expose Functions**: Made `extract_add_columns_expr` public in `expr_helper.rs` and `AutoCreateTableType` public in `insert.rs`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Commit Message
Enhance HTTP Server and Prometheus Integration
- **`http.rs`**: Made `extractor` module public to allow external access.
- **`prom_store.rs`**: Refactored `decode_remote_write_request` to return `TablesBuilder` and adjusted logic for processing requests based on pipeline usage.
- **`lib.rs`**: Made `metrics` module public for broader accessibility.
- **`prom_row_builder.rs`**: Exposed `tables` field in `TablesBuilder` for external manipulation.
- **`proto.rs`**: Changed visibility of `table_data` in `PromWriteRequest` to `pub(crate)` for internal module access.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
### Add Accessor Methods for Managers and Executors
- **`src/frontend/src/instance.rs`**: Added accessor methods for `NodeManagerRef`, `PartitionRuleManagerRef`, `CacheInvalidatorRef`, and `ProcedureExecutorRef` to the `Instance` struct.
- **`src/operator/src/insert.rs`**: Introduced methods to access `NodeManagerRef` and `PartitionRuleManagerRef` in the `Inserter` struct.
- **`src/operator/src/statement.rs`**: Added methods to retrieve `ProcedureExecutorRef` and `CacheInvalidatorRef` in the `StatementExecutor` struct.
### Change HashMap Implementation
- **`src/servers/src/prom_row_builder.rs`**: Replaced `ahash::HashMap` with `std::collections::HashMap`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
Refactor table option handling in `insert.rs`
- Replaced `Vec` with `HashMap` for `table_options` to improve efficiency.
- Extracted logic for filling table options into a new function `fill_table_options_for_create`.
- Modified `fill_table_options_for_create` to return the engine name based on `create_type`.
- Simplified the insertion of table options into `create_table_expr` by using `extend` method.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* refactor/expose-bulk-symbols:
Refactor `insert.rs` to separate engine name logic from table options
- Updated `Inserter` implementation to determine `engine_name` separately from `fill_table_options_for_create`.
- Modified `fill_table_options_for_create` to no longer return an engine name, focusing solely on populating table options.
- Adjusted logic to set `engine_name` based on `AutoCreateTableType`, using `METRIC_ENGINE_NAME` for logical tables and `default_engine()` otherwise.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/filter-empty-batch-in-bulk-insert-api:
**Add Early Return for Empty Record Batches in `bulk_insert.rs`**
- Implemented an early return in the `Inserter` implementation to handle cases where `record_batch.num_rows()` is zero, improving efficiency by avoiding unnecessary processing.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/filter-empty-batch-in-bulk-insert-api:
**Improve Bulk Insert Handling**
- **`handle_bulk_insert.rs`**: Added a check to handle cases where the batch has zero rows, immediately returning and sending a success response with zero rows processed.
- **`bulk_insert.rs`**: Enhanced logic to skip processing for masks that select none, optimizing the bulk insert operation by avoiding unnecessary iterations.
These changes improve the efficiency and robustness of the bulk insert process by handling edge cases more effectively.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/filter-empty-batch-in-bulk-insert-api:
### Refactor and Error Handling Enhancements
- **Refactored Timestamp Handling**: Introduced `timestamp_array_to_primitive` function in `timestamp.rs` to streamline conversion of timestamp arrays to primitive arrays, reducing redundancy in `handle_bulk_insert.rs` and `bulk_insert.rs`.
- **Error Handling**: Added `InconsistentTimestampLength` error in `error.rs` to handle mismatched timestamp column lengths in bulk insert operations.
- **Bulk Insert Logic**: Updated `handle_bulk_insert.rs` to utilize the new timestamp conversion function and added checks for timestamp length consistency.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/filter-empty-batch-in-bulk-insert-api:
**Refactor `bulk_insert.rs` to streamline imports**
- Simplified import statements by removing unused timestamp-related arrays and data types from the `arrow` crate in `bulk_insert.rs`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
refactor/building-backend-in-object-store:
### Refactor Object Store Configuration
- **Centralize Object Store Configurations**: Moved object store configurations (`FileConfig`, `S3Config`, `OssConfig`, `AzblobConfig`, `GcsConfig`) to `object-store/src/config.rs`.
- **Error Handling Enhancements**: Introduced `object-store/src/error.rs` for improved error handling related to object store operations.
- **Factory Pattern for Object Store**: Implemented `object-store/src/factory.rs` to create object store instances, consolidating logic from `datanode/src/store.rs`.
- **Remove Redundant Store Implementations**: Deleted individual store files (`azblob.rs`, `fs.rs`, `gcs.rs`, `oss.rs`, `s3.rs`) from `datanode/src/store/`.
- **Update Usage of Object Store Config**: Updated references to `ObjectStoreConfig` in `datanode.rs`, `standalone.rs`, `config.rs`, and `error.rs` to use the new centralized configuration.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/reordered-write-cause-incorrect-kv:
- **Enhance Testing in `partition_tree.rs`**: Added comprehensive test functions such as `kv_region_metadata`, `key_values`, and `collect_kvs` to improve the robustness of key-value operations and ensure correct behavior of the `PartitionTreeMemtable`.
- **Improve Key Handling in `dict.rs`**: Modified `KeyDictBuilder` to handle both full and sparse keys, ensuring correct mapping and insertion. Added a new test `test_builder_finish_with_sparse_key` to validate the handling of sparse keys.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/reordered-write-cause-incorrect-kv:
### Refactor `partition_tree.rs` for Improved Key Handling
- **Refactored Key Handling**: Simplified the `key_values` function to accept an iterator of keys, removing hardcoded key-value pairs. This change enhances flexibility and reduces redundancy in key management.
- **Updated Test Cases**: Modified test cases to use the new `key_values` function signature, ensuring they iterate over keys dynamically rather than relying on predefined lists.
Files affected:
- `src/mito2/src/memtable/partition_tree.rs`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* fix/reordered-write-cause-incorrect-kv:
Enhance Testing in `partition_tree.rs`
- Added assertions to verify key-value collection after `memtable` and `forked` operations.
- Refactored key-value writing logic for clarity in `forked` operations.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/print-series-count-after-wal-replay:
### Add Series Count Functionality and Logging Enhancements
- **`time_partition.rs`**: Introduced `series_count` method to calculate the total timeseries count across all time partitions.
- **`opener.rs`**: Enhanced logging to include the total timeseries replayed during WAL replay.
- **`version.rs`**: Added `series_count` method to `VersionControlData` for approximating timeseries count in the current version.
- **`handler.rs`**: Added entry and exit logging for the `sql` function to trace execution flow.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/print-series-count-after-wal-replay:
### Remove Unused Import
- **File Modified**: `src/servers/src/http/handler.rs`
- **Change Summary**: Removed the unused `info` import from `common_telemetry`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
### Add Metrics for Active Series and Values in Memtable
- **`simple_bulk_memtable.rs`**: Implemented `Drop` trait for `SimpleBulkMemtable` to decrement `MEMTABLE_ACTIVE_SERIES_COUNT` and `MEMTABLE_ACTIVE_VALUES_COUNT` upon dropping.
- **`time_series.rs`**:
- Introduced `SeriesMap` with `Drop` implementation to manage active series and values count.
- Updated `SeriesSet` and `Iter` to use `SeriesMap`.
- Added `num_values` method in `Series` to calculate the number of values.
- **`metrics.rs`**: Added `MEMTABLE_ACTIVE_SERIES_COUNT` and `MEMTABLE_ACTIVE_VALUES_COUNT` metrics to track active series and values in `TimeSeriesMemtable`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
- Add metrics for active series and field builders
- Update dashboard
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
**Add Series Count Tracking in Memtables**
- **`flush.rs`**: Updated `RegionFlushTask` to track and log the series count during memtable flush operations.
- **`memtable.rs`**: Introduced `series_count` in `MemtableStats` and added a method to retrieve it.
- **`partition_tree.rs`, `partition.rs`, `tree.rs`**: Implemented series count calculation in `PartitionTreeMemtable` and its components.
- **`simple_bulk_memtable.rs`, `time_series.rs`**: Integrated series count tracking in `SimpleBulkMemtable` and `TimeSeriesMemtable` implementations.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* Update src/mito2/src/memtable.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
