* chore/update-opendal-dashboard:
### Update Grafana Dashboard Queries
- **Enhanced Metrics Queries**: Updated Prometheus queries in `dashboard.json`, `dashboard.md`, and `dashboard.yaml` files for both `cluster` and `standalone` dashboards to include additional operations (`Reader::read`, `Writer::write`, `Writer::close`) in the metrics calculations.
- **Legend Format Adjustments**: Modified legend formats to include the `operation` field for better clarity in visualizations.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/update-opendal-dashboard:
Enhance Legend Format in Grafana Dashboards
- Updated the `legendFormat` in `dashboard.json`, `dashboard.md`, and `dashboard.yaml` files for both `cluster` and `standalone` dashboards to include the `operation` field.
- This change affects the following files:
- `grafana/dashboards/metrics/cluster/dashboard.json`
- `grafana/dashboards/metrics/cluster/dashboard.md`
- `grafana/dashboards/metrics/cluster/dashboard.yaml`
- `grafana/dashboards/metrics/standalone/dashboard.json`
- `grafana/dashboards/metrics/standalone/dashboard.md`
- `grafana/dashboards/metrics/standalone/dashboard.yaml`
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
### Add Metrics for Active Series and Values in Memtable
- **`simple_bulk_memtable.rs`**: Implemented `Drop` trait for `SimpleBulkMemtable` to decrement `MEMTABLE_ACTIVE_SERIES_COUNT` and `MEMTABLE_ACTIVE_VALUES_COUNT` upon dropping.
- **`time_series.rs`**:
- Introduced `SeriesMap` with `Drop` implementation to manage active series and values count.
- Updated `SeriesSet` and `Iter` to use `SeriesMap`.
- Added `num_values` method in `Series` to calculate the number of values.
- **`metrics.rs`**: Added `MEMTABLE_ACTIVE_SERIES_COUNT` and `MEMTABLE_ACTIVE_VALUES_COUNT` metrics to track active series and values in `TimeSeriesMemtable`.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
- Add metrics for active series and field builders
- Update dashboard
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* chore/series-metrics:
**Add Series Count Tracking in Memtables**
- **`flush.rs`**: Updated `RegionFlushTask` to track and log the series count during memtable flush operations.
- **`memtable.rs`**: Introduced `series_count` in `MemtableStats` and added a method to retrieve it.
- **`partition_tree.rs`, `partition.rs`, `tree.rs`**: Implemented series count calculation in `PartitionTreeMemtable` and its components.
- **`simple_bulk_memtable.rs`, `time_series.rs`**: Integrated series count tracking in `SimpleBulkMemtable` and `TimeSeriesMemtable` implementations.
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
* Update src/mito2/src/memtable.rs
Co-authored-by: Yingwen <realevenyag@gmail.com>
---------
Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
Co-authored-by: Yingwen <realevenyag@gmail.com>
* 1. rename `greptime_mito_flush_errors_total` metric to `greptime_mito_flush_errors_total` for consistency
2. update grafana dashboard to add following panel:
- compaction input/output bytes
- bulk insert handle elasped time in frontend and region worker
* add metrics
* chore/bench-metrics: Add INFLIGHT_FLUSH_COUNT Metric to Flush Process
• Introduced INFLIGHT_FLUSH_COUNT metric to track the number of ongoing flush operations.
• Incremented INFLIGHT_FLUSH_COUNT in FlushScheduler to monitor active flushes.
• Removed redundant increment of INFLIGHT_FLUSH_COUNT in RegionWorkerLoop to prevent double counting.
* chore/bench-metrics: Add Metrics for Compaction and Flush Operations
• Introduced INFLIGHT_COMPACTION_COUNT and INFLIGHT_FLUSH_COUNT metrics to track the number of ongoing compaction and flush operations.
• Incremented INFLIGHT_COMPACTION_COUNT when scheduling remote and local compaction jobs, and decremented it upon completion.
• Added INFLIGHT_FLUSH_COUNT increment and decrement logic around flush tasks to monitor active flush operations.
• Removed redundant metric updates in worker.rs and handle_compaction.rs to streamline metric handling.
* chore: add metrics for remote compaction jobs
* chore: format
* chore: also add dashbaord
* chore: update grafana dashboard to reflect recent metric changes
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
* chore: add a blank line at the end
---------
Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: dennis zhuang <killme2008@gmail.com>
