feat/objbench:

### Add Metrics for Indexing and Conversion in `access_layer.rs` and `writer.rs`

 - **Enhancements in `access_layer.rs`:**
   - Added new metrics `convert` and `index` to the `Metrics` struct to track conversion and indexing durations.

 - **Updates in `writer.rs`:**
   - Implemented tracking of indexing duration by measuring the time taken for `update` in the indexer.
   - Added measurement of conversion duration for `convert_batch` to enhance performance monitoring.

Signed-off-by: Lei, HUANG <mrsatangel@gmail.com>
This commit is contained in:
Lei, HUANG
2025-09-24 18:14:00 +08:00
parent f3aa967aae
commit dcf5a62014
2 changed files with 6 additions and 1 deletions

View File

@@ -47,7 +47,8 @@ pub type SstInfoArray = SmallVec<[SstInfo; 2]>;
pub struct Metrics {
pub read: Duration,
pub write: Duration,
pub convert: Duration,
pub index: Duration,
pub close: Duration,
}

View File

@@ -172,7 +172,9 @@ where
match res {
Ok(mut batch) => {
stats.update(&batch);
let index_start = Instant::now();
self.get_or_create_indexer().await.update(&mut batch).await;
metrics.index += index_start.elapsed();
}
Err(e) => {
self.get_or_create_indexer().await.abort().await;
@@ -250,7 +252,9 @@ where
};
metrics.read += read_start.elapsed();
let convert_start = Instant::now();
let arrow_batch = write_format.convert_batch(&batch)?;
metrics.convert += convert_start.elapsed();
let write_start = Instant::now();
self.maybe_init_writer(write_format.arrow_schema(), opts)