fix: snap histogram scan start down to bucket boundary to fix left-edge drop (#12313)

## Problem

Histogram panels showed a false drop in the **leftmost bucket**.

`histogram()` rewrites to `date_bin(interval, _timestamp, epoch)`, which
assigns each record to the **left boundary** of its bucket. So a record
at `08:10:17` gets key `08:10:00`.

But the scan filter was `_timestamp >= start_time` where `start_time`
was the raw query time (e.g. `08:10:17`). Records in `[08:10:00,
08:10:17)` passed the bucket label check but were excluded from the scan
— making the first bucket appear partially filled.

Fixes #1753 (o2-enterprise)

## Fix

Snap `start_time` **down to the nearest bucket boundary** at the
**leader node**, before it is broadcast to all follower nodes via
`FlightSearchRequest`.

Location: `RemoteScanNodes::get_remote_node()` in
`distributed_plan/node.rs`

```rust
start_time: {
    let t = self.req.time_range.as_ref().map(|x| x.0).unwrap_or(0);
    if self.req.histogram_interval > 0 {
        histogram_bucket_start(t, self.req.histogram_interval * 1_000_000)
    } else {
        t
    }
},
```

The arithmetic is extracted into `histogram_bucket_start(timestamp_us,
interval_us)` in `sql/histogram.rs` so the formula lives in one place.

Doing the snap at the leader means:
- All followers receive an already-aligned `start_time`
- `SimpleHistogramVisitor` / `SimpleMultiHistogramVisitor` on followers
need no changes (their `start_time % rounding_by` is already 0)
- Covers both cluster leader and super cluster leader (both go through
`generate_remote_scan_rules → get_remote_node()`)

## Test plan

- [ ] Deploy and run a histogram query spanning a non-aligned
`start_time`
- [ ] Verify first bucket count matches adjacent full buckets
- [ ] Verify queries without `histogram_interval` are unaffected
This commit is contained in:
Loakesh Indiran
2026-05-27 13:56:35 +00:00
committed by GitHub
parent 4d2291d335
commit 71663da9cd
2 changed files with 17 additions and 1 deletions
@@ -23,6 +23,8 @@ use datafusion::common::TableReference;
use hashbrown::HashMap;
use proto::cluster_rpc::{IndexInfo, KvItem, QueryIdentifier, SearchInfo, SuperClusterInfo};
use crate::service::search::sql::histogram::histogram_bucket_start;
#[derive(Debug, Clone)]
pub struct RemoteScanNodes {
pub req: Request,
@@ -73,7 +75,14 @@ impl RemoteScanNodes {
.get(table_name)
.unwrap_or(&vec![])
.clone(),
start_time: self.req.time_range.as_ref().map(|x| x.0).unwrap_or(0),
start_time: {
let t = self.req.time_range.as_ref().map(|x| x.0).unwrap_or(0);
if self.req.histogram_interval > 0 {
histogram_bucket_start(t, self.req.histogram_interval * 1_000_000)
} else {
t
}
},
end_time: self.req.time_range.as_ref().map(|x| x.1).unwrap_or(0),
timeout: self.req.timeout as u64,
use_cache: self.req.use_cache,
+7
View File
@@ -190,6 +190,13 @@ fn extract_where_clause(statement: &Statement) -> Result<String, Error> {
}
}
/// Snaps `timestamp_us` down to the nearest histogram bucket boundary.
/// Both arguments are in microseconds. Used to ensure the first bucket
/// returned by date_bin() is fully populated rather than partial.
pub fn histogram_bucket_start(timestamp_us: i64, interval_us: i64) -> i64 {
timestamp_us - timestamp_us % interval_us
}
#[cfg(test)]
mod tests {
use super::*;