mirror of
https://github.com/quickwit-oss/tantivy.git
synced 2026-11-20 01:24:15 +00:00
The metric reducers folded values one at a time through recurrences that carry state across every iteration: Kahan compensated summation in `collect_stats` (sum/avg/min/max/count/stats) and Welford's online variance in `extended_stats`. These strict dependency chains block both CPU pipelining and auto-vectorization. `collect_stats` now reduces each block of values as a slice into 4 independent Kahan (sum, delta) lanes plus 4 min/max lanes, combined at the end of the block. `extended_stats` accumulates each block into 4 independent `IntermediateExtendedStats` lanes and folds them into the bucket with the existing `merge_fruits` (Chan parallel-variance combination), the same operation used to merge segment results. The lanes carry the bucket's sigma, since `merge_fruits` replaces an empty bucket with the incoming lane. Blocks with fewer than two values per lane are collected value by value, where setting up and combining lanes costs more than it saves. Only the floating-point summation order changes, as it already does across segment merges. All aggregation tests pass within the existing `2e-12` tolerance, including the rounding-sensitive `test_aggregation_level1`, and a new test covers sigma through the lane path.