fixes multivalue ff regression by avoiding using `get_val`. Line::train calls repeatedly get_val, but get_val implementation on Column for multivalues is very slow. The fix is to use the iterator instead. Longterm fix should be to remove get_val access in serialization.
Old Code
test fastfield::bench::bench_multi_value_ff_merge_few_segments ... bench: 46,103,960 ns/iter (+/- 2,066,083)
test fastfield::bench::bench_multi_value_ff_merge_many_segments ... bench: 83,073,036 ns/iter (+/- 4,373,615)
est fastfield::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 64,178,576 ns/iter (+/- 1,466,700)
Current
running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments ... bench: 57,379,523 ns/iter (+/- 3,220,787)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments ... bench: 90,831,688 ns/iter (+/- 1,445,486)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 158,313,264 ns/iter (+/- 28,823,250)
With Fix
running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments ... bench: 57,635,671 ns/iter (+/- 2,707,361)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments ... bench: 91,468,712 ns/iter (+/- 11,393,581)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 73,909,138 ns/iter (+/- 15,846,097)
The Monotonic mapping was using the default implementation
for `get_range` and `.iter`.
As a result, some of the column used in merge (e.g. multivalued
fast fields) were exhibiting a very strong performance regression.
* Rename BlockwiseLinear to BlockwiseLinearLegacy
Reimplements the blockwise multilinear codec using integer arithmetics.
Added comments
* add estimate for blockwise
* Added one unit test
* use int based for linear interpol
* fix merge conflicts
* reuse code
* cargo fmt
* fix clippy
* fix test
* fix off by one
fix off by one to accurately interpolate autoincrement fields
* extend test, fix estimate
* remove legacy codec
Co-authored-by: Pascal Seitz <pascal.seitz@gmail.com>
