rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-09-09 23:02:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,660 Commits 266 Branches 82 Tags
17de33d8f0db0701c85ce38dde7a12259f7c8114
Commit Graph
2690 Commits
Author SHA1 Message Date
Pascal Seitz 17de33d8f0 merge multi-terms code 2026-08-04 13:23:19 +02:00
Pascal Seitz 295d5e4aa8 fix merge 2026-08-04 13:23:19 +02:00
Pascal Seitz 90ff347cc9 move impl 2026-08-04 13:23:19 +02:00
Pascal Seitz 766454d32a improve naming 2026-08-04 13:23:19 +02:00
Pascal Seitz 3010dbb8ea Optimize sparse multi-terms collection 
Filter documents without values after each single-valued field so later
decoding and key construction only process viable documents. Use exact
block lengths to validate aligned results and keep unsupported missing-value
combinations on the general path.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 864a1d4b2d improve missing performance 2026-08-04 13:23:19 +02:00
Pascal Seitz 2d1ddca54d fmt 2026-08-04 13:23:19 +02:00
Pascal Seitz f1efd748fa cleanup 2026-08-04 13:23:19 +02:00
Pascal Seitz 164f62c4cc Simplify optional multi-terms document filtering 
Track valid document positions in a reusable bitset instead of compacting
parallel document and index buffers after each optional field. Decode each
field against the original block and consult the mask before writing keys or
populating buckets.

This favors simpler collection logic over shrinking later sparse decodes.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 035fbe4d47 Refactor multi-terms collection around typed key codecs 
Fan out dynamic/JSON physical column combinations during aggregation tree
construction so each segment collector handles one typed accessor per
field and applies missing values exactly once.

Use a generic key codec and shared term bucket maps for both packed and
unpacked keys. Keep optional and multivalued fields eligible for packed
storage, short-circuit sparse documents, and account for spilled composite
keys.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 2d91c31468 rename vars 2026-08-04 13:23:19 +02:00
Pascal Seitz 59b6859c16 move term req data to collector 2026-08-04 13:23:19 +02:00
trinity-1686a 9bcd99c396 rustfmt 2026-08-04 13:23:19 +02:00
trinity.pointard d6666d09f3 extract per-field fast-path compatiblity check 2026-08-04 13:23:19 +02:00
trinity.pointard 8af115f802 decode column once per doc max 2026-08-04 13:23:19 +02:00
trinity.pointard 2f477e9414 more codex review 2026-08-04 13:23:19 +02:00
trinity.pointard 1fcf935f5c fix handling of bytes for cardinality agg 2026-08-04 13:23:19 +02:00
trinity.pointard 73010f4d86 codex review 2026-08-04 13:23:19 +02:00
trinity.pointard 53d7af7b50 rustfmt 2026-08-04 13:23:19 +02:00
trinity.pointard 901af02fe0 fix bool serialization 2026-08-04 13:23:19 +02:00
trinity.pointard 9a3b4f3a8a first pass of optimising multi-terms agg 2026-08-04 13:23:19 +02:00
trinity.pointard 261c0a11d4 allow extracting values from IntermediateMultiTermsBucketResult 2026-08-04 13:23:19 +02:00
trinity.pointard ebd37278ca first swab at multi-terms impl 2026-08-04 13:23:19 +02:00
Pascal Seitz 2b9ba006b8 test: exercise parsed exists queries against documents 2026-08-03 12:14:51 +02:00
Pascal Seitz e1bc31e27e docs: document exists query syntax 2026-08-03 12:14:51 +02:00
Darkheir 276740bed4 feat: PR review suggestion 
Signed-off-by: Darkheir <raphael.cohen@sekoia.io>
 2026-08-03 12:14:51 +02:00
Darkheir f981e6e1f1 feat: Add support for Exists leaf in query parser 
Signed-off-by: Darkheir <raphael.cohen@sekoia.io>
 2026-08-03 12:14:51 +02:00
Paul MasurelandPaul Masurel 3bb9a430dd Refactoring TextFastFieldOptions (#3020) 
Co-authored-by: Paul Masurel <paul.masurel@datadoghq.com>
 2026-08-02 17:17:02 +02:00
Pascal Seitz 667132fa7a rename to WithLanes 2026-07-29 08:57:54 +02:00
PSeitz 95f2c0c5b6 Optimize low-cardinality term aggregation counters 2026-07-29 08:57:54 +02:00
Pascal Seitz 277e59776b Extend Vec term storage to 20k with eager/lazy bucket ids 
Raise MAX_NUM_TERMS_FOR_VEC to 20_000 so the dense Vec term storage
(direct-indexed, no hashing/paging) is used for low/moderate cardinality
both with and without sub-aggregations, not just the <100 low-card case.
Split the old single threshold into MAX_NUM_TERMS_FOR_LOWCARD_SUBAGG (100,
still gates the Vec + LowCard-buffer pairing) and MAX_NUM_TERMS_FOR_VEC.

VecTermBuckets is now generic over a compile-time `const LAZY: bool`:
- eager (default): assigns all ids up front in `new`, branchless term_entry
- lazy: assigns on first occurrence, keeping the sub-agg bucket range down
  to the terms that actually occur (no blowup for sparse ordinal spaces)

A new BucketIdSlot::ASSIGNS_ID const plus the LAZY const fully gate the
first-seen branch out at monomorphization, so the only branchy path is the
deliberately-lazy <BucketId, true> instantiation. The with-sub-agg Vec path
picks eager below MAX_NUM_TERMS_FOR_EAGER_BUCKET_IDS (4096) and lazy above,
avoiding the ~6-11% hot-loop regression eager fixes while keeping large
sparse cases compact.

Biggest impact on terms_zipf_1000_only with -40%
```
full
terms_7                                               Memory: 46.5 KB              Avg: 2.3566ms (+4.29%)      Median: 2.3566ms (+4.29%)      [2.3566ms .. 2.3566ms]
terms_all_unique                                      Memory: 11.5 MB              Avg: 4.8532ms (-5.74%)      Median: 4.8532ms (-5.74%)      [4.8532ms .. 4.8532ms]
terms_all_unique_order_by_key                         Memory: 11.5 MB              Avg: 4.8448ms (-0.53%)      Median: 4.8448ms (-0.53%)      [4.8448ms .. 4.8448ms]
terms_150_000                                         Memory: 2.7 MB               Avg: 5.5295ms (+0.64%)      Median: 5.5295ms (+0.64%)      [5.5295ms .. 5.5295ms]
terms_many_top_1000                                   Memory: 5.3 MB               Avg: 8.4738ms (+0.58%)      Median: 8.4738ms (+0.58%)      [8.4738ms .. 8.4738ms]
terms_many_order_by_term                              Memory: 2.7 MB (-0.01%)      Avg: 4.5959ms (-0.15%)      Median: 4.5959ms (-0.15%)      [4.5959ms .. 4.5959ms]
terms_many_with_top_hits                              Memory: 48.9 MB (-0.00%)     Avg: 85.8206ms (-5.46%)     Median: 85.8206ms (-5.46%)     [85.8206ms .. 85.8206ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.7 MB (-0.00%)     Avg: 17.8747ms (+8.16%)     Median: 17.8747ms (+8.16%)     [17.8747ms .. 17.8747ms]
terms_many_with_avg_sub_agg                           Memory: 13.5 MB (-0.00%)     Avg: 14.7797ms (+2.54%)     Median: 14.7797ms (+2.54%)     [14.7797ms .. 14.7797ms]
terms_status_with_avg_sub_agg                         Memory: 92.1 KB (-0.35%)     Avg: 5.1467ms (+0.72%)      Median: 5.1467ms (+0.72%)      [5.1467ms .. 5.1467ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 213.2 KB             Avg: 3.9576ms (-2.73%)      Median: 3.9576ms (-2.73%)      [3.9576ms .. 3.9576ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 726.5 KB (+0.28%)    Avg: 12.1432ms (+3.30%)     Median: 12.1432ms (+3.30%)     [12.1432ms .. 12.1432ms]
terms_status_with_histogram                           Memory: 139.0 KB             Avg: 2.4717ms (+2.45%)      Median: 2.4717ms (+2.45%)      [2.4717ms .. 2.4717ms]
terms_status_with_date_histogram                      Memory: 145.9 KB             Avg: 2.3644ms (-0.44%)      Median: 2.3644ms (-0.44%)      [2.3644ms .. 2.3644ms]
terms_status_with_date_histogram_hard_bounds          Memory: 145.4 KB             Avg: 2.5705ms (+4.70%)      Median: 2.5705ms (+4.70%)      [2.5705ms .. 2.5705ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 143.3 KB             Avg: 3.8946ms (+1.61%)      Median: 3.8946ms (+1.61%)      [3.8946ms .. 3.8946ms]
terms_zipf_1000_only                                  Memory: 75.8 KB (-0.02%)     Avg: 1.2684ms (-41.50%)     Median: 1.2684ms (-41.50%)     [1.2684ms .. 1.2684ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB (+0.17%)      Avg: 20.9323ms (+4.08%)     Median: 20.9323ms (+4.08%)     [20.9323ms .. 20.9323ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 486.7 KB (+2.95%)    Avg: 8.5297ms (-1.66%)      Median: 8.5297ms (-1.66%)      [8.5297ms .. 8.5297ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.9 MB              Avg: 27.0210ms (+1.81%)     Median: 27.0210ms (+1.81%)     [27.0210ms .. 27.0210ms]
terms_status_with_cardinality_agg                     Memory: 93.9 KB              Avg: 3.2022ms (-0.64%)      Median: 3.2022ms (-0.64%)      [3.2022ms .. 3.2022ms]
terms_100_buckets_with_cardinality_agg                Memory: 9.8 MB (+0.31%)      Avg: 49.3188ms (-1.70%)     Median: 49.3188ms (-1.70%)     [49.3188ms .. 49.3188ms]
terms_many_with_single_term_order_by_card             Memory: 48.9 MB              Avg: 77.2127ms (-5.02%)     Median: 77.2127ms (-5.02%)     [77.2127ms .. 77.2127ms]
terms_many_with_single_term_2_order_by_card           Memory: 40.6 MB (-0.02%)     Avg: 45.8004ms (-17.57%)    Median: 45.8004ms (-17.57%)    [45.8004ms .. 45.8004ms]
```
 2026-07-27 18:00:37 +02:00
PSeitz-dd 70f0b039f5 Merge pull request #2978 from quickwit-oss/mallets/finalize-docidmapping 
feat: add custom doc id mapping finalization
 2026-07-22 15:20:19 +02:00
Luca CominardiandCursor 2554072e8f fix: address manual mapping review comments 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-22 14:50:29 +02:00
Luca Cominardi fe1bdf4a54 fix: make fmt 2026-07-22 11:48:17 +02:00
Luca CominardiandCursor 2c967e5d20 refactor: share doc id mapping construction 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-22 11:46:39 +02:00
Luca CominardiandPSeitz 2350c8e645 Update src/indexer/segment_writer.rs 
Co-authored-by: PSeitz <PSeitz@users.noreply.github.com>
 2026-07-22 10:22:51 +02:00
Luca CominardiandCursor 0a071c55f7 fix: clarify manual doc mapping serialization test 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-21 12:41:32 +02:00
Luca CominardiandCursor 56bd23bafe fix: move doc mapping tests to doc mapping module 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-21 12:40:08 +02:00
trinity-1686a 057458bf14 use enum PruneMode instead of bool 2026-07-13 12:16:18 +02:00
trinity-1686a f05ef0c4cc cr 2026-07-13 12:16:18 +02:00
trinity-1686a 16dfddf31a add method to prune intermediate agg results 2026-07-13 12:16:18 +02:00
Pascal Seitz 61693134be fix cache flush in aggregations 
fixes #2992

```
full
terms_7                                               Memory: 37.2 KB               Avg: 2.3958ms (+0.31%)     Median: 2.3896ms (+0.18%)     [2.3573ms .. 2.5176ms]
terms_all_unique                                      Memory: 10.8 MB               Avg: 5.5144ms (-1.07%)     Median: 5.4625ms (-1.98%)     [5.3364ms .. 5.9712ms]
terms_all_unique_order_by_key                         Memory: 10.8 MB               Avg: 5.2614ms (-0.85%)     Median: 5.2177ms (-1.21%)     [5.0823ms .. 5.6316ms]
terms_150_000                                         Memory: 2.7 MB                Avg: 5.5335ms (-1.07%)     Median: 5.5152ms (-1.06%)     [5.4151ms .. 5.9654ms]
terms_many_top_1000                                   Memory: 5.2 MB                Avg: 8.3579ms (-1.53%)     Median: 8.3604ms (-0.95%)     [8.2184ms .. 8.5421ms]
terms_many_order_by_term                              Memory: 2.7 MB                Avg: 4.6713ms (-0.07%)     Median: 4.6569ms (-0.15%)     [4.5994ms .. 4.9115ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.0 MB               Avg: 17.4981ms (-2.43%)    Median: 17.6075ms (-1.75%)    [15.8166ms .. 18.9250ms]
terms_status_with_avg_sub_agg                         Memory: 90.3 KB               Avg: 5.6365ms (+7.77%)     Median: 5.6255ms (+7.97%)     [5.5489ms .. 5.8254ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 318.5 KB (+56.52%)    Avg: 4.4504ms (+11.55%)    Median: 4.4436ms (+11.59%)    [4.3858ms .. 4.5692ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 684.9 KB              Avg: 11.8606ms (+0.19%)    Median: 11.8360ms (-0.02%)    [11.7478ms .. 12.0609ms]
terms_status_with_histogram                           Memory: 139.5 KB              Avg: 2.4524ms (-1.09%)     Median: 2.4521ms (-0.23%)     [2.4179ms .. 2.5049ms]
terms_status_with_date_histogram                      Memory: 136.7 KB              Avg: 2.3407ms (-1.28%)     Median: 2.3359ms (-1.06%)     [2.3001ms .. 2.4310ms]
terms_status_with_date_histogram_hard_bounds          Memory: 136.1 KB              Avg: 2.5113ms (-2.04%)     Median: 2.5073ms (-0.97%)     [2.4455ms .. 2.7280ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 137.3 KB              Avg: 3.8695ms (-0.48%)     Median: 3.8653ms (+0.11%)     [3.8093ms .. 4.0528ms]
terms_zipf_1000                                       Memory: 69.8 KB               Avg: 2.2022ms (-1.95%)     Median: 2.2026ms (-1.12%)     [2.1705ms .. 2.2859ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB                Avg: 20.4087ms (-0.02%)    Median: 20.3665ms (+0.11%)    [20.1912ms .. 20.7933ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 472.0 KB              Avg: 8.7387ms (-3.48%)     Median: 8.7043ms (-3.44%)     [8.6466ms .. 9.1396ms]
terms_zipf_90                                         Memory: 55.3 KB               Avg: 1.3784ms (-2.04%)     Median: 1.3787ms (-1.61%)     [1.3484ms .. 1.4611ms]
terms_zipf_90_with_sum_sub_agg                        Memory: 367.6 KB              Avg: 4.8520ms (+8.43%)     Median: 4.8326ms (+8.94%)     [4.8058ms .. 5.1278ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.8 MB               Avg: 25.0853ms (-7.70%)    Median: 25.0591ms (-7.12%)    [24.8103ms .. 25.4936ms]
terms_status_with_cardinality_agg                     Memory: 91.8 KB               Avg: 3.3667ms (+1.47%)     Median: 3.3690ms (+1.66%)     [3.3311ms .. 3.4070ms]
terms_100_buckets_with_cardinality_agg                Memory: 9.9 MB                Avg: 48.4768ms (-3.07%)    Median: 48.3745ms (-3.38%)    [48.1425ms .. 49.5503ms]
```
 2026-07-12 16:40:05 +02:00
Pascal Seitz 7152d53182 clippy 2026-07-10 12:33:33 +02:00
Luca Cominardi 1c5af9489a fix: cargo fmt 2026-07-07 15:25:35 +02:00
Luca CominardiandCursor 6f7cc10a9b fix: delete remap temp store after single segment finalize 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-07 15:16:24 +02:00
Pascal Seitz 6b8bd7b884 reorder if block 2026-07-03 17:44:44 +02:00
Pascal Seitz 9db05b660e fix overflow issue 2026-07-03 17:44:44 +02:00
Pascal Seitz 057e9d6618 make BucketId optional in aggregations 
If term aggregations don't have sub-aggregations, we don't need to carry
BucketId(u32).

```
full
terms_7                                               Memory: 46.5 KB             Avg: 2.4024ms (+1.96%)      Median: 2.4024ms (+1.96%)      [2.4024ms .. 2.4024ms]
terms_all_unique                                      Memory: 11.5 MB (-9.93%)    Avg: 4.7910ms (-22.62%)     Median: 4.7910ms (-22.62%)     [4.7910ms .. 4.7910ms]
terms_all_unique_order_by_key                         Memory: 11.5 MB (-9.94%)    Avg: 4.8056ms (-21.31%)     Median: 4.8056ms (-21.31%)     [4.8056ms .. 4.8056ms]
terms_150_000                                         Memory: 2.7 MB (-9.90%)     Avg: 5.7312ms (-5.96%)      Median: 5.7312ms (-5.96%)      [5.7312ms .. 5.7312ms]
terms_many_top_1000                                   Memory: 5.3 MB              Avg: 8.5912ms (-3.34%)      Median: 8.5912ms (-3.34%)      [8.5912ms .. 8.5912ms]
terms_many_order_by_term                              Memory: 2.7 MB (-9.95%)     Avg: 4.5581ms (-10.79%)     Median: 4.5581ms (-10.79%)     [4.5581ms .. 4.5581ms]
terms_many_with_top_hits                              Memory: 48.9 MB             Avg: 102.3672ms (-2.30%)    Median: 102.3672ms (-2.30%)    [102.3672ms .. 102.3672ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.7 MB             Avg: 18.4644ms (-0.11%)     Median: 18.4644ms (-0.11%)     [18.4644ms .. 18.4644ms]
terms_many_with_avg_sub_agg                           Memory: 13.5 MB             Avg: 16.7345ms (-4.09%)     Median: 16.7345ms (-4.09%)     [16.7345ms .. 16.7345ms]
terms_status_with_avg_sub_agg                         Memory: 92.1 KB             Avg: 5.1590ms (+0.33%)      Median: 5.1590ms (+0.33%)      [5.1590ms .. 5.1590ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 213.2 KB            Avg: 3.9562ms (+1.49%)      Median: 3.9562ms (+1.49%)      [3.9562ms .. 3.9562ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 724.4 KB            Avg: 11.7206ms (-4.68%)     Median: 11.7206ms (-4.68%)     [11.7206ms .. 11.7206ms]
terms_status_with_histogram                           Memory: 139.0 KB            Avg: 2.3985ms (-1.07%)      Median: 2.3985ms (-1.07%)      [2.3985ms .. 2.3985ms]
terms_status_with_date_histogram                      Memory: 145.9 KB            Avg: 2.2963ms (-1.86%)      Median: 2.2963ms (-1.86%)      [2.2963ms .. 2.2963ms]
terms_status_with_date_histogram_hard_bounds          Memory: 145.4 KB            Avg: 2.4312ms (-1.37%)      Median: 2.4312ms (-1.37%)      [2.4312ms .. 2.4312ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 143.3 KB            Avg: 3.8401ms (+0.44%)      Median: 3.8401ms (+0.44%)      [3.8401ms .. 3.8401ms]
terms_zipf_1000                                       Memory: 75.8 KB             Avg: 2.1611ms (-2.70%)      Median: 2.1611ms (-2.70%)      [2.1611ms .. 2.1611ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB              Avg: 20.1369ms (-1.16%)     Median: 20.1369ms (-1.16%)     [20.1369ms .. 20.1369ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 472.7 KB            Avg: 8.5851ms (-4.93%)      Median: 8.5851ms (-4.93%)      [8.5851ms .. 8.5851ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.9 MB             Avg: 26.3988ms (-6.61%)     Median: 26.3988ms (-6.61%)     [26.3988ms .. 26.3988ms]
```
 2026-07-03 17:44:44 +02:00
Pascal Seitz f313f22df7 Speed up range-query intersections via seek_danger on RangeDocSet (up to ~50x faster) 
A regular seek on RangeDocSet is costly: on a miss it fetches blocks and
scans the column forward to materialize the next matching doc. As a
non-leading docset in an intersection that work is wasted — the driver only
asks "does this candidate match?". seek_danger answers that with a cheap
point lookup via Column::values_for_doc, returning a lower bound on a miss
and leaving forward progress to the caller.

Forward seek_danger through ConstScorer.

Benchmarks (bool_queries_with_range, _all_results / DocSetCollector):

```
dense and 0.1% a
a_AND_num_rand:[0_TO_9]_all_results                                 Avg: 0.0827ms (-4.60%)     Median: 0.0825ms (-4.82%)     [0.0809ms .. 0.0891ms]    Output: 43
a_AND_num_asc:[0_TO_9]_all_results                                  Avg: 0.1937ms (-3.70%)     Median: 0.1930ms (-3.59%)     [0.1806ms .. 0.2044ms]    Output: 100
a_AND_num_rand_fast:[0_TO_9]_all_results                            Avg: 0.0367ms (-92.67%)    Median: 0.0365ms (-92.65%)    [0.0340ms .. 0.0398ms]    Output: 43
a_AND_num_asc_fast:[0_TO_9]_all_results                             Avg: 0.1052ms (-98.05%)    Median: 0.1050ms (-97.98%)    [0.1009ms .. 0.1117ms]    Output: 100
num_rand_fast:[0_TO_9]_AND_num_asc_fast:[0_TO_9]_all_results        Avg: 2.7147ms (-51.42%)    Median: 2.7075ms (-49.58%)    [2.6806ms .. 2.7799ms]    Output: 968
dense and 1% a
a_AND_num_rand:[0_TO_9]_all_results                                 Avg: 0.4373ms (-9.71%)     Median: 0.4357ms (-10.12%)    [0.4117ms .. 0.4711ms]    Output: 463
a_AND_num_asc:[0_TO_9]_all_results                                  Avg: 0.2342ms (-2.50%)     Median: 0.2338ms (-2.56%)     [0.2247ms .. 0.2452ms]    Output: 1_054
a_AND_num_rand_fast:[0_TO_9]_all_results                            Avg: 0.3956ms (-82.86%)    Median: 0.3943ms (-82.90%)    [0.3815ms .. 0.4119ms]    Output: 463
a_AND_num_asc_fast:[0_TO_9]_all_results                             Avg: 0.4896ms (-91.16%)    Median: 0.4862ms (-90.81%)    [0.4797ms .. 0.5084ms]    Output: 1_054
num_rand_fast:[0_TO_9]_AND_num_asc_fast:[0_TO_9]_all_results        Avg: 2.7108ms (-50.81%)    Median: 2.6925ms (-49.51%)    [2.6688ms .. 2.7868ms]    Output: 968
dense and 10% a
a_AND_num_rand:[0_TO_9]_all_results                                 Avg: 0.9869ms (-3.71%)     Median: 0.9833ms (-3.83%)     [0.9518ms .. 1.1218ms]    Output: 4_914
a_AND_num_asc:[0_TO_9]_all_results                                  Avg: 0.6352ms (-3.74%)     Median: 0.6363ms (-3.32%)     [0.6158ms .. 0.6488ms]    Output: 10_152
a_AND_num_rand_fast:[0_TO_9]_all_results                            Avg: 3.1264ms (+0.39%)     Median: 3.1466ms (+1.34%)     [3.0261ms .. 3.2051ms]    Output: 4_914
a_AND_num_asc_fast:[0_TO_9]_all_results                             Avg: 4.1547ms (-31.12%)    Median: 4.0933ms (-28.55%)    [3.7648ms .. 4.7600ms]    Output: 10_152
num_rand_fast:[0_TO_9]_AND_num_asc_fast:[0_TO_9]_all_results        Avg: 2.6973ms (-52.30%)    Median: 2.6901ms (-49.86%)    [2.6689ms .. 2.7677ms]    Output: 968
```

Gains are largest when the range query is the non-leading docset of a low-cardinality intersection.
 2026-07-03 13:42:33 +02:00
Pascal Seitz 82bee54a00 block_search: drop unsafe indexing, remove K=64 
For K∈{2,4,8,16,32} LLVM proves the index bounds and elides the checks,
so get_unchecked buys nothing on the production K=8 path. K=64 was the
only value that defeated bounds-check elision (one check in the tail
scan) and it was instantiated in tests only — drop it.

block_search: cite the k-ary search paper

Document that kary_search is the 'k-ary search on a sorted array' variant
from Schlegel, Gemulla & Lehner (DaMoN 2009), specialized to a lower-bound.
 2026-07-02 18:27:40 +02:00