rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-10-07 20:32:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,655 Commits 276 Branches 82 Tags
3010dbb8ea4b2ec08530afd7d8d7b143c02452c6
Commit Graph
323 Commits
Author SHA1 Message Date
Pascal Seitz 3010dbb8ea Optimize sparse multi-terms collection 
Filter documents without values after each single-valued field so later
decoding and key construction only process viable documents. Use exact
block lengths to validate aligned results and keep unsupported missing-value
combinations on the general path.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 864a1d4b2d improve missing performance 2026-08-04 13:23:19 +02:00
Pascal Seitz 2d1ddca54d fmt 2026-08-04 13:23:19 +02:00
Pascal Seitz f1efd748fa cleanup 2026-08-04 13:23:19 +02:00
Pascal Seitz 164f62c4cc Simplify optional multi-terms document filtering 
Track valid document positions in a reusable bitset instead of compacting
parallel document and index buffers after each optional field. Decode each
field against the original block and consult the mask before writing keys or
populating buckets.

This favors simpler collection logic over shrinking later sparse decodes.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 035fbe4d47 Refactor multi-terms collection around typed key codecs 
Fan out dynamic/JSON physical column combinations during aggregation tree
construction so each segment collector handles one typed accessor per
field and applies missing values exactly once.

Use a generic key codec and shared term bucket maps for both packed and
unpacked keys. Keep optional and multivalued fields eligible for packed
storage, short-circuit sparse documents, and account for spilled composite
keys.
 2026-08-04 13:23:19 +02:00
Pascal Seitz 2d91c31468 rename vars 2026-08-04 13:23:19 +02:00
Pascal Seitz 59b6859c16 move term req data to collector 2026-08-04 13:23:19 +02:00
trinity-1686a 9bcd99c396 rustfmt 2026-08-04 13:23:19 +02:00
trinity.pointard d6666d09f3 extract per-field fast-path compatiblity check 2026-08-04 13:23:19 +02:00
trinity.pointard 8af115f802 decode column once per doc max 2026-08-04 13:23:19 +02:00
trinity.pointard 2f477e9414 more codex review 2026-08-04 13:23:19 +02:00
trinity.pointard 1fcf935f5c fix handling of bytes for cardinality agg 2026-08-04 13:23:19 +02:00
trinity.pointard 73010f4d86 codex review 2026-08-04 13:23:19 +02:00
trinity.pointard 53d7af7b50 rustfmt 2026-08-04 13:23:19 +02:00
trinity.pointard 901af02fe0 fix bool serialization 2026-08-04 13:23:19 +02:00
trinity.pointard 9a3b4f3a8a first pass of optimising multi-terms agg 2026-08-04 13:23:19 +02:00
trinity.pointard 261c0a11d4 allow extracting values from IntermediateMultiTermsBucketResult 2026-08-04 13:23:19 +02:00
trinity.pointard ebd37278ca first swab at multi-terms impl 2026-08-04 13:23:19 +02:00
Paul MasurelandPaul Masurel 3bb9a430dd Refactoring TextFastFieldOptions (#3020) 
Co-authored-by: Paul Masurel <paul.masurel@datadoghq.com>
 2026-08-02 17:17:02 +02:00
Pascal Seitz 667132fa7a rename to WithLanes 2026-07-29 08:57:54 +02:00
PSeitz 95f2c0c5b6 Optimize low-cardinality term aggregation counters 2026-07-29 08:57:54 +02:00
Pascal Seitz 277e59776b Extend Vec term storage to 20k with eager/lazy bucket ids 
Raise MAX_NUM_TERMS_FOR_VEC to 20_000 so the dense Vec term storage
(direct-indexed, no hashing/paging) is used for low/moderate cardinality
both with and without sub-aggregations, not just the <100 low-card case.
Split the old single threshold into MAX_NUM_TERMS_FOR_LOWCARD_SUBAGG (100,
still gates the Vec + LowCard-buffer pairing) and MAX_NUM_TERMS_FOR_VEC.

VecTermBuckets is now generic over a compile-time `const LAZY: bool`:
- eager (default): assigns all ids up front in `new`, branchless term_entry
- lazy: assigns on first occurrence, keeping the sub-agg bucket range down
  to the terms that actually occur (no blowup for sparse ordinal spaces)

A new BucketIdSlot::ASSIGNS_ID const plus the LAZY const fully gate the
first-seen branch out at monomorphization, so the only branchy path is the
deliberately-lazy <BucketId, true> instantiation. The with-sub-agg Vec path
picks eager below MAX_NUM_TERMS_FOR_EAGER_BUCKET_IDS (4096) and lazy above,
avoiding the ~6-11% hot-loop regression eager fixes while keeping large
sparse cases compact.

Biggest impact on terms_zipf_1000_only with -40%
```
full
terms_7                                               Memory: 46.5 KB              Avg: 2.3566ms (+4.29%)      Median: 2.3566ms (+4.29%)      [2.3566ms .. 2.3566ms]
terms_all_unique                                      Memory: 11.5 MB              Avg: 4.8532ms (-5.74%)      Median: 4.8532ms (-5.74%)      [4.8532ms .. 4.8532ms]
terms_all_unique_order_by_key                         Memory: 11.5 MB              Avg: 4.8448ms (-0.53%)      Median: 4.8448ms (-0.53%)      [4.8448ms .. 4.8448ms]
terms_150_000                                         Memory: 2.7 MB               Avg: 5.5295ms (+0.64%)      Median: 5.5295ms (+0.64%)      [5.5295ms .. 5.5295ms]
terms_many_top_1000                                   Memory: 5.3 MB               Avg: 8.4738ms (+0.58%)      Median: 8.4738ms (+0.58%)      [8.4738ms .. 8.4738ms]
terms_many_order_by_term                              Memory: 2.7 MB (-0.01%)      Avg: 4.5959ms (-0.15%)      Median: 4.5959ms (-0.15%)      [4.5959ms .. 4.5959ms]
terms_many_with_top_hits                              Memory: 48.9 MB (-0.00%)     Avg: 85.8206ms (-5.46%)     Median: 85.8206ms (-5.46%)     [85.8206ms .. 85.8206ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.7 MB (-0.00%)     Avg: 17.8747ms (+8.16%)     Median: 17.8747ms (+8.16%)     [17.8747ms .. 17.8747ms]
terms_many_with_avg_sub_agg                           Memory: 13.5 MB (-0.00%)     Avg: 14.7797ms (+2.54%)     Median: 14.7797ms (+2.54%)     [14.7797ms .. 14.7797ms]
terms_status_with_avg_sub_agg                         Memory: 92.1 KB (-0.35%)     Avg: 5.1467ms (+0.72%)      Median: 5.1467ms (+0.72%)      [5.1467ms .. 5.1467ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 213.2 KB             Avg: 3.9576ms (-2.73%)      Median: 3.9576ms (-2.73%)      [3.9576ms .. 3.9576ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 726.5 KB (+0.28%)    Avg: 12.1432ms (+3.30%)     Median: 12.1432ms (+3.30%)     [12.1432ms .. 12.1432ms]
terms_status_with_histogram                           Memory: 139.0 KB             Avg: 2.4717ms (+2.45%)      Median: 2.4717ms (+2.45%)      [2.4717ms .. 2.4717ms]
terms_status_with_date_histogram                      Memory: 145.9 KB             Avg: 2.3644ms (-0.44%)      Median: 2.3644ms (-0.44%)      [2.3644ms .. 2.3644ms]
terms_status_with_date_histogram_hard_bounds          Memory: 145.4 KB             Avg: 2.5705ms (+4.70%)      Median: 2.5705ms (+4.70%)      [2.5705ms .. 2.5705ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 143.3 KB             Avg: 3.8946ms (+1.61%)      Median: 3.8946ms (+1.61%)      [3.8946ms .. 3.8946ms]
terms_zipf_1000_only                                  Memory: 75.8 KB (-0.02%)     Avg: 1.2684ms (-41.50%)     Median: 1.2684ms (-41.50%)     [1.2684ms .. 1.2684ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB (+0.17%)      Avg: 20.9323ms (+4.08%)     Median: 20.9323ms (+4.08%)     [20.9323ms .. 20.9323ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 486.7 KB (+2.95%)    Avg: 8.5297ms (-1.66%)      Median: 8.5297ms (-1.66%)      [8.5297ms .. 8.5297ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.9 MB              Avg: 27.0210ms (+1.81%)     Median: 27.0210ms (+1.81%)     [27.0210ms .. 27.0210ms]
terms_status_with_cardinality_agg                     Memory: 93.9 KB              Avg: 3.2022ms (-0.64%)      Median: 3.2022ms (-0.64%)      [3.2022ms .. 3.2022ms]
terms_100_buckets_with_cardinality_agg                Memory: 9.8 MB (+0.31%)      Avg: 49.3188ms (-1.70%)     Median: 49.3188ms (-1.70%)     [49.3188ms .. 49.3188ms]
terms_many_with_single_term_order_by_card             Memory: 48.9 MB              Avg: 77.2127ms (-5.02%)     Median: 77.2127ms (-5.02%)     [77.2127ms .. 77.2127ms]
terms_many_with_single_term_2_order_by_card           Memory: 40.6 MB (-0.02%)     Avg: 45.8004ms (-17.57%)    Median: 45.8004ms (-17.57%)    [45.8004ms .. 45.8004ms]
```
 2026-07-27 18:00:37 +02:00
PSeitz-dd 70f0b039f5 Merge pull request #2978 from quickwit-oss/mallets/finalize-docidmapping 
feat: add custom doc id mapping finalization
 2026-07-22 15:20:19 +02:00
trinity-1686a 057458bf14 use enum PruneMode instead of bool 2026-07-13 12:16:18 +02:00
trinity-1686a f05ef0c4cc cr 2026-07-13 12:16:18 +02:00
trinity-1686a 16dfddf31a add method to prune intermediate agg results 2026-07-13 12:16:18 +02:00
Pascal Seitz 61693134be fix cache flush in aggregations 
fixes #2992

```
full
terms_7                                               Memory: 37.2 KB               Avg: 2.3958ms (+0.31%)     Median: 2.3896ms (+0.18%)     [2.3573ms .. 2.5176ms]
terms_all_unique                                      Memory: 10.8 MB               Avg: 5.5144ms (-1.07%)     Median: 5.4625ms (-1.98%)     [5.3364ms .. 5.9712ms]
terms_all_unique_order_by_key                         Memory: 10.8 MB               Avg: 5.2614ms (-0.85%)     Median: 5.2177ms (-1.21%)     [5.0823ms .. 5.6316ms]
terms_150_000                                         Memory: 2.7 MB                Avg: 5.5335ms (-1.07%)     Median: 5.5152ms (-1.06%)     [5.4151ms .. 5.9654ms]
terms_many_top_1000                                   Memory: 5.2 MB                Avg: 8.3579ms (-1.53%)     Median: 8.3604ms (-0.95%)     [8.2184ms .. 8.5421ms]
terms_many_order_by_term                              Memory: 2.7 MB                Avg: 4.6713ms (-0.07%)     Median: 4.6569ms (-0.15%)     [4.5994ms .. 4.9115ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.0 MB               Avg: 17.4981ms (-2.43%)    Median: 17.6075ms (-1.75%)    [15.8166ms .. 18.9250ms]
terms_status_with_avg_sub_agg                         Memory: 90.3 KB               Avg: 5.6365ms (+7.77%)     Median: 5.6255ms (+7.97%)     [5.5489ms .. 5.8254ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 318.5 KB (+56.52%)    Avg: 4.4504ms (+11.55%)    Median: 4.4436ms (+11.59%)    [4.3858ms .. 4.5692ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 684.9 KB              Avg: 11.8606ms (+0.19%)    Median: 11.8360ms (-0.02%)    [11.7478ms .. 12.0609ms]
terms_status_with_histogram                           Memory: 139.5 KB              Avg: 2.4524ms (-1.09%)     Median: 2.4521ms (-0.23%)     [2.4179ms .. 2.5049ms]
terms_status_with_date_histogram                      Memory: 136.7 KB              Avg: 2.3407ms (-1.28%)     Median: 2.3359ms (-1.06%)     [2.3001ms .. 2.4310ms]
terms_status_with_date_histogram_hard_bounds          Memory: 136.1 KB              Avg: 2.5113ms (-2.04%)     Median: 2.5073ms (-0.97%)     [2.4455ms .. 2.7280ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 137.3 KB              Avg: 3.8695ms (-0.48%)     Median: 3.8653ms (+0.11%)     [3.8093ms .. 4.0528ms]
terms_zipf_1000                                       Memory: 69.8 KB               Avg: 2.2022ms (-1.95%)     Median: 2.2026ms (-1.12%)     [2.1705ms .. 2.2859ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB                Avg: 20.4087ms (-0.02%)    Median: 20.3665ms (+0.11%)    [20.1912ms .. 20.7933ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 472.0 KB              Avg: 8.7387ms (-3.48%)     Median: 8.7043ms (-3.44%)     [8.6466ms .. 9.1396ms]
terms_zipf_90                                         Memory: 55.3 KB               Avg: 1.3784ms (-2.04%)     Median: 1.3787ms (-1.61%)     [1.3484ms .. 1.4611ms]
terms_zipf_90_with_sum_sub_agg                        Memory: 367.6 KB              Avg: 4.8520ms (+8.43%)     Median: 4.8326ms (+8.94%)     [4.8058ms .. 5.1278ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.8 MB               Avg: 25.0853ms (-7.70%)    Median: 25.0591ms (-7.12%)    [24.8103ms .. 25.4936ms]
terms_status_with_cardinality_agg                     Memory: 91.8 KB               Avg: 3.3667ms (+1.47%)     Median: 3.3690ms (+1.66%)     [3.3311ms .. 3.4070ms]
terms_100_buckets_with_cardinality_agg                Memory: 9.9 MB                Avg: 48.4768ms (-3.07%)    Median: 48.3745ms (-3.38%)    [48.1425ms .. 49.5503ms]
```
 2026-07-12 16:40:05 +02:00
Pascal Seitz 7152d53182 clippy 2026-07-10 12:33:33 +02:00
Pascal Seitz 6b8bd7b884 reorder if block 2026-07-03 17:44:44 +02:00
Pascal Seitz 9db05b660e fix overflow issue 2026-07-03 17:44:44 +02:00
Pascal Seitz 057e9d6618 make BucketId optional in aggregations 
If term aggregations don't have sub-aggregations, we don't need to carry
BucketId(u32).

```
full
terms_7                                               Memory: 46.5 KB             Avg: 2.4024ms (+1.96%)      Median: 2.4024ms (+1.96%)      [2.4024ms .. 2.4024ms]
terms_all_unique                                      Memory: 11.5 MB (-9.93%)    Avg: 4.7910ms (-22.62%)     Median: 4.7910ms (-22.62%)     [4.7910ms .. 4.7910ms]
terms_all_unique_order_by_key                         Memory: 11.5 MB (-9.94%)    Avg: 4.8056ms (-21.31%)     Median: 4.8056ms (-21.31%)     [4.8056ms .. 4.8056ms]
terms_150_000                                         Memory: 2.7 MB (-9.90%)     Avg: 5.7312ms (-5.96%)      Median: 5.7312ms (-5.96%)      [5.7312ms .. 5.7312ms]
terms_many_top_1000                                   Memory: 5.3 MB              Avg: 8.5912ms (-3.34%)      Median: 8.5912ms (-3.34%)      [8.5912ms .. 8.5912ms]
terms_many_order_by_term                              Memory: 2.7 MB (-9.95%)     Avg: 4.5581ms (-10.79%)     Median: 4.5581ms (-10.79%)     [4.5581ms .. 4.5581ms]
terms_many_with_top_hits                              Memory: 48.9 MB             Avg: 102.3672ms (-2.30%)    Median: 102.3672ms (-2.30%)    [102.3672ms .. 102.3672ms]
terms_all_unique_with_avg_sub_agg                     Memory: 54.7 MB             Avg: 18.4644ms (-0.11%)     Median: 18.4644ms (-0.11%)     [18.4644ms .. 18.4644ms]
terms_many_with_avg_sub_agg                           Memory: 13.5 MB             Avg: 16.7345ms (-4.09%)     Median: 16.7345ms (-4.09%)     [16.7345ms .. 16.7345ms]
terms_status_with_avg_sub_agg                         Memory: 92.1 KB             Avg: 5.1590ms (+0.33%)      Median: 5.1590ms (+0.33%)      [5.1590ms .. 5.1590ms]
terms_status_with_terms_zipf_1000_sub_agg             Memory: 213.2 KB            Avg: 3.9562ms (+1.49%)      Median: 3.9562ms (+1.49%)      [3.9562ms .. 3.9562ms]
terms_zipf_1000_with_terms_status_sub_agg             Memory: 724.4 KB            Avg: 11.7206ms (-4.68%)     Median: 11.7206ms (-4.68%)     [11.7206ms .. 11.7206ms]
terms_status_with_histogram                           Memory: 139.0 KB            Avg: 2.3985ms (-1.07%)      Median: 2.3985ms (-1.07%)      [2.3985ms .. 2.3985ms]
terms_status_with_date_histogram                      Memory: 145.9 KB            Avg: 2.2963ms (-1.86%)      Median: 2.2963ms (-1.86%)      [2.2963ms .. 2.2963ms]
terms_status_with_date_histogram_hard_bounds          Memory: 145.4 KB            Avg: 2.4312ms (-1.37%)      Median: 2.4312ms (-1.37%)      [2.4312ms .. 2.4312ms]
terms_status_with_date_histogram_and_sibling_terms    Memory: 143.3 KB            Avg: 3.8401ms (+0.44%)      Median: 3.8401ms (+0.44%)      [3.8401ms .. 3.8401ms]
terms_zipf_1000                                       Memory: 75.8 KB             Avg: 2.1611ms (-2.70%)      Median: 2.1611ms (-2.70%)      [2.1611ms .. 2.1611ms]
terms_zipf_1000_with_histogram                        Memory: 1.2 MB              Avg: 20.1369ms (-1.16%)     Median: 20.1369ms (-1.16%)     [20.1369ms .. 20.1369ms]
terms_zipf_1000_with_avg_sub_agg                      Memory: 472.7 KB            Avg: 8.5851ms (-4.93%)      Median: 8.5851ms (-4.93%)      [8.5851ms .. 8.5851ms]
terms_many_json_mixed_type_with_avg_sub_agg           Memory: 17.9 MB             Avg: 26.3988ms (-6.61%)     Median: 26.3988ms (-6.61%)     [26.3988ms .. 26.3988ms]
```
 2026-07-03 17:44:44 +02:00
trinity.pointard 715590b357 rename local var 2026-07-02 12:00:00 +00:00
trinity.pointard d496e402ca rustfmt 2026-07-02 07:12:04 +00:00
trinity.pointard 348ca1e309 don't count matching doc twice 2026-06-30 16:09:11 +00:00
trinity.pointard 5e4fe3520c better handle sorted buckets 2026-06-30 14:56:24 +00:00
trinity.pointard 74a510cb56 try to use select-nth instead of full sort in segment level agg top-k selection 2026-06-29 09:13:21 +00:00
Luca CominardiandCursor 910861a3e9 feat: add custom doc id mapping finalization 
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-26 14:45:57 +02:00
Pascal Seitz 1e859fd78d fix term aggregation u32::MAX overflow issue 2026-06-18 17:07:43 +08:00
Pascal Seitz c096b2ad89 aggregation/terms: charge fused term_counts to the memory limit 
term_counts (one u32/term) was allocated but not charged to
AggregationLimitsGuard, so a memory limit could be exceeded silently.
Charge it, skip allocating it when unbounded, and add a regression test.
 2026-06-16 21:23:23 +08:00
Pascal Seitz ac7a3d347c add comment, hoist variables 2026-06-16 21:23:23 +08:00
Pascal Seitz 03520a0719 add top level comment 2026-06-16 21:23:23 +08:00
Pascal Seitz 86a4c47bed merge loops, histo with bounds may benefit from single vec opt 2026-06-16 21:23:23 +08:00
Pascal Seitz 3ca510dff0 aggregation/terms: tidy fused term×histogram grid construction 
Rename the value threaded through build_segment_term_collector and
maybe_build_collector from max_term_id to col_max_val/max_column_val — it
is the column's max value, only later reused as the max term id. Make the
grid-size arithmetic overflow-/zero-safe (saturating_add, checked_div).
 2026-06-16 21:23:23 +08:00
Pascal Seitz 3cb400c300 clarify counts/term_counts field docs 
Spell out that `counts` is the flattened per-term × time-bucket grid (each
term's own contiguous slice) and that `term_counts` is only needed when the
per-term total can't be derived from that grid (i.e. with hard bounds).
 2026-06-16 21:23:23 +08:00
Pascal Seitz ef13489d63 skip hard_bounds that can't exclude any value 
When a histogram's hard_bounds are wider than the column's value range, the
per-doc `bounds.contains` check can never fail. Collapse such bounds to the
unbounded sentinel in `normalize_histogram_req`, so both the general histogram
hot loop and the fused term×histogram path skip the check — the latter then
derives per-term counts from the grid (the ~17% win) instead of falling back to
per-doc counting just because `bounds != [MIN, MAX]`.

Only the collect-time filter is affected: empty-bucket emission reads
`req.hard_bounds` directly, and hard_bounds only ever clips that range, so a
wider-than-data bound leaves results unchanged. Covered by new tests on the
general and fused paths, including mid-interval (bucket-splitting) bounds.

Also tighten the fused-path u32-overflow guard to bound on `num_vals()` (the
per-value increment count) rather than `num_docs()`, and document why the fused
collector's hot-loop fields are hoisted into locals (re-reading them from memory
each iteration measured ~15% slower).
 2026-06-16 21:23:23 +08:00
Pascal Seitz 9f7aea4765 derive term counts 2026-06-16 21:23:23 +08:00
Pascal Seitz 2c8536ab11 add specialized TermHistogram 2026-06-16 21:23:23 +08:00
Pascal Seitz 05f4c02ac5 add dense histogram, optional sub-buckets 2026-06-16 21:23:23 +08:00
Pascal Seitz d137779219 add no sub-gg fastpath 2026-06-16 21:23:23 +08:00