rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-09-05 21:09:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,549 Commits 261 Branches 81 Tags
dependabot/github_actions/github/codeql-action-4.36.2
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Pascal Seitz edfb02b47e switch to enum, fix mixed types for cardinality agg 2026-05-05 16:39:51 +08:00
Pascal Seitz d0fad88bac use bitsets for card agg 2026-05-05 16:39:51 +08:00
Pascal Seitz 351280c0b4 add card bench for high card 2026-05-05 16:39:51 +08:00
Pascal Seitz d47abdf104 early cut off for order by sub agg in term agg 2026-04-28 16:59:59 +02:00
Pascal Seitz c11952eb7c add order by agg benchmark 2026-04-28 16:59:59 +02:00
Pascal Seitz 2e16243f9a fix memory consumption for histogram 2026-04-21 13:58:39 +02:00
Pascal Seitz e015abab8e docs: add 0.26.1 changelog entry for aggregation perf fix 2026-04-21 11:12:37 +02:00
Pascal Seitz 73c711ec74 perf(agg): only measure active parent bucket in composite collect 
Same change as 26a589e for SegmentCompositeCollector: get_memory_consumption
summed across all parent_buckets on every block, scaling with outer bucket
cardinality. Pass parent_bucket_id and index the single bucket.
 2026-04-21 07:26:58 +02:00
Pascal Seitz cb037c8079 add inline 2026-04-21 07:26:58 +02:00
Pascal Seitz ed3453606b agg fix: compute memory consumption only for current bucket 2026-04-21 07:26:58 +02:00
Pascal Seitz e9641f99c5 add nested term benchmark 2026-04-21 07:26:58 +02:00
Pascal Seitz 5c344db1bf chore: Release 2026-03-31 17:15:34 +08:00
Pascal Seitz dc0f31554d unbump for release and update Changelog.md 2026-03-31 17:15:34 +08:00
Pascal Seitz 8641155cbb remove column from MultiValuedU128FastFieldReader 2022-11-14 18:49:15 +08:00
Pascal Seitz b7d0dd154a fmt 2022-11-14 14:49:15 +08:00
Pascal Seitz e034328a8b Improve position_to_docid, refactor, add tests 2022-11-14 14:21:53 +08:00
Pascal Seitz f811d1616b add support for ip range query on multivalue fastfields 2022-11-14 14:21:52 +08:00
Pascal Seitz fb9f03118d switch total_num_val to u32 2022-11-11 17:35:52 +08:00
Pascal Seitz 32166682b3 add header deser test 2022-11-11 13:28:12 +08:00
Pascal Seitz e6acf8f76d add header with codec type for u128 2022-11-11 11:52:17 +08:00
Pascal Seitz 9e8a0c2cca Allow range query on fastfield without INDEXED 2022-11-10 15:56:08 +08:00
Pascal Seitz c69a873dd3 fix num_vals on value index after merge 2022-11-07 21:05:21 +08:00
Pascal Seitz 38ad46e580 fix clippy 2022-11-07 16:09:55 +08:00
Pascal Seitz 6e636c9cea fix num_vals in multivalue index after merge 2022-11-07 15:00:52 +08:00
Pascal Seitz 500a0d5e48 update criterion to 0.4 2022-11-04 13:26:29 +08:00
Pascal Seitz 83325d8f3f move multivalue index to own file 
start_doc parameter in positions to docids
 2022-11-01 10:36:13 +08:00
Pascal Seitz 43df356010 rename to docset 2022-10-27 16:53:38 +08:00
Pascal Seitz 279b1b28d3 switch to fx hashmap 2022-10-27 16:19:59 +08:00
Pascal Seitz dfab201191 for_each_docset to iterate without score 2022-10-26 17:25:05 +08:00
Pascal Seitz af839753e0 No score calls if score is not requested 2022-10-26 12:18:35 +08:00
Pascal Seitz fec2b63571 improve bench by adding more blanks in compact space 2022-10-25 22:09:01 +08:00
Pascal Seitz 6213ea476a pass positions parameter 2022-10-25 17:44:51 +08:00
Pascal Seitz 5e159c26bf add ip range query benchmark, add seek behaviour 2022-10-25 15:57:19 +08:00
Pascal Seitz e772d3170d switch get_val() to u32 
Fixes #1638
 2022-10-24 19:05:57 +08:00
Pascal Seitz 02328b0151 fix proptest 2022-10-24 17:46:06 +08:00
Pascal Seitz 7cc775256c add comments, rename 2022-10-24 17:08:37 +08:00
Pascal Seitz 07b40f8b8b add proptest 2022-10-24 16:52:55 +08:00
Pascal Seitz 6bb73a527f add range query via ip fast field 2022-10-24 16:00:38 +08:00
Pascal Seitz f2e5135870 allow more characters in range query 
closes #1642
 2022-10-21 18:05:15 +08:00
Pascal Seitz 791350091c switch num_vals() to u32 
fixes #1630
 2022-10-20 19:44:28 +08:00
Pascal Seitz f2b2628feb add test for phrase search on multi text field 2022-10-19 16:29:56 +08:00
Pascal Seitz a4485f7611 faster skipindex deserialization, larger blocksize on sort 2022-10-18 19:32:23 +08:00
Pascal Seitz 1082ff60f9 add range query handling for ip via term dictionary 
since IPs are mapped monotonically we can use the term dictionary for range queries
 2022-10-18 13:08:27 +08:00
Pascal Seitz 6800fdec9d add indexing for ip field 
Closes #1595
 2022-10-18 10:07:48 +08:00
Pascal Seitz 024e53a99c remove truncate 2022-10-17 12:14:35 +08:00
Pascal Seitz 8d75e451bd fix truncate, remove mutable access from term 2022-10-17 12:14:35 +08:00
Pascal Seitz fcfd76ec55 refactor Term 
fixes some issues with Term
Remove duplicate calls to truncate or resize
Replace Magic Number 5 with constant
Enforce minimum size of 5 for metadata
Fix broken truncate docs
use constructor instead new + set calls
normalize constructor stack
replace assert on internal behavior fixes #1585
 2022-10-17 12:14:34 +08:00
Pascal Seitz 129f7422f5 remove unused buffer 2022-10-14 20:01:10 +08:00
Pascal Seitz 952b048341 add term aggregation clarification 2022-10-14 16:12:19 +08:00
Pascal Seitz 84f9e77e1d update CHANGELOG 2022-10-14 15:10:33 +08:00
Pascal Seitz 63bc390b02 Fix missing fieldnorm indexing 
Fixes broken search (no results) with BM25 for u64, i64, f64, bool, bytes and date after deletion and merge.
There were no fieldnorms recorded for those field. After merge InvertedIndexReader::total_num_tokens returns 0 (Sum over the fieldnorms is 0). BM25 does not work when total_num_tokens is 0.
Fixes #1617
 2022-10-14 12:44:40 +08:00
Pascal Seitz 9cb8cfbea8 return Error instead panic in fastfields 
fixes #1572
 2022-10-11 14:15:22 +08:00
Pascal Seitz 2efebdb1bb remove tokenstream vec alloc 2022-10-11 10:30:56 +08:00
Pascal Seitz 5c9cbee29d handle IpV4 serialization case 2022-10-07 19:52:00 +08:00
Pascal Seitz b2ca83a93c switch to ipv6, add monotonic_mapping tests 2022-10-07 18:47:55 +08:00
Pascal Seitz b9b913510e fmt 2022-10-07 16:56:19 +08:00
Pascal Seitz 96315df20d use idx part only for positions_to_docid 2022-10-07 16:54:04 +08:00
Pascal Seitz 9a1609d364 add test 2022-10-07 16:25:01 +08:00
Pascal Seitz 39f4e58450 improve comment 2022-10-07 16:25:01 +08:00
Pascal Seitz a8a36b62cd enable test 2022-10-07 16:25:01 +08:00
Pascal Seitz 226a49338f add StrictlyMonotonicFn 2022-10-07 16:25:01 +08:00
Pascal Seitz 2864bf7123 use serializer for u128 2022-10-07 16:25:01 +08:00
Pascal Seitz 5171ff611b serialize ip as u128, add test for positions_to_docid 2022-10-07 16:25:01 +08:00
Pascal Seitz e50e74acf8 remove u128 type 2022-10-07 16:25:01 +08:00
Pascal Seitz 0b86658389 rename ip addr, use buffer 2022-10-07 16:25:01 +08:00
Pascal Seitz 5d6602a8d9 mark null handling TODO 2022-10-07 16:25:01 +08:00
Pascal Seitz 4d29ff4d01 finalize ip addr rename 2022-10-07 16:25:01 +08:00
Pascal Seitz cdc8e3a8be group montonic mapping and inverse 
fix mapping inverse
remove ip indexing
add get_between_vals test
 2022-10-07 16:25:01 +08:00
Pascal Seitz 67f453b534 rename to iter_gen 2022-10-07 16:25:01 +08:00
Pascal Seitz 787a37bacf expect instead of unwrap 2022-10-07 16:25:01 +08:00
Pascal Seitz f5039f1846 remove roaring 2022-10-07 16:25:01 +08:00
Pascal Seitz eeb1f19093 rename to iter_gen 2022-10-07 16:25:01 +08:00
Pascal Seitz 087beaf328 remove null handling 2022-10-07 16:25:01 +08:00
Pascal Seitz 309449dba3 rename to IpAddr 2022-10-07 16:25:01 +08:00
Pascal Seitz 5a76e6c5d3 fix get_between_vals forwarding 
fix get_between_vals forwarding in monotonicmapping column by adding an additional conversion function Output->Input
 2022-10-07 16:25:01 +08:00
Pascal Seitz c8713a01ed use iter api 2022-10-07 16:25:01 +08:00
Pascal Seitz 6113e0408c remove comment 2022-10-07 16:25:01 +08:00
Pascal Seitz 400a20b7af add ip field 
add u128 multivalue reader and writer
add ip to schema
add ip writers, handle merge
 2022-10-07 16:25:01 +08:00
Pascal Seitz 516e60900d remove unwrap 2022-10-07 14:22:37 +08:00
Pascal Seitz 36e1c79f37 replace cbor with cborium 
closes #1526
 2022-10-07 13:23:39 +08:00
Pascal Seitz d742275048 renames 2022-10-05 19:16:49 +08:00
Pascal Seitz 8b42c4c126 disable linear codec for multivalue value index 
don't materialize index column on merge
use simpler chain() variant
 2022-10-05 19:09:17 +08:00
Pascal Seitz f60a551890 add flat_map_with_buffer to Iterator trait 2022-10-05 17:44:26 +08:00
Pascal Seitz 0f5cff762f move enumerate and remove computation 2022-10-04 12:30:19 +08:00
Pascal Seitz 6d9a123cf2 remove get_val in serialization 
remove get_val in serialization and mark as unimplemented!()
replace get_val with iter in linear codec
remove MultivalueStartIndexRandomSeeker
replace MultivalueStartIndexIter with closure
Sample 100 values in linear codec
 2022-10-04 12:01:25 +08:00
Pascal Seitz b062ab2196 use groupby instead of vec allocation 2022-10-04 09:26:26 +08:00
Pascal Seitz a695edcc95 remove dead indexing code 2022-10-03 09:44:02 +08:00
Pascal Seitz 0e94213af0 validate index settings on create 2022-09-29 18:58:09 +08:00
Pascal Seitz 8f647b817f fix docstore settings for temp docstore 
fixes #1565
 2022-09-28 17:53:59 +08:00
Pascal Seitz f757471077 prepare for ip field 2022-09-26 16:27:35 +08:00
Pascal Seitz 22e56aaee3 add benchmarks for multivalued fastfield merge 2022-09-22 11:25:41 +08:00
Pascal Seitz 1ff5da5eb4 remove fast_field_cardinality from FastValue 
unused and at the wrong placed
 2022-09-21 09:38:46 +08:00
Pascal Seitz a06039dea8 fix benches 
move some benches to lib.rs to test unexported items
 2022-09-19 11:07:20 +08:00
Pascal Seitz 02599ebeb7 remove ip_to_u128 2022-09-16 18:16:16 +08:00
Pascal Seitz a16b466460 merge ColumnExt with Column trait 2022-09-16 18:15:18 +08:00
Pascal Seitz b8d8fdeb6e move benches, improve bench data 2022-09-16 16:42:23 +08:00
Pascal Seitz 12856d80fa change bench, update numbers 2022-09-16 16:41:01 +08:00
Pascal Seitz e75472ec9a add serialize_u128, open_u128, refactor 2022-09-16 16:40:59 +08:00
Pascal Seitz e2e6c94ba8 remove ColumnV2 2022-09-16 16:40:06 +08:00
Pascal Seitz 9f610b25af fix benches, add benches 2022-09-16 16:38:48 +08:00