rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-05-19 01:30:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,574 Commits 241 Branches 81 Tags
fix_estimate
Commit Graph

2574 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pascal Seitz
ba3215b469 reuse samples, add EstimateColumn 
estimations can be expensive since the samples span the whole column
and depending on the implementation get_val can not be easily computed
without an index.
EstimateColumn adds a view over the column which limits num_vals
to 100_000.
 2022-09-25 23:54:03 +08:00
PSeitz
dac7da780e Merge pull request #1545 from waywardmonkeys/remove-some-refs 
clippy: Remove borrows that the compiler will do.
 2022-09-23 15:33:23 +08:00
PSeitz
20c87903b2 fix multivalue ff index creation regression (#1543) 
fixes multivalue ff regression by avoiding using `get_val`. Line::train calls repeatedly get_val, but get_val implementation on Column for multivalues is very slow. The fix is to use the iterator instead. Longterm fix should be to remove get_val access in serialization.

Old Code

test fastfield::bench::bench_multi_value_ff_merge_few_segments                                                           ... bench:  46,103,960 ns/iter (+/- 2,066,083)
test fastfield::bench::bench_multi_value_ff_merge_many_segments                                                          ... bench:  83,073,036 ns/iter (+/- 4,373,615)
est fastfield::bench::bench_multi_value_ff_merge_many_segments_log_merge                                                ... bench:  64,178,576 ns/iter (+/- 1,466,700)

Current

running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments                                              ... bench:  57,379,523 ns/iter (+/- 3,220,787)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments                                             ... bench:  90,831,688 ns/iter (+/- 1,445,486)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge                                   ... bench: 158,313,264 ns/iter (+/- 28,823,250)

With Fix

running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments                                              ... bench:  57,635,671 ns/iter (+/- 2,707,361)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments                                             ... bench:  91,468,712 ns/iter (+/- 11,393,581)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge                                   ... bench:  73,909,138 ns/iter (+/- 15,846,097)
 2022-09-23 15:36:29 +09:00
PSeitz
f9c3947803 Merge pull request #1546 from waywardmonkeys/use-ux-from-bool 
Use u8::from(bool), u64::from(bool).
 2022-09-23 09:06:24 +08:00
Bruce Mitchener
e9a384bb15 Use u8::from(bool), u64::from(bool). 2022-09-22 22:44:53 +07:00
Bruce Mitchener
d231671fe2 clippy: Remove borrows that the compiler will do. 
This started showing up with clippy in rust 1.64.
 2022-09-22 22:38:23 +07:00
trinity-1686a
fa3d786a2f Add support for deleting all documents matching query (#1535) 
* add support for deleting all documents matching query

#1494
 2022-09-22 21:26:09 +09:00
Paul Masurel
75aafeeb9b Added a function to deep clone RamDirectory. (#1544) 2022-09-22 12:04:02 +02:00
PSeitz
6f066c7f65 Merge pull request #1541 from quickwit-oss/add_bench 
add benchmarks for multivalued fastfield merge
 2022-09-22 15:28:00 +08:00
Pascal Seitz
22e56aaee3 add benchmarks for multivalued fastfield merge 2022-09-22 11:25:41 +08:00
Paul Masurel
d641979127 Minor refactor of fast fields (#1538) 2022-09-21 12:55:03 +09:00
Paul Masurel
1998111521 Minor refactoring fast fields (#1537) 2022-09-21 12:46:11 +09:00
PSeitz
acb2e2e282 Merge pull request #1532 from quickwit-oss/refactor_ff 
remove fast_field_cardinality from FastValue
 2022-09-21 04:00:35 +02:00
Pascal Seitz
1ff5da5eb4 remove fast_field_cardinality from FastValue 
unused and at the wrong placed
 2022-09-21 09:38:46 +08:00
Bruce Mitchener
c3b25710ad doc: Improve directory::Lock docs. (#1534) 
Update the docs to reflect the lack of LockParams, correct an error,
and improve cross-linking.
 2022-09-20 18:03:35 +09:00
PSeitz
8492010d43 Merge pull request #1531 from waywardmonkeys/improve-docs-more 
Improvements to doc linking, grammar, etc.
 2022-09-20 15:37:07 +08:00
Bruce Mitchener
cf02e32578 Improvements to doc linking, grammar, etc. 2022-09-19 18:10:22 +07:00
PSeitz
8cca1014c9 Merge pull request #1527 from waywardmonkeys/remove-stream_field-reference 
docs: Remove mentions of stream_field method.
 2022-09-19 17:16:46 +08:00
PSeitz
938f884e32 Merge pull request #1525 from waywardmonkeys/fix-etsy-logo-alt-text-readme 
README: Fix Etsy logo and alt text.
 2022-09-19 16:55:08 +08:00
PSeitz
ed68afb698 Merge pull request #1528 from quickwit-oss/ff_refact 
fix benches
 2022-09-19 11:37:08 +08:00
PSeitz
8a7962dc22 Merge pull request #1524 from waywardmonkeys/improve-docs-1 
Documentation improvements.
 2022-09-19 11:15:42 +08:00
Pascal Seitz
a06039dea8 fix benches 
move some benches to lib.rs to test unexported items
 2022-09-19 11:07:20 +08:00
Bruce Mitchener
68b6254b09 docs: Remove mentions of stream_field method. 
This method doesn't exist, so no need to mention it.
 2022-09-18 23:13:41 +07:00
Bruce Mitchener
6a88ac3fe3 Documentation improvements. 
Fix some linking, some grammar, some typos, etc.
 2022-09-18 18:05:37 +07:00
Bruce Mitchener
191b934650 README: Fix Etsy logo and alt text. 2022-09-18 15:02:35 +07:00
PSeitz
1a2ba7025a Merge pull request #1513 from quickwit-oss/ip_codec 
add ip codec
 2022-09-16 18:53:08 +08:00
Pascal Seitz
02599ebeb7 remove ip_to_u128 2022-09-16 18:16:16 +08:00
Pascal Seitz
a16b466460 merge ColumnExt with Column trait 2022-09-16 18:15:18 +08:00
Pascal Seitz
b8d8fdeb6e move benches, improve bench data 2022-09-16 16:42:23 +08:00
Pascal Seitz
12856d80fa change bench, update numbers 2022-09-16 16:41:01 +08:00
Pascal Seitz
e75472ec9a add serialize_u128, open_u128, refactor 2022-09-16 16:40:59 +08:00
Pascal Seitz
e2e6c94ba8 remove ColumnV2 2022-09-16 16:40:06 +08:00
Pascal Seitz
9f610b25af fix benches, add benches 2022-09-16 16:38:48 +08:00
Pascal Seitz
237b64025e take ColumnV2 as parameter 
improve algorithm
stricter assertions
improve names
 2022-09-16 16:38:48 +08:00
Pascal Seitz
592caeefa0 renames 2022-09-16 16:38:48 +08:00
Pascal Seitz
570009b5b1 move to mod.rs 2022-09-16 16:38:48 +08:00
Pascal Seitz
61b5110db7 use 0 as null in compact space 2022-09-16 16:38:48 +08:00
PSeitz
58af1235e4 Apply suggestions from code review 
Co-authored-by: Paul Masurel <paul@quickwit.io>
 2022-09-16 16:38:48 +08:00
Pascal Seitz
d3e7c41a1f refactor to range_mapping 2022-09-16 16:38:48 +08:00
Pascal Seitz
11275854ca unroll get range iteration 2022-09-16 16:38:48 +08:00
Pascal Seitz
3ca48cd826 fix test 2022-09-16 16:38:48 +08:00
Pascal Seitz
47dc511733 add inline 2022-09-16 16:38:48 +08:00
Pascal Seitz
cae6b28a8f remove num_vals param 2022-09-16 16:38:48 +08:00
Pascal Seitz
9aa9efe2a4 fix bench 2022-09-16 16:38:48 +08:00
Pascal Seitz
57570b38a2 use vint, forward errors, removed unused var 2022-09-16 16:38:48 +08:00
Pascal Seitz
584394db1e fix Cargo.toml 2022-09-16 16:38:48 +08:00
Pascal Seitz
3aeb026970 fix blank_size, add comments 2022-09-16 16:38:48 +08:00
Pascal Seitz
df32ee2df2 refactor, use BTreeSet for sorted deduped values 2022-09-16 16:38:48 +08:00
Pascal Seitz
762e662bfd extend proptest for get_range 2022-09-16 16:38:48 +08:00
Pascal Seitz
63b2420058 fix get_range 
change blank handling
optimize blank collection
fix off by one errors
extend tests
fix get_range
dedupe values to save space
add bench
 2022-09-16 16:38:47 +08:00