PSeitz
d09d91a856
fix tests ( #1813 )
2023-01-19 23:41:21 +09:00
PSeitz
1176555eff
handle user input on get_docid_for_value_range ( #1760 )
...
* handle user input on get_docid_for_value_range
fixes #1757
* pass range as parameter
2023-01-12 14:20:16 +01:00
PSeitz
7c6cc818ae
enable range query on fast field for u64 compatible types ( #1762 )
...
* enable range query on fast field for u64 compatible types
* rename, update benches
2023-01-10 04:08:26 +01:00
Paul Masurel
4f9efe654c
Support for columnar ( #1734 )
...
* Added support for dynamic fast field.
See README for more information.
* Apply suggestions from code review
Co-authored-by: PSeitz <PSeitz@users.noreply.github.com >
2023-01-07 17:37:00 +09:00
pinkforest(she/her)
b78dc5e313
Bump prettytables ( #1746 )
2022-12-31 15:01:39 +01:00
Paul Masurel
9948a84ebe
Simplifies the count_ones definition. ( #1742 )
2022-12-26 16:08:01 +09:00
PSeitz
45156fd869
use group_by in translate_codec_idx_to_original_id ( #1736 )
2022-12-26 06:13:29 +01:00
Paul Masurel
bc959006fa
Ooops. Removing ordered_floats.
2022-12-22 19:50:34 +09:00
Paul Masurel
7385a8f80c
Supporting PartialCmp in VectorColumn. ( #1735 )
...
* Supporting PartialCmp in VectorColumn.
* Apply suggestions from code review
Co-authored-by: PSeitz <PSeitz@users.noreply.github.com >
2022-12-22 17:47:25 +09:00
Paul Masurel
13b89cba17
Adding inlines.
2022-12-22 14:29:41 +09:00
Paul Masurel
540a9972bd
Support for NotNaN in fast fields
2022-12-22 12:28:25 +09:00
Paul Masurel
f39165e1e7
Moving FileSlice to tantivy-common ( #1729 )
2022-12-21 16:35:11 +09:00
Paul Masurel
4a6bf50e78
Clippy
2022-12-21 15:43:34 +09:00
PSeitz
2ac1cc2fc0
add sparse codec ( #1723 )
...
* add sparse codec
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
* add the -1 u16 fix for metadata num_vals
* add dense block encoding to sparse codec
* add comment, refactor u16 reading
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-12-20 15:30:33 +01:00
PSeitz
f9171a3981
fix clippy ( #1725 )
...
* fix clippy
* fix clippy fastfield codecs
* fix clippy bitpacker
* fix clippy common
* fix clippy stacker
* fix clippy sstable
* fmt
2022-12-20 07:30:06 +01:00
PSeitz
a2cf6a79b4
Sparse dense index ( #1716 )
...
* add dense codec
* benchmark fix and important optimisation
* move code to DenseIndexBlock
improve benchmark
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
* extend benchmarks
* Apply suggestions from code review
Co-authored-by: Paul Masurel <paul@quickwit.io >
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-12-13 07:50:09 +01:00
PSeitz
3cdc8e7472
pass index info to serialize ( #1719 )
2022-12-13 04:20:31 +01:00
PSeitz
509adab79d
Bump version ( #1715 )
...
* group workspace deps
* update cargo.toml
* revert tant version
* chore: Release
2022-12-12 04:39:43 +01:00
PSeitz
1119e59eae
prepare fastfield format for null index ( #1691 )
...
* prepare fastfield format for null index
* add format version for fastfield
* Update fastfield_codecs/src/compact_space/mod.rs
* switch to variable size footer
* serialize delta of end
2022-11-28 17:15:24 +09:00
Pascal Seitz
8641155cbb
remove column from MultiValuedU128FastFieldReader
2022-11-14 18:49:15 +08:00
Pascal Seitz
f811d1616b
add support for ip range query on multivalue fastfields
2022-11-14 14:21:52 +08:00
Pascal Seitz
32166682b3
add header deser test
2022-11-11 13:28:12 +08:00
Pascal Seitz
e6acf8f76d
add header with codec type for u128
2022-11-11 11:52:17 +08:00
PSeitz
a5a80ffaea
Update fastfield_codecs/src/column.rs
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-11-02 06:37:27 +01:00
Pascal Seitz
83325d8f3f
move multivalue index to own file
...
start_doc parameter in positions to docids
2022-11-01 10:36:13 +08:00
Pascal Seitz
6213ea476a
pass positions parameter
2022-10-25 17:44:51 +08:00
Pascal Seitz
e772d3170d
switch get_val() to u32
...
Fixes #1638
2022-10-24 19:05:57 +08:00
Pascal Seitz
6bb73a527f
add range query via ip fast field
2022-10-24 16:00:38 +08:00
Pascal Seitz
791350091c
switch num_vals() to u32
...
fixes #1630
2022-10-20 19:44:28 +08:00
Pascal Seitz
952b048341
add term aggregation clarification
2022-10-14 16:12:19 +08:00
PSeitz
11d3409286
add missing docs for fastfield_codecs crate ( #1613 )
...
closes #1603
2022-10-11 18:54:24 +09:00
Pascal Seitz
b2ca83a93c
switch to ipv6, add monotonic_mapping tests
2022-10-07 18:47:55 +08:00
Pascal Seitz
b9b913510e
fmt
2022-10-07 16:56:19 +08:00
PSeitz
f465173872
Apply suggestions from code review
...
Co-authored-by: Paul Masurel <paul@quickwit.io >
2022-10-07 16:55:53 +08:00
Pascal Seitz
9a1609d364
add test
2022-10-07 16:25:01 +08:00
Pascal Seitz
39f4e58450
improve comment
2022-10-07 16:25:01 +08:00
Pascal Seitz
a8a36b62cd
enable test
2022-10-07 16:25:01 +08:00
Pascal Seitz
226a49338f
add StrictlyMonotonicFn
2022-10-07 16:25:01 +08:00
Pascal Seitz
cdc8e3a8be
group montonic mapping and inverse
...
fix mapping inverse
remove ip indexing
add get_between_vals test
2022-10-07 16:25:01 +08:00
Pascal Seitz
67f453b534
rename to iter_gen
2022-10-07 16:25:01 +08:00
Pascal Seitz
eeb1f19093
rename to iter_gen
2022-10-07 16:25:01 +08:00
Pascal Seitz
5a76e6c5d3
fix get_between_vals forwarding
...
fix get_between_vals forwarding in monotonicmapping column by adding an additional conversion function Output->Input
2022-10-07 16:25:01 +08:00
Pascal Seitz
c8713a01ed
use iter api
2022-10-07 16:25:01 +08:00
Pascal Seitz
400a20b7af
add ip field
...
add u128 multivalue reader and writer
add ip to schema
add ip writers, handle merge
2022-10-07 16:25:01 +08:00
Pascal Seitz
0f5cff762f
move enumerate and remove computation
2022-10-04 12:30:19 +08:00
Pascal Seitz
6d9a123cf2
remove get_val in serialization
...
remove get_val in serialization and mark as unimplemented!()
replace get_val with iter in linear codec
remove MultivalueStartIndexRandomSeeker
replace MultivalueStartIndexIter with closure
Sample 100 values in linear codec
2022-10-04 12:01:25 +08:00
Bruce Mitchener
cb252a42af
docs: "associated to" -> "associated with" ( #1557 )
...
This reads better this way.
2022-09-26 20:23:37 +09:00
Pascal Seitz
f757471077
prepare for ip field
2022-09-26 16:27:35 +08:00
PSeitz
20c87903b2
fix multivalue ff index creation regression ( #1543 )
...
fixes multivalue ff regression by avoiding using `get_val`. Line::train calls repeatedly get_val, but get_val implementation on Column for multivalues is very slow. The fix is to use the iterator instead. Longterm fix should be to remove get_val access in serialization.
Old Code
test fastfield::bench::bench_multi_value_ff_merge_few_segments ... bench: 46,103,960 ns/iter (+/- 2,066,083)
test fastfield::bench::bench_multi_value_ff_merge_many_segments ... bench: 83,073,036 ns/iter (+/- 4,373,615)
est fastfield::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 64,178,576 ns/iter (+/- 1,466,700)
Current
running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments ... bench: 57,379,523 ns/iter (+/- 3,220,787)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments ... bench: 90,831,688 ns/iter (+/- 1,445,486)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 158,313,264 ns/iter (+/- 28,823,250)
With Fix
running 3 tests
test fastfield::multivalued::bench::bench_multi_value_ff_merge_few_segments ... bench: 57,635,671 ns/iter (+/- 2,707,361)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments ... bench: 91,468,712 ns/iter (+/- 11,393,581)
test fastfield::multivalued::bench::bench_multi_value_ff_merge_many_segments_log_merge ... bench: 73,909,138 ns/iter (+/- 15,846,097)
2022-09-23 15:36:29 +09:00
Bruce Mitchener
e9a384bb15
Use u8::from(bool), u64::from(bool).
2022-09-22 22:44:53 +07:00
