PSeitz
646e41bec4
Merge pull request #1083 from PSeitz/termdict_block_layout
...
Move counting writer to common
2021-06-16 08:57:55 +02:00
Pascal Seitz
36528c5e83
move counting writer to common
...
move counting writer to common
reuse counting writer in fastfield codec
2021-06-16 08:14:04 +02:00
PSeitz
b5cc60f80b
Merge pull request #1080 from PSeitz/more_tests
...
test all features in github actions
2021-06-15 10:51:57 +02:00
Pascal Seitz
060b83159a
use nightly for tests
2021-06-15 10:08:49 +02:00
Pascal Seitz
a40ff35453
test all features
2021-06-15 09:31:39 +02:00
PSeitz
268e6bfe6e
update fast field codec readme
2021-06-15 09:19:39 +02:00
PSeitz
f902440b8b
Merge pull request #1072 from PSeitz/fastfieldcompression
...
Enable support for multiple fastfield codecs, add linear interpolation
2021-06-15 09:07:17 +02:00
Pascal Seitz
77a0902605
replace unwrap, use vec in bench
2021-06-14 17:01:46 +02:00
Pascal Seitz
c889ae10e4
add is_applicable to fast field codecs
2021-06-14 16:16:25 +02:00
Pascal Seitz
0a534c6ee0
rename create to serialize
2021-06-14 15:40:07 +02:00
Pascal Seitz
167d88b449
fix tests behind unstable feature flag
2021-06-14 15:31:12 +02:00
Pascal Seitz
1071ed84f2
fix cond compilation
2021-06-14 14:05:04 +02:00
Pascal Seitz
abb5624af2
add contributing guidelines, add codec comparer binary
...
add contributing guidelines
add codec comparer binary to test codec compressions with different test data sets
2021-06-14 13:56:40 +02:00
Pascal Seitz
1d41b96d32
rename, add codec_tester
2021-06-14 13:56:40 +02:00
Pascal Seitz
ef4665945f
rename file
2021-06-14 13:56:40 +02:00
Pascal Seitz
294cd5fd0b
streamline traits and tests
2021-06-14 13:56:40 +02:00
Pascal Seitz
f4d271177c
add inline, add readme
2021-06-14 13:56:40 +02:00
Pascal Seitz
451538fecf
add serialize for bool
2021-06-14 13:56:40 +02:00
Pascal Seitz
e78e0fec59
add multilinearinterpolation
...
add multilinearinterpolation, which compresses blocks of size 512
add checks for linear interpolation
2021-06-14 13:56:40 +02:00
Pascal Seitz
2e639cebf8
fix bitpacker bug, reset internal value
2021-06-14 13:56:40 +02:00
Pascal Seitz
e296da7ade
add debug and failsafes
2021-06-14 13:56:40 +02:00
Pascal Seitz
3b3e26c4b8
use f64 precision for slope calculation
2021-06-14 13:56:40 +02:00
Pascal Seitz
6a4883ac69
use uniform distribution sampling
2021-06-14 13:56:40 +02:00
Pascal Seitz
0ba05df545
add f32::MAX to disable a compressor
2021-06-14 13:56:40 +02:00
Pascal Seitz
aa3c4d4029
use f32 precision, add inline
2021-06-14 13:56:40 +02:00
Pascal Seitz
60df629725
cargo.toml license desc and author
2021-06-14 13:56:40 +02:00
Pascal Seitz
2570b005ac
fix estimation test
2021-06-14 13:56:40 +02:00
Pascal Seitz
d5212cd19d
fix clippy
2021-06-14 13:56:40 +02:00
Pascal Seitz
2193d85622
fix clippy and common crate tests
2021-06-14 13:56:40 +02:00
Pascal Seitz
dfdbfe9eff
add benchmark for fast field codecs
...
test tests::bench_fastfield_bitpack_create ... bench: 57,628 ns/iter (+/- 23,486)
test tests::bench_fastfield_bitpack_get ... bench: 43,323 ns/iter (+/- 4,286)
test tests::bench_fastfield_linearinterpol_create ... bench: 223,625 ns/iter (+/- 33,563)
test tests::bench_fastfield_linearinterpol_get ... bench: 82,839 ns/iter (+/- 9,575)
2021-06-14 13:56:40 +02:00
Pascal Seitz
b999e836b2
replace BitpackedFastFieldReader, delete FastFieldSerializer trait
2021-06-14 13:56:40 +02:00
Pascal Seitz
be2dd41e69
add interface to create and read codecs
...
add CodecReader as common interface in fastfield codec crate
add LinearInterpolation to DynamicFastFieldReader
calc estimation and choose best codec
cleanup
2021-06-14 13:56:40 +02:00
Pascal Seitz
483fdb79cc
add linear interpolation estimation
...
add estimation tests
add codec test data in tests
2021-06-14 13:56:40 +02:00
Pascal Seitz
aefd0fc907
refactor, add fastfield metadata to footer
...
change api to fastfield reader in codec crate
add fastfield metadata to footer
remove old code
merge codec files
2021-06-14 13:56:40 +02:00
Pascal Seitz
3298d6cb71
move common to common crate, create fastfield_codecs crate
...
move common to common crate
create fastfield_codecs crate
add bitpacker to fast field codecs
add linear interpolation to fast field codecs
add tests
2021-06-14 13:56:40 +02:00
Pascal Seitz
c02c78ea73
implement linear interpol serializer
2021-06-14 13:56:40 +02:00
Pascal Seitz
6bf4fee1ba
support multiple codecs
...
support multiple codes
prepend codec id to all fast fields
add new api to create fastfields with access to all data
use new fastfield creation api in initial creation and merge
remove unused collect of data in doc_id_mapping
2021-06-14 13:56:40 +02:00
PSeitz
5209238c1b
use github actions for tests
2021-06-14 12:51:46 +02:00
Paul Masurel
7ef25ec400
Bump to 0.15.1 to publish bugfix
2021-06-14 18:45:38 +09:00
PSeitz
221e7cbb55
Merge pull request #1076 from appaquet/fix/store-reader-iterator
...
Fix panic in store reader raw document iterator during segment merge
2021-06-14 11:22:58 +02:00
Pascal Seitz
873ac1a3ac
cleanup import
2021-06-14 10:31:45 +02:00
Pascal Seitz
ebe55a7ae1
refactor test, fixes #1077
...
replace test with smaller test in doc_store
2021-06-14 10:10:05 +02:00
Bernard Swart
9f32d40b27
Misspelling of misspelled was fixed ( #1078 )
2021-06-14 16:29:12 +09:00
Andre-Philippe Paquet
8ae10a930a
fix formatting
2021-06-13 17:23:40 -04:00
Andre-Philippe Paquet
473a346814
remove debugging
2021-06-13 16:49:44 -04:00
Andre-Philippe Paquet
3a8a0fe79a
add fuzzy merge test
2021-06-13 16:42:24 -04:00
Andre-Philippe Paquet
511dc8f87f
fix store reader iterator
2021-06-13 16:00:13 -04:00
Paul Masurel
3901295329
Bumped query-grammar version
2021-06-07 10:00:14 +09:00
Paul Masurel
f5918c6c74
Completed bitpacker README
2021-06-07 09:57:17 +09:00
Paul Masurel
abe6b4baec
Bumped tantivy version to 0.15
2021-06-07 09:52:48 +09:00
