Tantivy used to assume that all files could be somehow memory mapped. After this change, Directory return a `FileSlice` that can be reduced and eventually read into an `OwnedBytes` object. Long and blocking io operation are still required by they do not span over the entire file.
* add support for indexed bytes fast field
* remove backup code file
* refine test cases
* Simplified unit test. Renamed it as it is testing the storable part. Not the indexed part.
* Small refactoring and added unit test. If multivalued we only retain the first FAST value.
Co-authored-by: Raul <raul.tang.lc@gmail.com>
* Unsatisfactory implementation.
The fastfield are hit. But for performance, we want the comparison to happen on u64,
and the conversion to the FastType to be done only on the selected TopK
elements.
For i64, the current approach might be ok.
For DateTime, it is most likely catastrophic.
Closes#822
* Decoupled SegmentCollector Fruit from Collector Fruit.
Deferred conversion from u64 to the proper FastField type to after the overall collection.
(tantivy guarantees that u64 encoding is consistent with the original
ordering of the fastfield)
Closes#882
* Alternative take on boosted queries
* Fixing unit test
* Added boosting to the query grammar.
* Made BoostQuery public.
* Added support for boosting field in QueryParser
Closes#547
* Prevent tokens from being stored in the document store.
Commit adds prepare_for_store method to Document, which changes all
PreTokenizedString values into String values. The method is called
before adding document to the document store to prevent tokens from
being saved there. Commit also adds small changes to comments in
pre_tokenized_text example.
* Avoid storing the pretokenized text.
* Added handling of pre-tokenized text fields (#642).
* * Updated changelog and examples concerning #642.
* Added tokenized_text method to Value implementation.
* Implemented From<TokenizedString> for TokenizedStream.
* * Removed tokenized flag from TextOptions and code reliance on the flag.
* Changed naming to use word "pre-tokenized" instead of "tokenized".
* Updated example code.
* Fixed comments.
* Minor code refactoring. Test improvements.
* add basic support for float
as for i64, they are mapped to u64 for indexing
query parser don't work yet
* Update value.rs
* implement support for float in query parser
* Update README.md
* initial version, still a work in progress
* remove redudant or
* add chrono::DateTime and index i64
* add more tests
* fix tests
* pass DateTime by ptr
* remove println!
* document query_parser rfc 3339 date support
* added some more docs about implementation to schema.rs
* enforce DateTime is UTC, and re-export chrono
* added DateField to changelog
* fixed conflict
* use INDEXED instead of INT_INDEXED for date fields
