rust/tantivy
1
0
Fork 0
mirror of https://github.com/quickwit-oss/tantivy.git synced 2026-06-02 08:30:41 +00:00
Code Issues Packages Projects Releases Wiki Activity

Collector refactoring and multithreaded search (#437)

Browse Source 
* Split Collector into an overall Collector and a per-segment SegmentCollector. Precursor to cross-segment parallelism, and as a side benefit cleans up any per-segment fields from being Option<T> to just T.

* Attempt to add MultiCollector back

* working. Chained collector is broken though

* Fix chained collector

* Fix test

* Make Weight Send+Sync for parallelization purposes

* Expose parameters of RangeQuery for external usage

* Removed &mut self

* fixing tests

* Restored TestCollectors

* blop

* multicollector working

* chained collector working

* test broken

* fixing unit test

* blop

* blop

* Blop

* simplifying APi

* blop

* better syntax

* Simplifying top_collector

* refactoring

* blop

* Sync with master

* Added multithread search

* Collector refactoring

* Schema::builder

* CR and rustdoc

* CR comments

* blop

* Added an executor

* Sorted the segment readers in the searcher

* Update searcher.rs

* Fixed unit testst

* changed the place where we have the sort-segment-by-count heuristic

* using crossbeam::channel

* inlining

* Comments about panics propagating

* Added unit test for executor panicking

* Readded default

* Removed Default impl

* Added unit test for executor
This commit is contained in:
Paul Masurel
2018-11-30 22:46:59 +09:00
committed by GitHub
parent 8b0b0133dd
commit 07d87e154b
66 changed files with 2111 additions and 1530 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
src/query/boolean_query/mod.rs
View File

@@ -20,9 +20,10 @@ mod tests {
use query::TermQuery;
use schema::*;
use Index;
use DocId;
fn aux_test_helper() -> (Index, Field) {
let mut schema_builder = SchemaBuilder::default();
let mut schema_builder = Schema::builder();
let text_field = schema_builder.add_text_field("text", TEXT);
let schema = schema_builder.build();
let index = Index::create_in_ram(schema);
@@ -130,9 +131,12 @@ mod tests {
let matching_docs = |boolean_query: &Query| {
let searcher = index.searcher();
let mut test_collector = TestCollector::default();
searcher.search(boolean_query, &mut test_collector).unwrap();
test_collector.docs()
let test_docs = searcher.search(boolean_query, &TestCollector).unwrap();
test_docs.docs()
.iter()
.cloned()
.map(|doc| doc.1)
.collect::<Vec<DocId>>()
};
{
@@ -186,9 +190,8 @@ mod tests {
let score_docs = |boolean_query: &Query| {
let searcher = index.searcher();
let mut test_collector = TestCollector::default();
searcher.search(boolean_query, &mut test_collector).unwrap();
test_collector.scores()
let fruit = searcher.search(boolean_query, &TestCollector).unwrap();
fruit.scores().to_vec()
};
{