SegmentReader as trait

.claude/skills/rationalize-deps/SKILL.md

@@ -1,125 +0,0 @@
----
-name: rationalize-deps
-description: Analyze Cargo.toml dependencies and attempt to remove unused features to reduce compile times and binary size
----
-
-# Rationalize Dependencies
-
-This skill analyzes Cargo.toml dependencies to identify and remove unused features.
-
-## Overview
-
-Many crates enable features by default that may not be needed. This skill:
-1. Identifies dependencies with default features enabled
-2. Tests if `default-features = false` works
-3. Identifies which specific features are actually needed
-4. Verifies compilation after changes
-
-## Step 1: Identify the target
-
-Ask the user which crate(s) to analyze:
-- A specific crate name (e.g., "tokio", "serde")
-- A specific workspace member (e.g., "quickwit-search")
-- "all" to scan the entire workspace
-
-## Step 2: Analyze current dependencies
-
-For the workspace Cargo.toml (`quickwit/Cargo.toml`), list dependencies that:
-- Do NOT have `default-features = false`
-- Have default features that might be unnecessary
-
-Run: `cargo tree -p <crate> -f "{p} {f}" --edges features` to see what features are actually used.
-
-## Step 3: For each candidate dependency
-
-### 3a: Check the crate's default features
-
-Look up the crate on crates.io or check its Cargo.toml to understand:
-- What features are enabled by default
-- What each feature provides
-
-Use: `cargo metadata --format-version=1 | jq '.packages[] | select(.name == "<crate>") | .features'`
-
-### 3b: Try disabling default features
-
-Modify the dependency in `quickwit/Cargo.toml`:
-
-From:
-```toml
-some-crate = { version = "1.0" }
-```
-
-To:
-```toml
-some-crate = { version = "1.0", default-features = false }
-```
-
-### 3c: Run cargo check
-
-Run: `cargo check --workspace` (or target specific packages for faster feedback)
-
-If compilation fails:
-1. Read the error messages to identify which features are needed
-2. Add only the required features explicitly:
-   ```toml
-   some-crate = { version = "1.0", default-features = false, features = ["needed-feature"] }
-   ```
-3. Re-run cargo check
-
-### 3d: Binary search for minimal features
-
-If there are many default features, use binary search:
-1. Start with no features
-2. If it fails, add half the default features
-3. Continue until you find the minimal set
-
-## Step 4: Document findings
-
-For each dependency analyzed, report:
-- Original configuration
-- New configuration (if changed)
-- Features that were removed
-- Any features that are required
-
-## Step 5: Verify full build
-
-After all changes, run:
-```bash
-cargo check --workspace --all-targets
-cargo test --workspace --no-run
-```
-
-## Common Patterns
-
-### Serde
-Often only needs `derive`:
-```toml
-serde = { version = "1.0", default-features = false, features = ["derive", "std"] }
-```
-
-### Tokio
-Identify which runtime features are actually used:
-```toml
-tokio = { version = "1.0", default-features = false, features = ["rt-multi-thread", "macros", "sync"] }
-```
-
-### Reqwest
-Often doesn't need all TLS backends:
-```toml
-reqwest = { version = "0.11", default-features = false, features = ["rustls-tls", "json"] }
-```
-
-## Rollback
-
-If changes cause issues:
-```bash
-git checkout quickwit/Cargo.toml
-cargo check --workspace
-```
-
-## Tips
-
-- Start with large crates that have many default features (tokio, reqwest, hyper)
-- Use `cargo bloat --crates` to identify large dependencies
-- Check `cargo tree -d` for duplicate dependencies that might indicate feature conflicts
-- Some features are needed only for tests - consider using `[dev-dependencies]` features

.claude/skills/simple-pr/SKILL.md

@@ -1,60 +0,0 @@
----
-name: simple-pr
-description: Create a simple PR from staged changes with an auto-generated commit message
-disable-model-invocation: true
----
-
-# Simple PR
-
-Follow these steps to create a simple PR from staged changes:
-
-## Step 1: Check workspace state
-
-Run: `git status`
-
-Verify that all changes have been staged (no unstaged changes). If there are unstaged changes, abort and ask the user to stage their changes first with `git add`.
-
-Also verify that we are on the `main` branch. If not, abort and ask the user to switch to main first.
-
-## Step 2: Ensure main is up to date
-
-Run: `git pull origin main`
-
-This ensures we're working from the latest code.
-
-## Step 3: Review staged changes
-
-Run: `git diff --cached`
-
-Review the staged changes to understand what the PR will contain.
-
-## Step 4: Generate commit message
-
-Based on the staged changes, generate a concise commit message (1-2 sentences) that describes the "why" rather than the "what".
-
-Display the proposed commit message to the user and ask for confirmation before proceeding.
-
-## Step 5: Create a new branch
-
-Get the git username: `git config user.name | tr ' ' '-' | tr '[:upper:]' '[:lower:]'`
-
-Create a short, descriptive branch name based on the changes (e.g., `fix-typo-in-readme`, `add-retry-logic`, `update-deps`).
-
-Create and checkout the branch: `git checkout -b {username}/{short-descriptive-name}`
-
-## Step 6: Commit changes
-
-Commit with the message from step 3:
-```
-git commit -m "{commit-message}"
-```
-
-## Step 7: Push and open a PR
-
-Push the branch and open a PR:
-```
-git push -u origin {branch-name}
-gh pr create --title "{commit-message-title}" --body "{longer-description-if-needed}"
-```
-
-Report the PR URL to the user when complete.

Cargo.toml

@@ -15,7 +15,7 @@ rust-version = "1.85"
 exclude = ["benches/*.json", "benches/*.txt"]
 
 [dependencies]
-oneshot = "0.1.13"
+oneshot = "0.1.7"
 base64 = "0.22.0"
 byteorder = "1.4.3"
 crc32fast = "1.3.2"
@@ -193,12 +193,3 @@ harness = false
 [[bench]]
 name = "str_search_and_get"
 harness = false
-
-[[bench]]
-name = "merge_segments"
-harness = false
-
-[[bench]]
-name = "regex_all_terms"
-harness = false
-

benches/merge_segments.rs

@@ -1,224 +0,0 @@
-// Benchmarks segment merging
-//
-// Notes:
-// - Input segments are kept intact (no deletes / no IndexWriter merge).
-// - Output is written to a `NullDirectory` that discards all files except
-//  fieldnorms (needed for merging).
-
-use std::collections::HashMap;
-use std::io::{self, Write};
-use std::path::{Path, PathBuf};
-use std::sync::{Arc, RwLock};
-
-use binggan::{black_box, BenchRunner};
-use rand::prelude::*;
-use rand::rngs::StdRng;
-use rand::SeedableRng;
-use tantivy::directory::error::{DeleteError, OpenReadError, OpenWriteError};
-use tantivy::directory::{
-    AntiCallToken, Directory, FileHandle, OwnedBytes, TerminatingWrite, WatchCallback, WatchHandle,
-    WritePtr,
-};
-use tantivy::indexer::{merge_filtered_segments, NoMergePolicy};
-use tantivy::schema::{Schema, TEXT};
-use tantivy::{doc, HasLen, Index, IndexSettings, Segment};
-
-#[derive(Clone, Default, Debug)]
-struct NullDirectory {
-    blobs: Arc<RwLock<HashMap<PathBuf, OwnedBytes>>>,
-}
-
-struct NullWriter;
-
-impl Write for NullWriter {
-    fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
-        Ok(buf.len())
-    }
-
-    fn flush(&mut self) -> io::Result<()> {
-        Ok(())
-    }
-}
-
-impl TerminatingWrite for NullWriter {
-    fn terminate_ref(&mut self, _token: AntiCallToken) -> io::Result<()> {
-        Ok(())
-    }
-}
-
-struct InMemoryWriter {
-    path: PathBuf,
-    buffer: Vec<u8>,
-    blobs: Arc<RwLock<HashMap<PathBuf, OwnedBytes>>>,
-}
-
-impl Write for InMemoryWriter {
-    fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
-        self.buffer.extend_from_slice(buf);
-        Ok(buf.len())
-    }
-
-    fn flush(&mut self) -> io::Result<()> {
-        Ok(())
-    }
-}
-
-impl TerminatingWrite for InMemoryWriter {
-    fn terminate_ref(&mut self, _token: AntiCallToken) -> io::Result<()> {
-        let bytes = OwnedBytes::new(std::mem::take(&mut self.buffer));
-        self.blobs.write().unwrap().insert(self.path.clone(), bytes);
-        Ok(())
-    }
-}
-
-#[derive(Debug, Default)]
-struct NullFileHandle;
-impl HasLen for NullFileHandle {
-    fn len(&self) -> usize {
-        0
-    }
-}
-impl FileHandle for NullFileHandle {
-    fn read_bytes(&self, _range: std::ops::Range<usize>) -> io::Result<OwnedBytes> {
-        unimplemented!()
-    }
-}
-
-impl Directory for NullDirectory {
-    fn get_file_handle(&self, path: &Path) -> Result<Arc<dyn FileHandle>, OpenReadError> {
-        if let Some(bytes) = self.blobs.read().unwrap().get(path) {
-            return Ok(Arc::new(bytes.clone()));
-        }
-        Ok(Arc::new(NullFileHandle))
-    }
-
-    fn delete(&self, _path: &Path) -> Result<(), DeleteError> {
-        Ok(())
-    }
-
-    fn exists(&self, _path: &Path) -> Result<bool, OpenReadError> {
-        Ok(true)
-    }
-
-    fn open_write(&self, path: &Path) -> Result<WritePtr, OpenWriteError> {
-        let path_buf = path.to_path_buf();
-        if path.to_string_lossy().ends_with(".fieldnorm") {
-            let writer = InMemoryWriter {
-                path: path_buf,
-                buffer: Vec::new(),
-                blobs: Arc::clone(&self.blobs),
-            };
-            Ok(io::BufWriter::new(Box::new(writer)))
-        } else {
-            Ok(io::BufWriter::new(Box::new(NullWriter)))
-        }
-    }
-
-    fn atomic_read(&self, path: &Path) -> Result<Vec<u8>, OpenReadError> {
-        if let Some(bytes) = self.blobs.read().unwrap().get(path) {
-            return Ok(bytes.as_slice().to_vec());
-        }
-        Err(OpenReadError::FileDoesNotExist(path.to_path_buf()))
-    }
-
-    fn atomic_write(&self, _path: &Path, _data: &[u8]) -> io::Result<()> {
-        Ok(())
-    }
-
-    fn sync_directory(&self) -> io::Result<()> {
-        Ok(())
-    }
-
-    fn watch(&self, _watch_callback: WatchCallback) -> tantivy::Result<WatchHandle> {
-        Ok(WatchHandle::empty())
-    }
-}
-
-struct MergeScenario {
-    #[allow(dead_code)]
-    index: Index,
-    segments: Vec<Segment>,
-    settings: IndexSettings,
-    label: String,
-}
-
-fn build_index(
-    num_segments: usize,
-    docs_per_segment: usize,
-    tokens_per_doc: usize,
-    vocab_size: usize,
-) -> MergeScenario {
-    let mut schema_builder = Schema::builder();
-    let body = schema_builder.add_text_field("body", TEXT);
-    let schema = schema_builder.build();
-    let index = Index::create_in_ram(schema.clone());
-
-    assert!(vocab_size > 0);
-    let total_tokens = num_segments * docs_per_segment * tokens_per_doc;
-    let use_unique_terms = vocab_size >= total_tokens;
-    let mut rng = StdRng::from_seed([7u8; 32]);
-    let mut next_token_id: u64 = 0;
-
-    {
-        let mut writer = index.writer_with_num_threads(1, 256_000_000).unwrap();
-        writer.set_merge_policy(Box::new(NoMergePolicy));
-        for _ in 0..num_segments {
-            for _ in 0..docs_per_segment {
-                let mut tokens = Vec::with_capacity(tokens_per_doc);
-                for _ in 0..tokens_per_doc {
-                    let token_id = if use_unique_terms {
-                        let id = next_token_id;
-                        next_token_id += 1;
-                        id
-                    } else {
-                        rng.random_range(0..vocab_size as u64)
-                    };
-                    tokens.push(format!("term_{token_id}"));
-                }
-                writer.add_document(doc!(body => tokens.join(" "))).unwrap();
-            }
-            writer.commit().unwrap();
-        }
-    }
-
-    let segments = index.searchable_segments().unwrap();
-    let settings = index.settings().clone();
-    let label = format!(
-        "segments={}, docs/seg={}, tokens/doc={}, vocab={}",
-        num_segments, docs_per_segment, tokens_per_doc, vocab_size
-    );
-
-    MergeScenario {
-        index,
-        segments,
-        settings,
-        label,
-    }
-}
-
-fn main() {
-    let scenarios = vec![
-        build_index(8, 50_000, 12, 8),
-        build_index(16, 50_000, 12, 8),
-        build_index(16, 100_000, 12, 8),
-        build_index(8, 50_000, 8, 8 * 50_000 * 8),
-    ];
-
-    let mut runner = BenchRunner::new();
-    for scenario in scenarios {
-        let mut group = runner.new_group();
-        group.set_name(format!("merge_segments inv_index — {}", scenario.label));
-        let segments = scenario.segments.clone();
-        let settings = scenario.settings.clone();
-        group.register("merge", move |_| {
-            let output_dir = NullDirectory::default();
-            let filter_doc_ids = vec![None; segments.len()];
-            let merged_index =
-                merge_filtered_segments(&segments, settings.clone(), filter_doc_ids, output_dir)
-                    .unwrap();
-            black_box(merged_index);
-        });
-
-        group.run();
-    }
-}

benches/regex_all_terms.rs

@@ -1,113 +0,0 @@
-// Benchmarks regex query that matches all terms in a synthetic index.
-//
-// Corpus model:
-// - N unique terms: t000000, t000001, ...
-// - M docs
-// - K tokens per doc: doc i gets terms derived from (i, token_index)
-//
-// Query:
-// - Regex "t.*" to match all terms
-//
-// Run with:
-// - cargo bench --bench regex_all_terms
-//
-
-use std::fmt::Write;
-
-use binggan::{black_box, BenchRunner};
-use tantivy::collector::Count;
-use tantivy::query::RegexQuery;
-use tantivy::schema::{Schema, TEXT};
-use tantivy::{doc, Index, ReloadPolicy};
-
-const HEAP_SIZE_BYTES: usize = 200_000_000;
-
-#[derive(Clone, Copy)]
-struct BenchConfig {
-    num_terms: usize,
-    num_docs: usize,
-    tokens_per_doc: usize,
-}
-
-fn main() {
-    let configs = default_configs();
-
-    let mut runner = BenchRunner::new();
-    for config in configs {
-        let (index, text_field) = build_index(config, HEAP_SIZE_BYTES);
-        let reader = index
-            .reader_builder()
-            .reload_policy(ReloadPolicy::Manual)
-            .try_into()
-            .expect("reader");
-        let searcher = reader.searcher();
-        let query = RegexQuery::from_pattern("t.*", text_field).expect("regex query");
-
-        let mut group = runner.new_group();
-        group.set_name(format!(
-            "regex_all_terms_t{}_d{}_k{}",
-            config.num_terms, config.num_docs, config.tokens_per_doc
-        ));
-        group.register("regex_count", move |_| {
-            let count = searcher.search(&query, &Count).expect("search");
-            black_box(count);
-        });
-        group.run();
-    }
-}
-
-fn default_configs() -> Vec<BenchConfig> {
-    vec![
-        BenchConfig {
-            num_terms: 10_000,
-            num_docs: 100_000,
-            tokens_per_doc: 1,
-        },
-        BenchConfig {
-            num_terms: 10_000,
-            num_docs: 100_000,
-            tokens_per_doc: 8,
-        },
-        BenchConfig {
-            num_terms: 100_000,
-            num_docs: 100_000,
-            tokens_per_doc: 1,
-        },
-        BenchConfig {
-            num_terms: 100_000,
-            num_docs: 100_000,
-            tokens_per_doc: 8,
-        },
-    ]
-}
-
-fn build_index(config: BenchConfig, heap_size_bytes: usize) -> (Index, tantivy::schema::Field) {
-    let mut schema_builder = Schema::builder();
-    let text_field = schema_builder.add_text_field("text", TEXT);
-    let schema = schema_builder.build();
-    let index = Index::create_in_ram(schema);
-
-    let term_width = config.num_terms.to_string().len();
-    {
-        let mut writer = index
-            .writer_with_num_threads(1, heap_size_bytes)
-            .expect("writer");
-        let mut buffer = String::new();
-        for doc_id in 0..config.num_docs {
-            buffer.clear();
-            for token_idx in 0..config.tokens_per_doc {
-                if token_idx > 0 {
-                    buffer.push(' ');
-                }
-                let term_id = (doc_id * config.tokens_per_doc + token_idx) % config.num_terms;
-                write!(&mut buffer, "t{term_id:0term_width$}").expect("write token");
-            }
-            writer
-                .add_document(doc!(text_field => buffer.as_str()))
-                .expect("add_document");
-        }
-        writer.commit().expect("commit");
-    }
-
-    (index, text_field)
-}

doc/src/json.md

@@ -60,7 +60,7 @@ At indexing, tantivy will try to interpret number and strings as different type
 priority order.
 
 Numbers will be interpreted as u64, i64 and f64 in that order.
-Strings will be interpreted as rfc3339 dates or simple strings.
+Strings will be interpreted as rfc3999 dates or simple strings.
 
 The first working type is picked and is the only term that is emitted for indexing.
 Note this interpretation happens on a per-document basis, and there is no effort to try to sniff
@@ -81,7 +81,7 @@ Will be interpreted as
 (my_path.my_segment, String, 233) or (my_path.my_segment, u64, 233)
 ```
 
-Likewise, we need to emit two tokens if the query contains an rfc3339 date.
+Likewise, we need to emit two tokens if the query contains an rfc3999 date.
 Indeed the date could have been actually a single token inside the text of a document at ingestion time. Generally speaking, we will always at least emit a string token in query parsing, and sometimes more.
 
 If one more json field is defined, things get even more complicated.

examples/custom_collector.rs

@@ -70,7 +70,7 @@ impl Collector for StatsCollector {
     fn for_segment(
         &self,
         _segment_local_id: u32,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> tantivy::Result<StatsSegmentCollector> {
         let fast_field_reader = segment_reader.fast_fields().u64(&self.field)?;
         Ok(StatsSegmentCollector {

examples/faceted_search_with_tweaked_score.rs

@@ -65,7 +65,7 @@ fn main() -> tantivy::Result<()> {
         );
         let top_docs_by_custom_score =
             // Call TopDocs with a custom tweak score
-            TopDocs::with_limit(2).tweak_score(move |segment_reader: &SegmentReader| {
+            TopDocs::with_limit(2).tweak_score(move |segment_reader: &dyn SegmentReader| {
                 let ingredient_reader = segment_reader.facet_reader("ingredient").unwrap();
                 let facet_dict = ingredient_reader.facet_dict();
 

examples/warmer.rs

@@ -43,7 +43,7 @@ impl DynamicPriceColumn {
         }
     }
 
-    pub fn price_for_segment(&self, segment_reader: &SegmentReader) -> Option<Arc<Vec<Price>>> {
+    pub fn price_for_segment(&self, segment_reader: &dyn SegmentReader) -> Option<Arc<Vec<Price>>> {
         let segment_key = (segment_reader.segment_id(), segment_reader.delete_opstamp());
         self.price_cache.read().unwrap().get(&segment_key).cloned()
     }
@@ -157,7 +157,7 @@ fn main() -> tantivy::Result<()> {
     let query = query_parser.parse_query("cooking")?;
 
     let searcher = reader.searcher();
-    let score_by_price = move |segment_reader: &SegmentReader| {
+    let score_by_price = move |segment_reader: &dyn SegmentReader| {
         let price = price_dynamic_column
             .price_for_segment(segment_reader)
             .unwrap();

query-grammar/src/query_grammar.rs

@@ -560,7 +560,7 @@ fn range_infallible(inp: &str) -> JResult<&str, UserInputLeaf> {
             (
                 (
                     value((), tag(">=")),
-                    map(word_infallible(")", false), |(bound, err)| {
+                    map(word_infallible("", false), |(bound, err)| {
                         (
                             (
                                 bound
@@ -574,7 +574,7 @@ fn range_infallible(inp: &str) -> JResult<&str, UserInputLeaf> {
                 ),
                 (
                     value((), tag("<=")),
-                    map(word_infallible(")", false), |(bound, err)| {
+                    map(word_infallible("", false), |(bound, err)| {
                         (
                             (
                                 UserInputBound::Unbounded,
@@ -588,7 +588,7 @@ fn range_infallible(inp: &str) -> JResult<&str, UserInputLeaf> {
                 ),
                 (
                     value((), tag(">")),
-                    map(word_infallible(")", false), |(bound, err)| {
+                    map(word_infallible("", false), |(bound, err)| {
                         (
                             (
                                 bound
@@ -602,7 +602,7 @@ fn range_infallible(inp: &str) -> JResult<&str, UserInputLeaf> {
                 ),
                 (
                     value((), tag("<")),
-                    map(word_infallible(")", false), |(bound, err)| {
+                    map(word_infallible("", false), |(bound, err)| {
                         (
                             (
                                 UserInputBound::Unbounded,
@@ -1323,14 +1323,6 @@ mod test {
         test_parse_query_to_ast_helper("<a", "{\"*\" TO \"a\"}");
         test_parse_query_to_ast_helper("<=a", "{\"*\" TO \"a\"]");
         test_parse_query_to_ast_helper("<=bsd", "{\"*\" TO \"bsd\"]");
-
-        test_parse_query_to_ast_helper("(<=42)", "{\"*\" TO \"42\"]");
-        test_parse_query_to_ast_helper("(<=42 )", "{\"*\" TO \"42\"]");
-        test_parse_query_to_ast_helper("(age:>5)", "\"age\":{\"5\" TO \"*\"}");
-        test_parse_query_to_ast_helper(
-            "(title:bar AND age:>12)",
-            "(+\"title\":bar +\"age\":{\"12\" TO \"*\"})",
-        );
     }
 
     #[test]

src/aggregation/accessor_helpers.rs

@@ -57,7 +57,7 @@ pub(crate) fn get_numeric_or_date_column_types() -> &'static [ColumnType] {
 
 /// Get fast field reader or empty as default.
 pub(crate) fn get_ff_reader(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field_name: &str,
     allowed_column_types: Option<&[ColumnType]>,
 ) -> crate::Result<(columnar::Column<u64>, ColumnType)> {
@@ -74,7 +74,7 @@ pub(crate) fn get_ff_reader(
 }
 
 pub(crate) fn get_dynamic_columns(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field_name: &str,
 ) -> crate::Result<Vec<columnar::DynamicColumn>> {
     let ff_fields = reader.fast_fields().dynamic_column_handles(field_name)?;
@@ -90,7 +90,7 @@ pub(crate) fn get_dynamic_columns(
 ///
 /// Is guaranteed to return at least one column.
 pub(crate) fn get_all_ff_reader_or_empty(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field_name: &str,
     allowed_column_types: Option<&[ColumnType]>,
     fallback_type: ColumnType,

src/aggregation/agg_data.rs

@@ -469,7 +469,7 @@ impl AggKind {
 /// Build AggregationsData by walking the request tree.
 pub(crate) fn build_aggregations_data_from_req(
     aggs: &Aggregations,
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     segment_ordinal: SegmentOrdinal,
     context: AggContextParams,
 ) -> crate::Result<AggregationsSegmentCtx> {
@@ -489,7 +489,7 @@ pub(crate) fn build_aggregations_data_from_req(
 fn build_nodes(
     agg_name: &str,
     req: &Aggregation,
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     segment_ordinal: SegmentOrdinal,
     data: &mut AggregationsSegmentCtx,
     is_top_level: bool,
@@ -728,7 +728,6 @@ fn build_nodes(
             let idx_in_req_data = data.push_filter_req_data(FilterAggReqData {
                 name: agg_name.to_string(),
                 req: filter_req.clone(),
-                segment_reader: reader.clone(),
                 evaluator,
                 matching_docs_buffer,
                 is_top_level,
@@ -745,7 +744,7 @@ fn build_nodes(
 
 fn build_children(
     aggs: &Aggregations,
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     segment_ordinal: SegmentOrdinal,
     data: &mut AggregationsSegmentCtx,
 ) -> crate::Result<Vec<AggRefNode>> {
@@ -764,7 +763,7 @@ fn build_children(
 }
 
 fn get_term_agg_accessors(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field_name: &str,
     missing: &Option<Key>,
 ) -> crate::Result<Vec<(Column<u64>, ColumnType)>> {
@@ -817,7 +816,7 @@ fn build_terms_or_cardinality_nodes(
     agg_name: &str,
     field_name: &str,
     missing: &Option<Key>,
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     segment_ordinal: SegmentOrdinal,
     data: &mut AggregationsSegmentCtx,
     sub_aggs: &Aggregations,

src/aggregation/agg_result.rs

@@ -10,8 +10,7 @@ use serde::{Deserialize, Serialize};
 
 use super::bucket::GetDocCount;
 use super::metric::{
-    AverageMetricResult, CardinalityMetricResult, ExtendedStats, PercentilesMetricResult,
-    SingleMetricResult, Stats, TopHitsMetricResult,
+    ExtendedStats, PercentilesMetricResult, SingleMetricResult, Stats, TopHitsMetricResult,
 };
 use super::{AggregationError, Key};
 use crate::TantivyError;
@@ -82,8 +81,8 @@ impl AggregationResult {
 #[serde(untagged)]
 /// MetricResult
 pub enum MetricResult {
-    /// Average metric result with sum and count for multi-step merging.
-    Average(AverageMetricResult),
+    /// Average metric result.
+    Average(SingleMetricResult),
     /// Count metric result.
     Count(SingleMetricResult),
     /// Max metric result.
@@ -100,8 +99,8 @@ pub enum MetricResult {
     Percentiles(PercentilesMetricResult),
     /// Top hits metric result
     TopHits(TopHitsMetricResult),
-    /// Cardinality metric result with HLL sketch for multi-step merging.
-    Cardinality(CardinalityMetricResult),
+    /// Cardinality metric result
+    Cardinality(SingleMetricResult),
 }
 
 impl MetricResult {
@@ -120,7 +119,7 @@ impl MetricResult {
             MetricResult::TopHits(_) => Err(TantivyError::AggregationError(
                 AggregationError::InvalidRequest("top_hits can't be used to order".to_string()),
             )),
-            MetricResult::Cardinality(card) => Ok(card.value), // CardinalityMetricResult.value
+            MetricResult::Cardinality(card) => Ok(card.value),
         }
     }
 }

src/aggregation/agg_tests.rs

@@ -1359,10 +1359,10 @@ fn test_aggregation_on_json_object_mixed_types() {
         &serde_json::json!({
           "rangeagg": {
             "buckets": [
-              { "average_in_range": { "value": -20.5, "sum": -20.5, "count": 1 }, "doc_count": 1, "key": "*-3", "to": 3.0 },
-              { "average_in_range": { "value": 10.0, "sum": 10.0, "count": 1 }, "doc_count": 1, "from": 3.0, "key": "3-19", "to": 19.0 },
-              { "average_in_range": { "value": null, "sum": 0.0, "count": 0 }, "doc_count": 0, "from": 19.0, "key": "19-20", "to": 20.0 },
-              { "average_in_range": { "value": null, "sum": 0.0, "count": 0 }, "doc_count": 0, "from": 20.0, "key": "20-*" }
+              { "average_in_range": { "value": -20.5 }, "doc_count": 1, "key": "*-3", "to": 3.0 },
+              { "average_in_range": { "value": 10.0 }, "doc_count": 1, "from": 3.0, "key": "3-19", "to": 19.0 },
+              { "average_in_range": { "value": null }, "doc_count": 0, "from": 19.0, "key": "19-20", "to": 20.0 },
+              { "average_in_range": { "value": null }, "doc_count": 0, "from": 20.0, "key": "20-*" }
             ]
           },
           "termagg": {

src/aggregation/bucket/filter.rs

@@ -401,8 +401,6 @@ pub struct FilterAggReqData {
     pub name: String,
     /// The filter aggregation
     pub req: FilterAggregation,
-    /// The segment reader
-    pub segment_reader: SegmentReader,
     /// Document evaluator for the filter query (precomputed BitSet)
     /// This is built once when the request data is created
     pub evaluator: DocumentQueryEvaluator,
@@ -414,9 +412,8 @@ pub struct FilterAggReqData {
 
 impl FilterAggReqData {
     pub(crate) fn get_memory_consumption(&self) -> usize {
-        // Estimate: name + segment reader reference + bitset + buffer capacity
+        // Estimate: name + bitset + buffer capacity
         self.name.len()
-        + std::mem::size_of::<SegmentReader>()
         + self.evaluator.bitset.len() / 8 // BitSet memory (bits to bytes)
         + self.matching_docs_buffer.capacity() * std::mem::size_of::<DocId>()
         + std::mem::size_of::<bool>()
@@ -438,7 +435,7 @@ impl DocumentQueryEvaluator {
     pub(crate) fn new(
         query: Box<dyn Query>,
         schema: Schema,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self> {
         let max_doc = segment_reader.max_doc();
 
@@ -838,7 +835,7 @@ mod tests {
         let expected = json!({
             "electronics": {
                 "doc_count": 2,
-                "avg_price": { "value": 899.0, "sum": 1798.0, "count": 2 }  // (999 + 799) / 2
+                "avg_price": { "value": 899.0 }  // (999 + 799) / 2
             }
         });
 
@@ -868,7 +865,7 @@ mod tests {
         let expected = json!({
             "furniture": {
                 "doc_count": 0,
-                "avg_price": { "value": null, "sum": 0.0, "count": 0 }
+                "avg_price": { "value": null }
             }
         });
 
@@ -904,7 +901,7 @@ mod tests {
         let expected = json!({
             "electronics": {
                 "doc_count": 2,
-                "avg_price": { "value": 899.0, "sum": 1798.0, "count": 2 }
+                "avg_price": { "value": 899.0 }
             },
             "in_stock": {
                 "doc_count": 3,  // apple, samsung, penguin
@@ -1000,7 +997,7 @@ mod tests {
         let expected = json!({
             "premium_electronics": {
                 "doc_count": 1,  // Only apple (999) is >= 800 in tantivy's range semantics
-                "avg_rating": { "value": 4.5, "sum": 4.5, "count": 1 }
+                "avg_rating": { "value": 4.5 }
             }
         });
 
@@ -1032,7 +1029,7 @@ mod tests {
         let expected = json!({
             "in_stock": {
                 "doc_count": 3,  // apple, samsung, penguin
-                "avg_price": { "value": 607.67, "sum": 1823.0, "count": 3 }  // (999 + 799 + 25) / 3 ≈ 607.67
+                "avg_price": { "value": 607.67 }  // (999 + 799 + 25) / 3 ≈ 607.67
             },
             "out_of_stock": {
                 "doc_count": 1,  // nike
@@ -1183,7 +1180,7 @@ mod tests {
                 "doc_count": 4,
                 "electronics_branch": {
                     "doc_count": 2,
-                    "avg_price": { "value": 899.0, "sum": 1798.0, "count": 2 }
+                    "avg_price": { "value": 899.0 }
                 },
                 "in_stock_branch": {
                     "doc_count": 3,
@@ -1259,7 +1256,7 @@ mod tests {
                     "doc_count": 2,  // apple (999), samsung (799)
                     "electronics": {
                         "doc_count": 2,  // both are electronics
-                        "avg_rating": { "value": 4.35, "sum": 8.7, "count": 2 }  // (4.5 + 4.2) / 2
+                        "avg_rating": { "value": 4.35 }  // (4.5 + 4.2) / 2
                     },
                     "in_stock": {
                         "doc_count": 2,  // both are in stock
@@ -1321,12 +1318,12 @@ mod tests {
                         {
                             "key": "samsung",
                             "doc_count": 1,
-                            "avg_price": { "value": 799.0, "sum": 799.0, "count": 1 }
+                            "avg_price": { "value": 799.0 }
                         },
                         {
                             "key": "apple",
                             "doc_count": 1,
-                            "avg_price": { "value": 999.0, "sum": 999.0, "count": 1 }
+                            "avg_price": { "value": 999.0 }
                         }
                     ],
                     "sum_other_doc_count": 0,
@@ -1370,7 +1367,7 @@ mod tests {
                     "sum": 1798.0,
                     "avg": 899.0
                 },
-                "rating_avg": { "value": 4.35, "sum": 8.7, "count": 2 },
+                "rating_avg": { "value": 4.35 },
                 "count": { "value": 2.0 }
             }
         });
@@ -1411,7 +1408,7 @@ mod tests {
         let expected = json!({
             "electronics": {
                 "doc_count": 0,
-                "avg_price": { "value": null, "sum": 0.0, "count": 0 }
+                "avg_price": { "value": null }
             }
         });
 
@@ -1698,15 +1695,13 @@ mod tests {
         let filter_expected = json!({
             "electronics": {
                 "doc_count": 2,
-                "avg_price": { "value": 899.0, "sum": 1798.0, "count": 2 }
+                "avg_price": { "value": 899.0 }
             }
         });
 
         let separate_expected = json!({
             "result": {
-                "value": 899.0,
-                "sum": 1798.0,
-                "count": 2
+                "value": 899.0
             }
         });
 

src/aggregation/bucket/histogram/histogram.rs

@@ -1222,9 +1222,7 @@ mod tests {
             res["histogram"]["buckets"][0],
             json!({
                 "avg": {
-                    "value": Value::Null,
-                    "sum": 0.0,
-                    "count": 0
+                    "value": Value::Null
                 },
                 "doc_count": 0,
                 "key": 2.0,

src/aggregation/collector.rs

@@ -66,7 +66,7 @@ impl Collector for DistributedAggregationCollector {
     fn for_segment(
         &self,
         segment_local_id: crate::SegmentOrdinal,
-        reader: &crate::SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         AggregationSegmentCollector::from_agg_req_and_reader(
             &self.agg,
@@ -96,7 +96,7 @@ impl Collector for AggregationCollector {
     fn for_segment(
         &self,
         segment_local_id: crate::SegmentOrdinal,
-        reader: &crate::SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         AggregationSegmentCollector::from_agg_req_and_reader(
             &self.agg,
@@ -145,7 +145,7 @@ impl AggregationSegmentCollector {
     /// reader. Also includes validation, e.g. checking field types and existence.
     pub fn from_agg_req_and_reader(
         agg: &Aggregations,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         segment_ordinal: SegmentOrdinal,
         context: &AggContextParams,
     ) -> crate::Result<Self> {

src/aggregation/intermediate_agg_result.rs

@@ -19,9 +19,8 @@ use super::bucket::{
     GetDocCount, Order, OrderTarget, RangeAggregation, TermsAggregation,
 };
 use super::metric::{
-    AverageMetricResult, CardinalityMetricResult, IntermediateAverage, IntermediateCount,
-    IntermediateExtendedStats, IntermediateMax, IntermediateMin, IntermediateStats,
-    IntermediateSum, PercentilesCollector, TopHitsTopNComputer,
+    IntermediateAverage, IntermediateCount, IntermediateExtendedStats, IntermediateMax,
+    IntermediateMin, IntermediateStats, IntermediateSum, PercentilesCollector, TopHitsTopNComputer,
 };
 use super::segment_agg_result::AggregationLimitsGuard;
 use super::{format_date, AggregationError, Key, SerializedKey};
@@ -326,11 +325,7 @@ impl IntermediateMetricResult {
     fn into_final_metric_result(self, req: &Aggregation) -> MetricResult {
         match self {
             IntermediateMetricResult::Average(intermediate_avg) => {
-                MetricResult::Average(AverageMetricResult {
-                    value: intermediate_avg.finalize(),
-                    sum: intermediate_avg.sum(),
-                    count: intermediate_avg.count(),
-                })
+                MetricResult::Average(intermediate_avg.finalize().into())
             }
             IntermediateMetricResult::Count(intermediate_count) => {
                 MetricResult::Count(intermediate_count.finalize().into())
@@ -358,11 +353,7 @@ impl IntermediateMetricResult {
                 MetricResult::TopHits(top_hits.into_final_result())
             }
             IntermediateMetricResult::Cardinality(cardinality) => {
-                let value = cardinality.finalize();
-                MetricResult::Cardinality(CardinalityMetricResult {
-                    value,
-                    sketch: Some(cardinality),
-                })
+                MetricResult::Cardinality(cardinality.finalize().into())
             }
         }
     }
@@ -829,7 +820,7 @@ impl IntermediateRangeBucketEntry {
         };
 
         // If we have a date type on the histogram buckets, we add the `key_as_string` field as
-        // rfc3339
+        // rfc339
         if column_type == Some(ColumnType::DateTime) {
             if let Some(val) = range_bucket_entry.to {
                 let key_as_string = format_date(val as i64)?;

src/aggregation/metric/average.rs

@@ -63,16 +63,6 @@ impl IntermediateAverage {
     pub fn finalize(&self) -> Option<f64> {
         self.stats.finalize().avg
     }
-
-    /// Returns the sum of all collected values.
-    pub fn sum(&self) -> f64 {
-        self.stats.sum
-    }
-
-    /// Returns the count of all collected values.
-    pub fn count(&self) -> u64 {
-        self.stats.count
-    }
 }
 
 #[cfg(test)]

src/aggregation/metric/cardinality.rs

@@ -340,7 +340,7 @@ impl PartialEq for CardinalityCollector {
 
 impl CardinalityCollector {
     /// Compute the final cardinality estimate.
-    pub fn finalize(&self) -> Option<f64> {
+    pub fn finalize(self) -> Option<f64> {
         Some(self.sketch.clone().count().trunc())
     }
 

src/aggregation/metric/mod.rs

@@ -93,41 +93,6 @@ impl From<Option<f64>> for SingleMetricResult {
     }
 }
 
-/// Average metric result with intermediate data for merging.
-///
-/// Unlike [`SingleMetricResult`], this struct includes the raw `sum` and `count`
-/// values that can be used for multi-step query merging.
-#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
-pub struct AverageMetricResult {
-    /// The computed average value. None if no documents matched.
-    pub value: Option<f64>,
-    /// The sum of all values (for multi-step merging).
-    pub sum: f64,
-    /// The count of all values (for multi-step merging).
-    pub count: u64,
-}
-
-/// Cardinality metric result with computed value and raw HLL sketch for multi-step merging.
-///
-/// The `value` field contains the computed cardinality estimate.
-/// The `sketch` field contains the serialized HyperLogLog++ sketch that can be used
-/// for merging results across multiple query steps.
-#[derive(Clone, Debug, Serialize, Deserialize)]
-pub struct CardinalityMetricResult {
-    /// The computed cardinality estimate.
-    pub value: Option<f64>,
-    /// The serialized HyperLogLog++ sketch for multi-step merging.
-    #[serde(skip_serializing_if = "Option::is_none")]
-    pub sketch: Option<CardinalityCollector>,
-}
-
-impl PartialEq for CardinalityMetricResult {
-    fn eq(&self, other: &Self) -> bool {
-        // Only compare values, not sketch (sketch comparison is complex)
-        self.value == other.value
-    }
-}
-
 /// This is the wrapper of percentile entries, which can be vector or hashmap
 /// depending on if it's keyed or not.
 #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
@@ -146,26 +111,13 @@ pub struct PercentileValuesVecEntry {
     value: f64,
 }
 
-/// Percentiles metric result with computed values and raw sketch for multi-step merging.
+/// Single-metric aggregations use this common result structure.
 ///
-/// The `values` field contains the computed percentile values.
-/// The `sketch` field contains the serialized DDSketch that can be used for merging
-/// results across multiple query steps.
-#[derive(Clone, Debug, Serialize, Deserialize)]
+/// Main reason to wrap it in value is to match elasticsearch output structure.
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
 pub struct PercentilesMetricResult {
-    /// The computed percentile values.
+    /// The result of the percentile metric.
     pub values: PercentileValues,
-    /// The serialized DDSketch for multi-step merging.
-    /// This is the raw sketch data that can be deserialized and merged with other sketches.
-    #[serde(skip_serializing_if = "Option::is_none")]
-    pub sketch: Option<PercentilesCollector>,
-}
-
-impl PartialEq for PercentilesMetricResult {
-    fn eq(&self, other: &Self) -> bool {
-        // Only compare values, not sketch (sketch comparison is complex)
-        self.values == other.values
-    }
 }
 
 /// The top_hits metric results entry
@@ -246,105 +198,4 @@ mod tests {
         assert_eq!(aggregations_res_json["price_min"]["value"], 0.0);
         assert_eq!(aggregations_res_json["price_sum"]["value"], 15.0);
     }
-
-    #[test]
-    fn test_average_returns_sum_and_count() {
-        let mut schema_builder = Schema::builder();
-        let field_options = NumericOptions::default().set_fast();
-        let field = schema_builder.add_f64_field("price", field_options);
-        let index = Index::create_in_ram(schema_builder.build());
-        let mut index_writer: IndexWriter = index.writer_for_tests().unwrap();
-
-        // Add documents with values 0, 1, 2, 3, 4, 5
-        // sum = 15, count = 6, avg = 2.5
-        for i in 0..6 {
-            index_writer
-                .add_document(doc!(
-                    field => i as f64,
-                ))
-                .unwrap();
-        }
-        index_writer.commit().unwrap();
-
-        let aggregations_json = r#"{ "price_avg": { "avg": { "field": "price" } } }"#;
-        let aggregations: Aggregations = serde_json::from_str(aggregations_json).unwrap();
-        let collector = AggregationCollector::from_aggs(aggregations, Default::default());
-        let reader = index.reader().unwrap();
-        let searcher = reader.searcher();
-        let aggregations_res: AggregationResults = searcher.search(&AllQuery, &collector).unwrap();
-        let aggregations_res_json = serde_json::to_value(aggregations_res).unwrap();
-
-        // Verify all three fields are present and correct
-        assert_eq!(aggregations_res_json["price_avg"]["value"], 2.5);
-        assert_eq!(aggregations_res_json["price_avg"]["sum"], 15.0);
-        assert_eq!(aggregations_res_json["price_avg"]["count"], 6);
-    }
-
-    #[test]
-    fn test_percentiles_returns_sketch() {
-        let mut schema_builder = Schema::builder();
-        let field_options = NumericOptions::default().set_fast();
-        let field = schema_builder.add_f64_field("latency", field_options);
-        let index = Index::create_in_ram(schema_builder.build());
-        let mut index_writer: IndexWriter = index.writer_for_tests().unwrap();
-
-        // Add documents with latency values
-        for i in 0..100 {
-            index_writer
-                .add_document(doc!(
-                    field => i as f64,
-                ))
-                .unwrap();
-        }
-        index_writer.commit().unwrap();
-
-        let aggregations_json =
-            r#"{ "latency_percentiles": { "percentiles": { "field": "latency" } } }"#;
-        let aggregations: Aggregations = serde_json::from_str(aggregations_json).unwrap();
-        let collector = AggregationCollector::from_aggs(aggregations, Default::default());
-        let reader = index.reader().unwrap();
-        let searcher = reader.searcher();
-        let aggregations_res: AggregationResults = searcher.search(&AllQuery, &collector).unwrap();
-        let aggregations_res_json = serde_json::to_value(aggregations_res).unwrap();
-
-        // Verify percentile values are present
-        assert!(aggregations_res_json["latency_percentiles"]["values"].is_object());
-        // Verify sketch is present (serialized DDSketch)
-        assert!(aggregations_res_json["latency_percentiles"]["sketch"].is_object());
-    }
-
-    #[test]
-    fn test_cardinality_returns_sketch() {
-        let mut schema_builder = Schema::builder();
-        let field_options = NumericOptions::default().set_fast();
-        let field = schema_builder.add_u64_field("user_id", field_options);
-        let index = Index::create_in_ram(schema_builder.build());
-        let mut index_writer: IndexWriter = index.writer_for_tests().unwrap();
-
-        // Add documents with some duplicate user_ids
-        for i in 0..50 {
-            index_writer
-                .add_document(doc!(
-                    field => (i % 10) as u64,  // 10 unique values
-                ))
-                .unwrap();
-        }
-        index_writer.commit().unwrap();
-
-        let aggregations_json = r#"{ "unique_users": { "cardinality": { "field": "user_id" } } }"#;
-        let aggregations: Aggregations = serde_json::from_str(aggregations_json).unwrap();
-        let collector = AggregationCollector::from_aggs(aggregations, Default::default());
-        let reader = index.reader().unwrap();
-        let searcher = reader.searcher();
-        let aggregations_res: AggregationResults = searcher.search(&AllQuery, &collector).unwrap();
-        let aggregations_res_json = serde_json::to_value(aggregations_res).unwrap();
-
-        // Verify cardinality value is present and approximately correct
-        let cardinality = aggregations_res_json["unique_users"]["value"]
-            .as_f64()
-            .unwrap();
-        assert!(cardinality >= 9.0 && cardinality <= 11.0); // HLL is approximate
-                                                            // Verify sketch is present (serialized HyperLogLog++)
-        assert!(aggregations_res_json["unique_users"]["sketch"].is_object());
-    }
 }

src/aggregation/metric/percentiles.rs

@@ -178,9 +178,6 @@ fn format_percentile(percentile: f64) -> String {
 impl PercentilesCollector {
     /// Convert result into final result. This will query the quantils from the underlying quantil
     /// collector.
-    ///
-    /// The result includes both the computed percentile values and the raw DDSketch
-    /// for multi-step query merging.
     pub fn into_final_result(self, req: &PercentilesAggregationReq) -> PercentilesMetricResult {
         let percentiles: &[f64] = req
             .percents
@@ -213,15 +210,7 @@ impl PercentilesCollector {
                     .collect(),
             )
         };
-        PercentilesMetricResult {
-            values,
-            sketch: Some(self),
-        }
-    }
-
-    /// Returns a reference to the underlying DDSketch.
-    pub fn sketch(&self) -> &sketches_ddsketch::DDSketch {
-        &self.sketch
+        PercentilesMetricResult { values }
     }
 
     fn new() -> Self {

src/collector/count_collector.rs

@@ -43,7 +43,7 @@ impl Collector for Count {
     fn for_segment(
         &self,
         _: SegmentOrdinal,
-        _: &SegmentReader,
+        _: &dyn SegmentReader,
     ) -> crate::Result<SegmentCountCollector> {
         Ok(SegmentCountCollector::default())
     }

src/collector/docset_collector.rs

@@ -1,7 +1,7 @@
 use std::collections::HashSet;
 
 use super::{Collector, SegmentCollector};
-use crate::{DocAddress, DocId, Score};
+use crate::{DocAddress, DocId, Score, SegmentReader};
 
 /// Collectors that returns the set of DocAddress that matches the query.
 ///
@@ -15,7 +15,7 @@ impl Collector for DocSetCollector {
     fn for_segment(
         &self,
         segment_local_id: crate::SegmentOrdinal,
-        _segment: &crate::SegmentReader,
+        _segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         Ok(DocSetChildCollector {
             segment_local_id,

src/collector/facet_collector.rs

@@ -265,7 +265,7 @@ impl Collector for FacetCollector {
     fn for_segment(
         &self,
         _: SegmentOrdinal,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<FacetSegmentCollector> {
         let facet_reader = reader.facet_reader(&self.field_name)?;
         let facet_dict = facet_reader.facet_dict();

src/collector/filter_collector_wrapper.rs

@@ -113,7 +113,7 @@ where
     fn for_segment(
         &self,
         segment_local_id: u32,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let column_opt = segment_reader.fast_fields().column_opt(&self.field)?;
 
@@ -287,7 +287,7 @@ where
     fn for_segment(
         &self,
         segment_local_id: u32,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let column_opt = segment_reader.fast_fields().bytes(&self.field)?;
 

src/collector/histogram_collector.rs

@@ -6,7 +6,7 @@ use fastdivide::DividerU64;
 use crate::collector::{Collector, SegmentCollector};
 use crate::fastfield::{FastFieldNotAvailableError, FastValue};
 use crate::schema::Type;
-use crate::{DocId, Score};
+use crate::{DocId, Score, SegmentReader};
 
 /// Histogram builds an histogram of the values of a fastfield for the
 /// collected DocSet.
@@ -110,7 +110,7 @@ impl Collector for HistogramCollector {
     fn for_segment(
         &self,
         _segment_local_id: crate::SegmentOrdinal,
-        segment: &crate::SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let column_opt = segment.fast_fields().u64_lenient(&self.field)?;
         let (column, _column_type) = column_opt.ok_or_else(|| FastFieldNotAvailableError {

src/collector/mod.rs

@@ -156,7 +156,7 @@ pub trait Collector: Sync + Send {
     fn for_segment(
         &self,
         segment_local_id: SegmentOrdinal,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child>;
 
     /// Returns true iff the collector requires to compute scores for documents.
@@ -174,7 +174,7 @@ pub trait Collector: Sync + Send {
         &self,
         weight: &dyn Weight,
         segment_ord: u32,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<<Self::Child as SegmentCollector>::Fruit> {
         let with_scoring = self.requires_scoring();
         let mut segment_collector = self.for_segment(segment_ord, reader)?;
@@ -186,7 +186,7 @@ pub trait Collector: Sync + Send {
 pub(crate) fn default_collect_segment_impl<TSegmentCollector: SegmentCollector>(
     segment_collector: &mut TSegmentCollector,
     weight: &dyn Weight,
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     with_scoring: bool,
 ) -> crate::Result<()> {
     match (reader.alive_bitset(), with_scoring) {
@@ -255,7 +255,7 @@ impl<TCollector: Collector> Collector for Option<TCollector> {
     fn for_segment(
         &self,
         segment_local_id: SegmentOrdinal,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         Ok(if let Some(inner) = self {
             let inner_segment_collector = inner.for_segment(segment_local_id, segment)?;
@@ -336,7 +336,7 @@ where
     fn for_segment(
         &self,
         segment_local_id: u32,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let left = self.0.for_segment(segment_local_id, segment)?;
         let right = self.1.for_segment(segment_local_id, segment)?;
@@ -407,7 +407,7 @@ where
     fn for_segment(
         &self,
         segment_local_id: u32,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let one = self.0.for_segment(segment_local_id, segment)?;
         let two = self.1.for_segment(segment_local_id, segment)?;
@@ -487,7 +487,7 @@ where
     fn for_segment(
         &self,
         segment_local_id: u32,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let one = self.0.for_segment(segment_local_id, segment)?;
         let two = self.1.for_segment(segment_local_id, segment)?;

src/collector/multi_collector.rs

@@ -24,7 +24,7 @@ impl<TCollector: Collector> Collector for CollectorWrapper<TCollector> {
     fn for_segment(
         &self,
         segment_local_id: u32,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<Box<dyn BoxableSegmentCollector>> {
         let child = self.0.for_segment(segment_local_id, reader)?;
         Ok(Box::new(SegmentCollectorWrapper(child)))
@@ -209,7 +209,7 @@ impl Collector for MultiCollector<'_> {
     fn for_segment(
         &self,
         segment_local_id: SegmentOrdinal,
-        segment: &SegmentReader,
+        segment: &dyn SegmentReader,
     ) -> crate::Result<MultiCollectorChild> {
         let children = self
             .collector_wrappers

src/collector/sort_key/order.rs

@@ -5,7 +5,7 @@ use serde::{Deserialize, Serialize};
 
 use crate::collector::{SegmentSortKeyComputer, SortKeyComputer};
 use crate::schema::{OwnedValue, Schema};
-use crate::{DocId, Order, Score};
+use crate::{DocId, Order, Score, SegmentReader};
 
 fn compare_owned_value<const NULLS_FIRST: bool>(lhs: &OwnedValue, rhs: &OwnedValue) -> Ordering {
     match (lhs, rhs) {
@@ -430,7 +430,7 @@ where
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &crate::SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let child = self.0.segment_sort_key_computer(segment_reader)?;
         Ok(SegmentSortKeyComputerWithComparator {
@@ -468,7 +468,7 @@ where
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &crate::SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let child = self.0.segment_sort_key_computer(segment_reader)?;
         Ok(SegmentSortKeyComputerWithComparator {

src/collector/sort_key/sort_by_erased_type.rs

@@ -6,7 +6,7 @@ use crate::collector::sort_key::{
 use crate::collector::{SegmentSortKeyComputer, SortKeyComputer};
 use crate::fastfield::FastFieldNotAvailableError;
 use crate::schema::OwnedValue;
-use crate::{DateTime, DocId, Score};
+use crate::{DateTime, DocId, Score, SegmentReader};
 
 /// Sort by the boxed / OwnedValue representation of either a fast field, or of the score.
 ///
@@ -86,7 +86,7 @@ impl SortKeyComputer for SortByErasedType {
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &crate::SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let inner: Box<dyn ErasedSegmentSortKeyComputer> = match self {
             Self::Field(column_name) => {

src/collector/sort_key/sort_by_score.rs

@@ -1,6 +1,6 @@
 use crate::collector::sort_key::NaturalComparator;
 use crate::collector::{SegmentSortKeyComputer, SortKeyComputer, TopNComputer};
-use crate::{DocAddress, DocId, Score};
+use crate::{DocAddress, DocId, Score, SegmentReader};
 
 /// Sort by similarity score.
 #[derive(Clone, Debug, Copy)]
@@ -19,7 +19,7 @@ impl SortKeyComputer for SortBySimilarityScore {
 
     fn segment_sort_key_computer(
         &self,
-        _segment_reader: &crate::SegmentReader,
+        _segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         Ok(SortBySimilarityScore)
     }
@@ -29,7 +29,7 @@ impl SortKeyComputer for SortBySimilarityScore {
         &self,
         k: usize,
         weight: &dyn crate::query::Weight,
-        reader: &crate::SegmentReader,
+        reader: &dyn SegmentReader,
         segment_ord: u32,
     ) -> crate::Result<Vec<(Self::SortKey, DocAddress)>> {
         let mut top_n: TopNComputer<Score, DocId, Self::Comparator> =

src/collector/sort_key/sort_by_static_fast_value.rs

@@ -61,7 +61,7 @@ impl<T: FastValue> SortKeyComputer for SortByStaticFastValue<T> {
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let sort_column_opt = segment_reader.fast_fields().u64_lenient(&self.field)?;
         let (sort_column, _sort_column_type) =

src/collector/sort_key/sort_by_string.rs

@@ -3,7 +3,7 @@ use columnar::StrColumn;
 use crate::collector::sort_key::NaturalComparator;
 use crate::collector::{SegmentSortKeyComputer, SortKeyComputer};
 use crate::termdict::TermOrdinal;
-use crate::{DocId, Score};
+use crate::{DocId, Score, SegmentReader};
 
 /// Sort by the first value of a string column.
 ///
@@ -35,7 +35,7 @@ impl SortKeyComputer for SortByString {
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &crate::SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         let str_column_opt = segment_reader.fast_fields().str(&self.column_name)?;
         Ok(ByStringColumnSegmentSortKeyComputer { str_column_opt })

src/collector/sort_key/sort_key_computer.rs

@@ -119,7 +119,7 @@ pub trait SortKeyComputer: Sync {
         &self,
         k: usize,
         weight: &dyn crate::query::Weight,
-        reader: &crate::SegmentReader,
+        reader: &dyn SegmentReader,
         segment_ord: u32,
     ) -> crate::Result<Vec<(Self::SortKey, DocAddress)>> {
         let with_scoring = self.requires_scoring();
@@ -135,7 +135,7 @@ pub trait SortKeyComputer: Sync {
     }
 
     /// Builds a child sort key computer for a specific segment.
-    fn segment_sort_key_computer(&self, segment_reader: &SegmentReader) -> Result<Self::Child>;
+    fn segment_sort_key_computer(&self, segment_reader: &dyn SegmentReader) -> Result<Self::Child>;
 }
 
 impl<HeadSortKeyComputer, TailSortKeyComputer> SortKeyComputer
@@ -156,7 +156,7 @@ where
         (self.0.comparator(), self.1.comparator())
     }
 
-    fn segment_sort_key_computer(&self, segment_reader: &SegmentReader) -> Result<Self::Child> {
+    fn segment_sort_key_computer(&self, segment_reader: &dyn SegmentReader) -> Result<Self::Child> {
         Ok((
             self.0.segment_sort_key_computer(segment_reader)?,
             self.1.segment_sort_key_computer(segment_reader)?,
@@ -357,7 +357,7 @@ where
         )
     }
 
-    fn segment_sort_key_computer(&self, segment_reader: &SegmentReader) -> Result<Self::Child> {
+    fn segment_sort_key_computer(&self, segment_reader: &dyn SegmentReader) -> Result<Self::Child> {
         let sort_key_computer1 = self.0.segment_sort_key_computer(segment_reader)?;
         let sort_key_computer2 = self.1.segment_sort_key_computer(segment_reader)?;
         let sort_key_computer3 = self.2.segment_sort_key_computer(segment_reader)?;
@@ -420,7 +420,7 @@ where
         SortKeyComputer4::Comparator,
     );
 
-    fn segment_sort_key_computer(&self, segment_reader: &SegmentReader) -> Result<Self::Child> {
+    fn segment_sort_key_computer(&self, segment_reader: &dyn SegmentReader) -> Result<Self::Child> {
         let sort_key_computer1 = self.0.segment_sort_key_computer(segment_reader)?;
         let sort_key_computer2 = self.1.segment_sort_key_computer(segment_reader)?;
         let sort_key_computer3 = self.2.segment_sort_key_computer(segment_reader)?;
@@ -454,7 +454,7 @@ where
 
 impl<F, SegmentF, TSortKey> SortKeyComputer for F
 where
-    F: 'static + Send + Sync + Fn(&SegmentReader) -> SegmentF,
+    F: 'static + Send + Sync + Fn(&dyn SegmentReader) -> SegmentF,
     SegmentF: 'static + FnMut(DocId) -> TSortKey,
     TSortKey: 'static + PartialOrd + Clone + Send + Sync + std::fmt::Debug,
 {
@@ -462,7 +462,7 @@ where
     type Child = SegmentF;
     type Comparator = NaturalComparator;
 
-    fn segment_sort_key_computer(&self, segment_reader: &SegmentReader) -> Result<Self::Child> {
+    fn segment_sort_key_computer(&self, segment_reader: &dyn SegmentReader) -> Result<Self::Child> {
         Ok((self)(segment_reader))
     }
 }
@@ -509,10 +509,10 @@ mod tests {
 
     #[test]
     fn test_lazy_score_computer() {
-        let score_computer_primary = |_segment_reader: &SegmentReader| |_doc: DocId| 200u32;
+        let score_computer_primary = |_segment_reader: &dyn SegmentReader| |_doc: DocId| 200u32;
         let call_count = Arc::new(AtomicUsize::new(0));
         let call_count_clone = call_count.clone();
-        let score_computer_secondary = move |_segment_reader: &SegmentReader| {
+        let score_computer_secondary = move |_segment_reader: &dyn SegmentReader| {
             let call_count_new_clone = call_count_clone.clone();
             move |_doc: DocId| {
                 call_count_new_clone.fetch_add(1, AtomicOrdering::SeqCst);
@@ -572,10 +572,10 @@ mod tests {
 
     #[test]
     fn test_lazy_score_computer_dynamic_ordering() {
-        let score_computer_primary = |_segment_reader: &SegmentReader| |_doc: DocId| 200u32;
+        let score_computer_primary = |_segment_reader: &dyn SegmentReader| |_doc: DocId| 200u32;
         let call_count = Arc::new(AtomicUsize::new(0));
         let call_count_clone = call_count.clone();
-        let score_computer_secondary = move |_segment_reader: &SegmentReader| {
+        let score_computer_secondary = move |_segment_reader: &dyn SegmentReader| {
             let call_count_new_clone = call_count_clone.clone();
             move |_doc: DocId| {
                 call_count_new_clone.fetch_add(1, AtomicOrdering::SeqCst);

src/collector/sort_key_top_collector.rs

@@ -32,7 +32,11 @@ where TSortKeyComputer: SortKeyComputer + Send + Sync + 'static
         self.sort_key_computer.check_schema(schema)
     }
 
-    fn for_segment(&self, segment_ord: u32, segment_reader: &SegmentReader) -> Result<Self::Child> {
+    fn for_segment(
+        &self,
+        segment_ord: u32,
+        segment_reader: &dyn SegmentReader,
+    ) -> Result<Self::Child> {
         let segment_sort_key_computer = self
             .sort_key_computer
             .segment_sort_key_computer(segment_reader)?;
@@ -63,7 +67,7 @@ where TSortKeyComputer: SortKeyComputer + Send + Sync + 'static
         &self,
         weight: &dyn Weight,
         segment_ord: u32,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
     ) -> crate::Result<Vec<(TSortKeyComputer::SortKey, DocAddress)>> {
         let k = self.doc_range.end;
         let docs = self

src/collector/tests.rs

@@ -5,7 +5,7 @@ use crate::query::{AllQuery, QueryParser};
 use crate::schema::{Schema, FAST, TEXT};
 use crate::time::format_description::well_known::Rfc3339;
 use crate::time::OffsetDateTime;
-use crate::{DateTime, DocAddress, Index, Searcher, TantivyDocument};
+use crate::{DateTime, DocAddress, Index, Searcher, SegmentReader, TantivyDocument};
 
 pub const TEST_COLLECTOR_WITH_SCORE: TestCollector = TestCollector {
     compute_score: true,
@@ -109,7 +109,7 @@ impl Collector for TestCollector {
     fn for_segment(
         &self,
         segment_id: SegmentOrdinal,
-        _reader: &SegmentReader,
+        _reader: &dyn SegmentReader,
     ) -> crate::Result<TestSegmentCollector> {
         Ok(TestSegmentCollector {
             segment_id,
@@ -180,7 +180,7 @@ impl Collector for FastFieldTestCollector {
     fn for_segment(
         &self,
         _: SegmentOrdinal,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<FastFieldSegmentCollector> {
         let reader = segment_reader
             .fast_fields()
@@ -243,7 +243,7 @@ impl Collector for BytesFastFieldTestCollector {
     fn for_segment(
         &self,
         _segment_local_id: u32,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<BytesFastFieldSegmentCollector> {
         let column_opt = segment_reader.fast_fields().bytes(&self.field)?;
         Ok(BytesFastFieldSegmentCollector {

src/collector/top_score_collector.rs

@@ -393,7 +393,7 @@ impl TopDocs {
     /// // This is where we build our collector with our custom score.
     /// let top_docs_by_custom_score = TopDocs
     ///         ::with_limit(10)
-    ///          .tweak_score(move |segment_reader: &SegmentReader| {
+    ///          .tweak_score(move |segment_reader: &dyn SegmentReader| {
     ///             // The argument is a function that returns our scoring
     ///             // function.
     ///             //
@@ -442,7 +442,7 @@ pub struct TweakScoreFn<F>(F);
 
 impl<F, TTweakScoreSortKeyFn, TSortKey> SortKeyComputer for TweakScoreFn<F>
 where
-    F: 'static + Send + Sync + Fn(&SegmentReader) -> TTweakScoreSortKeyFn,
+    F: 'static + Send + Sync + Fn(&dyn SegmentReader) -> TTweakScoreSortKeyFn,
     TTweakScoreSortKeyFn: 'static + Fn(DocId, Score) -> TSortKey,
     TweakScoreSegmentSortKeyComputer<TTweakScoreSortKeyFn>:
         SegmentSortKeyComputer<SortKey = TSortKey, SegmentSortKey = TSortKey>,
@@ -458,7 +458,7 @@ where
 
     fn segment_sort_key_computer(
         &self,
-        segment_reader: &SegmentReader,
+        segment_reader: &dyn SegmentReader,
     ) -> crate::Result<Self::Child> {
         Ok({
             TweakScoreSegmentSortKeyComputer {
@@ -1525,7 +1525,7 @@ mod tests {
         let text_query = query_parser.parse_query("droopy tax")?;
         let collector = TopDocs::with_limit(2)
             .and_offset(1)
-            .order_by(move |_segment_reader: &SegmentReader| move |doc: DocId| doc);
+            .order_by(move |_segment_reader: &dyn SegmentReader| move |doc: DocId| doc);
         let score_docs: Vec<(u32, DocAddress)> =
             index.reader()?.searcher().search(&text_query, &collector)?;
         assert_eq!(
@@ -1543,7 +1543,7 @@ mod tests {
         let text_query = query_parser.parse_query("droopy tax").unwrap();
         let collector = TopDocs::with_limit(2)
             .and_offset(1)
-            .order_by(move |_segment_reader: &SegmentReader| move |doc: DocId| doc);
+            .order_by(move |_segment_reader: &dyn SegmentReader| move |doc: DocId| doc);
         let score_docs: Vec<(u32, DocAddress)> = index
             .reader()
             .unwrap()

src/core/searcher.rs

@@ -4,7 +4,7 @@ use std::{fmt, io};
 
 use crate::collector::Collector;
 use crate::core::Executor;
-use crate::index::{SegmentId, SegmentReader};
+use crate::index::{ArcSegmentReader, SegmentId, SegmentReader};
 use crate::query::{Bm25StatisticsProvider, EnableScoring, Query};
 use crate::schema::document::DocumentDeserialize;
 use crate::schema::{Schema, Term};
@@ -36,7 +36,7 @@ pub struct SearcherGeneration {
 
 impl SearcherGeneration {
     pub(crate) fn from_segment_readers(
-        segment_readers: &[SegmentReader],
+        segment_readers: &[ArcSegmentReader],
         generation_id: u64,
     ) -> Self {
         let mut segment_id_to_del_opstamp = BTreeMap::new();
@@ -133,7 +133,7 @@ impl Searcher {
     pub fn doc_freq(&self, term: &Term) -> crate::Result<u64> {
         let mut total_doc_freq = 0;
         for segment_reader in &self.inner.segment_readers {
-            let inverted_index = segment_reader.inverted_index(term.field())?;
+            let inverted_index = segment_reader.as_ref().inverted_index(term.field())?;
             let doc_freq = inverted_index.doc_freq(term)?;
             total_doc_freq += u64::from(doc_freq);
         }
@@ -146,7 +146,7 @@ impl Searcher {
     pub async fn doc_freq_async(&self, term: &Term) -> crate::Result<u64> {
         let mut total_doc_freq = 0;
         for segment_reader in &self.inner.segment_readers {
-            let inverted_index = segment_reader.inverted_index(term.field())?;
+            let inverted_index = segment_reader.as_ref().inverted_index(term.field())?;
             let doc_freq = inverted_index.doc_freq_async(term).await?;
             total_doc_freq += u64::from(doc_freq);
         }
@@ -154,13 +154,13 @@ impl Searcher {
     }
 
     /// Return the list of segment readers
-    pub fn segment_readers(&self) -> &[SegmentReader] {
+    pub fn segment_readers(&self) -> &[ArcSegmentReader] {
         &self.inner.segment_readers
     }
 
     /// Returns the segment_reader associated with the given segment_ord
-    pub fn segment_reader(&self, segment_ord: u32) -> &SegmentReader {
-        &self.inner.segment_readers[segment_ord as usize]
+    pub fn segment_reader(&self, segment_ord: u32) -> &dyn SegmentReader {
+        self.inner.segment_readers[segment_ord as usize].as_ref()
     }
 
     /// Runs a query on the segment readers wrapped by the searcher.
@@ -229,7 +229,11 @@ impl Searcher {
         let segment_readers = self.segment_readers();
         let fruits = executor.map(
             |(segment_ord, segment_reader)| {
-                collector.collect_segment(weight.as_ref(), segment_ord as u32, segment_reader)
+                collector.collect_segment(
+                    weight.as_ref(),
+                    segment_ord as u32,
+                    segment_reader.as_ref(),
+                )
             },
             segment_readers.iter().enumerate(),
         )?;
@@ -259,7 +263,7 @@ impl From<Arc<SearcherInner>> for Searcher {
 pub(crate) struct SearcherInner {
     schema: Schema,
     index: Index,
-    segment_readers: Vec<SegmentReader>,
+    segment_readers: Vec<ArcSegmentReader>,
     store_readers: Vec<StoreReader>,
     generation: TrackedObject<SearcherGeneration>,
 }
@@ -269,7 +273,7 @@ impl SearcherInner {
     pub(crate) fn new(
         schema: Schema,
         index: Index,
-        segment_readers: Vec<SegmentReader>,
+        segment_readers: Vec<ArcSegmentReader>,
         generation: TrackedObject<SearcherGeneration>,
         doc_store_cache_num_blocks: usize,
     ) -> io::Result<SearcherInner> {
@@ -301,7 +305,7 @@ impl fmt::Debug for Searcher {
         let segment_ids = self
             .segment_readers()
             .iter()
-            .map(SegmentReader::segment_id)
+            .map(|segment_reader| segment_reader.segment_id())
             .collect::<Vec<_>>();
         write!(f, "Searcher({segment_ids:?})")
     }

src/directory/mmap_directory/mod.rs

@@ -676,7 +676,7 @@ mod tests {
             let num_segments = reader.searcher().segment_readers().len();
             assert!(num_segments <= 4);
             let num_components_except_deletes_and_tempstore =
-                crate::index::SegmentComponent::iterator().len() - 1;
+                crate::index::SegmentComponent::iterator().len() - 2;
             let max_num_mmapped = num_components_except_deletes_and_tempstore * num_segments;
             assert_eventually(|| {
                 let num_mmapped = mmap_directory.get_cache_info().mmapped.len();

src/docset.rs

@@ -65,8 +65,8 @@ pub trait DocSet: Send {
     ///   `seek_danger(..)` until it returns `Found`, and get back to a valid state.
     ///
     /// `seek_lower_bound` can be any `DocId` (in the docset or not) as long as it is in
-    /// `(target .. seek_result] U {TERMINATED}` where `seek_result` is the first document in the
-    /// docset greater than to `target`.
+    /// `(target .. seek_result]` where `seek_result` is the first document in the docset greater
+    /// than to `target`.
     ///
     /// `seek_danger` may return `SeekLowerBound(TERMINATED)`.
     ///
@@ -98,7 +98,7 @@ pub trait DocSet: Send {
         if doc == target {
             SeekDangerResult::Found
         } else {
-            SeekDangerResult::SeekLowerBound(doc)
+            SeekDangerResult::SeekLowerBound(self.doc())
         }
     }
 

src/fastfield/mod.rs

@@ -96,7 +96,7 @@ mod tests {
     };
     use crate::time::OffsetDateTime;
     use crate::tokenizer::{LowerCaser, RawTokenizer, TextAnalyzer, TokenizerManager};
-    use crate::{Index, IndexWriter, SegmentReader};
+    use crate::{Index, IndexWriter};
 
     pub static SCHEMA: Lazy<Schema> = Lazy::new(|| {
         let mut schema_builder = Schema::builder();
@@ -430,7 +430,7 @@ mod tests {
             .searcher()
             .segment_readers()
             .iter()
-            .map(SegmentReader::segment_id)
+            .map(|segment_reader| segment_reader.segment_id())
             .collect();
         assert_eq!(segment_ids.len(), 2);
         index_writer.merge(&segment_ids[..]).wait().unwrap();

src/index/index.rs

@@ -14,7 +14,7 @@ use crate::directory::error::OpenReadError;
 use crate::directory::MmapDirectory;
 use crate::directory::{Directory, ManagedDirectory, RamDirectory, INDEX_WRITER_LOCK};
 use crate::error::{DataCorruption, TantivyError};
-use crate::index::{IndexMeta, SegmentId, SegmentMeta, SegmentMetaInventory};
+use crate::index::{IndexMeta, SegmentId, SegmentMeta, SegmentMetaInventory, SegmentReader};
 use crate::indexer::index_writer::{
     IndexWriterOptions, MAX_NUM_THREAD, MEMORY_BUDGET_NUM_BYTES_MIN,
 };
@@ -24,7 +24,7 @@ use crate::reader::{IndexReader, IndexReaderBuilder};
 use crate::schema::document::Document;
 use crate::schema::{Field, FieldType, Schema};
 use crate::tokenizer::{TextAnalyzer, TokenizerManager};
-use crate::SegmentReader;
+use crate::TantivySegmentReader;
 
 fn load_metas(
     directory: &dyn Directory,
@@ -492,7 +492,7 @@ impl Index {
         let segments = self.searchable_segments()?;
         let fields_metadata: Vec<Vec<FieldMetadata>> = segments
             .into_iter()
-            .map(|segment| SegmentReader::open(&segment)?.fields_metadata())
+            .map(|segment| TantivySegmentReader::open(&segment)?.fields_metadata())
             .collect::<Result<_, _>>()?;
         Ok(merge_field_meta_data(fields_metadata))
     }

src/index/index_meta.rs

@@ -1,6 +1,8 @@
 use std::collections::HashSet;
 use std::fmt;
 use std::path::PathBuf;
+use std::sync::atomic::AtomicBool;
+use std::sync::Arc;
 
 use serde::{Deserialize, Serialize};
 
@@ -35,6 +37,7 @@ impl SegmentMetaInventory {
         let inner = InnerSegmentMeta {
             segment_id,
             max_doc,
+            include_temp_doc_store: Arc::new(AtomicBool::new(true)),
             deletes: None,
         };
         SegmentMeta::from(self.inventory.track(inner))
@@ -82,6 +85,15 @@ impl SegmentMeta {
         self.tracked.segment_id
     }
 
+    /// Removes the Component::TempStore from the alive list and
+    /// therefore marks the temp docstore file to be deleted by
+    /// the garbage collection.
+    pub fn untrack_temp_docstore(&self) {
+        self.tracked
+            .include_temp_doc_store
+            .store(false, std::sync::atomic::Ordering::Relaxed);
+    }
+
     /// Returns the number of deleted documents.
     pub fn num_deleted_docs(&self) -> u32 {
         self.tracked
@@ -99,9 +111,20 @@ impl SegmentMeta {
     /// is by removing all files that have been created by tantivy
     /// and are not used by any segment anymore.
     pub fn list_files(&self) -> HashSet<PathBuf> {
-        SegmentComponent::iterator()
-            .map(|component| self.relative_path(*component))
-            .collect::<HashSet<PathBuf>>()
+        if self
+            .tracked
+            .include_temp_doc_store
+            .load(std::sync::atomic::Ordering::Relaxed)
+        {
+            SegmentComponent::iterator()
+                .map(|component| self.relative_path(*component))
+                .collect::<HashSet<PathBuf>>()
+        } else {
+            SegmentComponent::iterator()
+                .filter(|comp| *comp != &SegmentComponent::TempStore)
+                .map(|component| self.relative_path(*component))
+                .collect::<HashSet<PathBuf>>()
+        }
     }
 
     /// Returns the relative path of a component of our segment.
@@ -115,6 +138,7 @@ impl SegmentMeta {
             SegmentComponent::Positions => ".pos".to_string(),
             SegmentComponent::Terms => ".term".to_string(),
             SegmentComponent::Store => ".store".to_string(),
+            SegmentComponent::TempStore => ".store.temp".to_string(),
             SegmentComponent::FastFields => ".fast".to_string(),
             SegmentComponent::FieldNorms => ".fieldnorm".to_string(),
             SegmentComponent::Delete => format!(".{}.del", self.delete_opstamp().unwrap_or(0)),
@@ -159,6 +183,7 @@ impl SegmentMeta {
             segment_id: inner_meta.segment_id,
             max_doc,
             deletes: None,
+            include_temp_doc_store: Arc::new(AtomicBool::new(true)),
         });
         SegmentMeta { tracked }
     }
@@ -177,6 +202,7 @@ impl SegmentMeta {
         let tracked = self.tracked.map(move |inner_meta| InnerSegmentMeta {
             segment_id: inner_meta.segment_id,
             max_doc: inner_meta.max_doc,
+            include_temp_doc_store: Arc::new(AtomicBool::new(true)),
             deletes: Some(delete_meta),
         });
         SegmentMeta { tracked }
@@ -188,6 +214,14 @@ struct InnerSegmentMeta {
     segment_id: SegmentId,
     max_doc: u32,
     pub deletes: Option<DeleteMeta>,
+    /// If you want to avoid the SegmentComponent::TempStore file to be covered by
+    /// garbage collection and deleted, set this to true. This is used during merge.
+    #[serde(skip)]
+    #[serde(default = "default_temp_store")]
+    pub(crate) include_temp_doc_store: Arc<AtomicBool>,
+}
+fn default_temp_store() -> Arc<AtomicBool> {
+    Arc::new(AtomicBool::new(false))
 }
 
 impl InnerSegmentMeta {

src/index/inverted_index_reader.rs

@@ -1,4 +1,9 @@
+#[cfg(feature = "quickwit")]
+use std::future::Future;
 use std::io;
+#[cfg(feature = "quickwit")]
+use std::pin::Pin;
+use std::sync::Arc;
 
 use common::json_path_writer::JSON_END_OF_PATH;
 use common::{BinarySerializable, ByteCount};
@@ -27,7 +32,102 @@ use crate::termdict::TermDictionary;
 ///
 /// `InvertedIndexReader` are created by calling
 /// [`SegmentReader::inverted_index()`](crate::SegmentReader::inverted_index).
-pub struct InvertedIndexReader {
+pub trait InvertedIndexReader: Send + Sync {
+    /// Returns the term info associated with the term.
+    fn get_term_info(&self, term: &Term) -> io::Result<Option<TermInfo>>;
+
+    /// Return the term dictionary datastructure.
+    fn terms(&self) -> &TermDictionary;
+
+    /// Return the fields and types encoded in the dictionary in lexicographic order.
+    /// Only valid on JSON fields.
+    ///
+    /// Notice: This requires a full scan and therefore **very expensive**.
+    /// TODO: Move to sstable to use the index.
+    #[doc(hidden)]
+    fn list_encoded_json_fields(&self) -> io::Result<Vec<InvertedIndexFieldSpace>>;
+
+    /// Returns a block postings given a `Term`.
+    /// This method is for an advanced usage only.
+    ///
+    /// Most users should prefer using [`Self::read_postings()`] instead.
+    fn read_block_postings(
+        &self,
+        term: &Term,
+        option: IndexRecordOption,
+    ) -> io::Result<Option<BlockSegmentPostings>>;
+
+    /// Returns a block postings given a `term_info`.
+    /// This method is for an advanced usage only.
+    ///
+    /// Most users should prefer using [`Self::read_postings()`] instead.
+    fn read_block_postings_from_terminfo(
+        &self,
+        term_info: &TermInfo,
+        requested_option: IndexRecordOption,
+    ) -> io::Result<BlockSegmentPostings>;
+
+    /// Returns a posting object given a `term_info`.
+    /// This method is for an advanced usage only.
+    ///
+    /// Most users should prefer using [`Self::read_postings()`] instead.
+    fn read_postings_from_terminfo(
+        &self,
+        term_info: &TermInfo,
+        option: IndexRecordOption,
+    ) -> io::Result<SegmentPostings>;
+
+    /// Returns the total number of tokens recorded for all documents
+    /// (including deleted documents).
+    fn total_num_tokens(&self) -> u64;
+
+    /// Returns the segment postings associated with the term, and with the given option,
+    /// or `None` if the term has never been encountered and indexed.
+    fn read_postings(
+        &self,
+        term: &Term,
+        option: IndexRecordOption,
+    ) -> io::Result<Option<SegmentPostings>>;
+
+    /// Returns the number of documents containing the term.
+    fn doc_freq(&self, term: &Term) -> io::Result<u32>;
+
+    /// Returns the number of documents containing the term asynchronously.
+    #[cfg(feature = "quickwit")]
+    fn doc_freq_async<'a>(&'a self, term: &'a Term) -> BoxFuture<'a, io::Result<u32>>;
+
+    /// Warmup a block postings given a `Term`.
+    /// This method is for an advanced usage only.
+    ///
+    /// returns a boolean, whether the term was found in the dictionary
+    #[cfg(feature = "quickwit")]
+    fn warm_postings<'a>(
+        &'a self,
+        term: &'a Term,
+        with_positions: bool,
+    ) -> BoxFuture<'a, io::Result<bool>>;
+
+    /// Warmup the block postings for all terms.
+    /// This method is for an advanced usage only.
+    ///
+    /// If you know which terms to pre-load, prefer using [`Self::warm_postings`] or
+    /// [`Self::warm_postings`] instead.
+    #[cfg(feature = "quickwit")]
+    fn warm_postings_full<'a>(&'a self, with_positions: bool) -> BoxFuture<'a, io::Result<()>>;
+}
+
+/// Convenient alias for an atomically reference counted inverted index reader handle.
+pub type ArcInvertedIndexReader = Arc<dyn InvertedIndexReader>;
+
+#[cfg(feature = "quickwit")]
+/// Boxed future used by async inverted index reader methods.
+pub type BoxFuture<'a, T> = Pin<Box<dyn Future<Output = T> + Send + 'a>>;
+
+/// The tantivy inverted index reader is in charge of accessing
+/// the inverted index associated with a specific field.
+///
+/// This is the default implementation of [`InvertedIndexReader`].
+pub struct TantivyInvertedIndexReader {
     termdict: TermDictionary,
     postings_file_slice: FileSlice,
     positions_file_slice: FileSlice,
@@ -36,11 +136,16 @@ pub struct InvertedIndexReader {
 }
 
 /// Object that records the amount of space used by a field in an inverted index.
-pub(crate) struct InvertedIndexFieldSpace {
+pub struct InvertedIndexFieldSpace {
+    /// The JSON field name (without the parent field).
     pub field_name: String,
+    /// The field type encoded in the term dictionary.
     pub field_type: Type,
+    /// Total postings size for this field.
     pub postings_size: ByteCount,
+    /// Total positions size for this field.
     pub positions_size: ByteCount,
+    /// Number of terms for this field.
     pub num_terms: u64,
 }
 
@@ -62,16 +167,16 @@ impl InvertedIndexFieldSpace {
     }
 }
 
-impl InvertedIndexReader {
+impl TantivyInvertedIndexReader {
     pub(crate) fn new(
         termdict: TermDictionary,
         postings_file_slice: FileSlice,
         positions_file_slice: FileSlice,
         record_option: IndexRecordOption,
-    ) -> io::Result<InvertedIndexReader> {
+    ) -> io::Result<TantivyInvertedIndexReader> {
         let (total_num_tokens_slice, postings_body) = postings_file_slice.split(8);
         let total_num_tokens = u64::deserialize(&mut total_num_tokens_slice.read_bytes()?)?;
-        Ok(InvertedIndexReader {
+        Ok(TantivyInvertedIndexReader {
             termdict,
             postings_file_slice: postings_body,
             positions_file_slice,
@@ -82,8 +187,8 @@ impl InvertedIndexReader {
 
     /// Creates an empty `InvertedIndexReader` object, which
     /// contains no terms at all.
-    pub fn empty(record_option: IndexRecordOption) -> InvertedIndexReader {
-        InvertedIndexReader {
+    pub fn empty(record_option: IndexRecordOption) -> TantivyInvertedIndexReader {
+        TantivyInvertedIndexReader {
             termdict: TermDictionary::empty(),
             postings_file_slice: FileSlice::empty(),
             positions_file_slice: FileSlice::empty(),
@@ -160,29 +265,6 @@ impl InvertedIndexReader {
         Ok(fields)
     }
 
-    /// Resets the block segment to another position of the postings
-    /// file.
-    ///
-    /// This is useful for enumerating through a list of terms,
-    /// and consuming the associated posting lists while avoiding
-    /// reallocating a [`BlockSegmentPostings`].
-    ///
-    /// # Warning
-    ///
-    /// This does not reset the positions list.
-    pub fn reset_block_postings_from_terminfo(
-        &self,
-        term_info: &TermInfo,
-        block_postings: &mut BlockSegmentPostings,
-    ) -> io::Result<()> {
-        let postings_slice = self
-            .postings_file_slice
-            .slice(term_info.postings_range.clone());
-        let postings_bytes = postings_slice.read_bytes()?;
-        block_postings.reset(term_info.doc_freq, postings_bytes)?;
-        Ok(())
-    }
-
     /// Returns a block postings given a `Term`.
     /// This method is for an advanced usage only.
     ///
@@ -282,7 +364,7 @@ impl InvertedIndexReader {
 }
 
 #[cfg(feature = "quickwit")]
-impl InvertedIndexReader {
+impl TantivyInvertedIndexReader {
     pub(crate) async fn get_term_info_async(&self, term: &Term) -> io::Result<Option<TermInfo>> {
         self.termdict.get_async(term.serialized_value_bytes()).await
     }
@@ -492,3 +574,84 @@ impl InvertedIndexReader {
             .unwrap_or(0u32))
     }
 }
+
+impl InvertedIndexReader for TantivyInvertedIndexReader {
+    fn get_term_info(&self, term: &Term) -> io::Result<Option<TermInfo>> {
+        TantivyInvertedIndexReader::get_term_info(self, term)
+    }
+
+    fn terms(&self) -> &TermDictionary {
+        TantivyInvertedIndexReader::terms(self)
+    }
+
+    fn list_encoded_json_fields(&self) -> io::Result<Vec<InvertedIndexFieldSpace>> {
+        TantivyInvertedIndexReader::list_encoded_json_fields(self)
+    }
+
+    fn read_block_postings(
+        &self,
+        term: &Term,
+        option: IndexRecordOption,
+    ) -> io::Result<Option<BlockSegmentPostings>> {
+        TantivyInvertedIndexReader::read_block_postings(self, term, option)
+    }
+
+    fn read_block_postings_from_terminfo(
+        &self,
+        term_info: &TermInfo,
+        requested_option: IndexRecordOption,
+    ) -> io::Result<BlockSegmentPostings> {
+        TantivyInvertedIndexReader::read_block_postings_from_terminfo(
+            self,
+            term_info,
+            requested_option,
+        )
+    }
+
+    fn read_postings_from_terminfo(
+        &self,
+        term_info: &TermInfo,
+        option: IndexRecordOption,
+    ) -> io::Result<SegmentPostings> {
+        TantivyInvertedIndexReader::read_postings_from_terminfo(self, term_info, option)
+    }
+
+    fn total_num_tokens(&self) -> u64 {
+        TantivyInvertedIndexReader::total_num_tokens(self)
+    }
+
+    fn read_postings(
+        &self,
+        term: &Term,
+        option: IndexRecordOption,
+    ) -> io::Result<Option<SegmentPostings>> {
+        TantivyInvertedIndexReader::read_postings(self, term, option)
+    }
+
+    fn doc_freq(&self, term: &Term) -> io::Result<u32> {
+        TantivyInvertedIndexReader::doc_freq(self, term)
+    }
+
+    #[cfg(feature = "quickwit")]
+    fn doc_freq_async<'a>(&'a self, term: &'a Term) -> BoxFuture<'a, io::Result<u32>> {
+        Box::pin(async move { TantivyInvertedIndexReader::doc_freq_async(self, term).await })
+    }
+
+    #[cfg(feature = "quickwit")]
+    fn warm_postings<'a>(
+        &'a self,
+        term: &'a Term,
+        with_positions: bool,
+    ) -> BoxFuture<'a, io::Result<bool>> {
+        Box::pin(async move {
+            TantivyInvertedIndexReader::warm_postings(self, term, with_positions).await
+        })
+    }
+
+    #[cfg(feature = "quickwit")]
+    fn warm_postings_full<'a>(&'a self, with_positions: bool) -> BoxFuture<'a, io::Result<()>> {
+        Box::pin(async move {
+            TantivyInvertedIndexReader::warm_postings_full(self, with_positions).await
+        })
+    }
+}

src/index/mod.rs

@@ -13,8 +13,13 @@ mod segment_reader;
 pub use self::index::{Index, IndexBuilder};
 pub(crate) use self::index_meta::SegmentMetaInventory;
 pub use self::index_meta::{IndexMeta, IndexSettings, Order, SegmentMeta};
-pub use self::inverted_index_reader::InvertedIndexReader;
+pub use self::inverted_index_reader::{
+    ArcInvertedIndexReader, InvertedIndexFieldSpace, InvertedIndexReader,
+    TantivyInvertedIndexReader,
+};
 pub use self::segment::Segment;
 pub use self::segment_component::SegmentComponent;
 pub use self::segment_id::SegmentId;
-pub use self::segment_reader::{FieldMetadata, SegmentReader};
+pub use self::segment_reader::{
+    ArcSegmentReader, FieldMetadata, SegmentReader, TantivySegmentReader,
+};

src/index/segment_component.rs

@@ -23,6 +23,8 @@ pub enum SegmentComponent {
     /// Accessing a document from the store is relatively slow, as it
     /// requires to decompress the entire block it belongs to.
     Store,
+    /// Temporary storage of the documents, before streamed to `Store`.
+    TempStore,
     /// Bitset describing which document of the segment is alive.
     /// (It was representing deleted docs but changed to represent alive docs from v0.17)
     Delete,
@@ -31,13 +33,14 @@ pub enum SegmentComponent {
 impl SegmentComponent {
     /// Iterates through the components.
     pub fn iterator() -> slice::Iter<'static, SegmentComponent> {
-        static SEGMENT_COMPONENTS: [SegmentComponent; 7] = [
+        static SEGMENT_COMPONENTS: [SegmentComponent; 8] = [
             SegmentComponent::Postings,
             SegmentComponent::Positions,
             SegmentComponent::FastFields,
             SegmentComponent::FieldNorms,
             SegmentComponent::Terms,
             SegmentComponent::Store,
+            SegmentComponent::TempStore,
             SegmentComponent::Delete,
         ];
         SEGMENT_COMPONENTS.iter()

src/index/segment_reader.rs

@@ -9,8 +9,10 @@ use itertools::Itertools;
 use crate::directory::{CompositeFile, FileSlice};
 use crate::error::DataCorruption;
 use crate::fastfield::{intersect_alive_bitsets, AliveBitSet, FacetReader, FastFieldReaders};
-use crate::fieldnorm::{FieldNormReader, FieldNormReaders};
-use crate::index::{InvertedIndexReader, Segment, SegmentComponent, SegmentId};
+use crate::fieldnorm::FieldNormReaders;
+use crate::index::{
+    ArcInvertedIndexReader, Segment, SegmentComponent, SegmentId, TantivyInvertedIndexReader,
+};
 use crate::json_utils::json_path_sep_to_dot;
 use crate::schema::{Field, IndexRecordOption, Schema, Type};
 use crate::space_usage::SegmentSpaceUsage;
@@ -18,6 +20,93 @@ use crate::store::StoreReader;
 use crate::termdict::TermDictionary;
 use crate::{DocId, Opstamp};
 
+/// Abstraction over a segment reader for accessing all data structures of a segment.
+///
+/// This trait exists to decouple the query layer from the concrete on-disk layout. Alternative
+/// codecs can implement it to expose their own segment representation.
+pub trait SegmentReader: Send + Sync {
+    /// Highest document id ever attributed in this segment + 1.
+    fn max_doc(&self) -> DocId;
+
+    /// Number of alive documents. Deleted documents are not counted.
+    fn num_docs(&self) -> DocId;
+
+    /// Returns the schema of the index this segment belongs to.
+    fn schema(&self) -> &Schema;
+
+    /// Return the number of documents that have been deleted in the segment.
+    fn num_deleted_docs(&self) -> DocId {
+        self.max_doc() - self.num_docs()
+    }
+
+    /// Returns true if some of the documents of the segment have been deleted.
+    fn has_deletes(&self) -> bool {
+        self.num_deleted_docs() > 0
+    }
+
+    /// Accessor to a segment's fast field reader.
+    fn fast_fields(&self) -> &FastFieldReaders;
+
+    /// Accessor to the `FacetReader` associated with a given `Field`.
+    fn facet_reader(&self, field_name: &str) -> crate::Result<FacetReader> {
+        let schema = self.schema();
+        let field = schema.get_field(field_name)?;
+        let field_entry = schema.get_field_entry(field);
+        if field_entry.field_type().value_type() != Type::Facet {
+            return Err(crate::TantivyError::SchemaError(format!(
+                "`{field_name}` is not a facet field.`"
+            )));
+        }
+        let Some(facet_column) = self.fast_fields().str(field_name)? else {
+            panic!("Facet Field `{field_name}` is missing. This should not happen");
+        };
+        Ok(FacetReader::new(facet_column))
+    }
+
+    /// Accessor to the segment's field norms readers container.
+    fn fieldnorms_readers(&self) -> &FieldNormReaders;
+
+    /// Accessor to the segment's [`StoreReader`](crate::store::StoreReader).
+    fn get_store_reader(&self, cache_num_blocks: usize) -> io::Result<StoreReader>;
+
+    /// Returns a field reader associated with the field given in argument.
+    fn inverted_index(&self, field: Field) -> crate::Result<ArcInvertedIndexReader>;
+
+    /// Returns the list of fields that have been indexed in the segment.
+    fn fields_metadata(&self) -> crate::Result<Vec<FieldMetadata>>;
+
+    /// Returns the segment id
+    fn segment_id(&self) -> SegmentId;
+
+    /// Returns the delete opstamp
+    fn delete_opstamp(&self) -> Option<Opstamp>;
+
+    /// Returns the bitset representing the alive `DocId`s.
+    fn alive_bitset(&self) -> Option<&AliveBitSet>;
+
+    /// Returns true if the `doc` is marked as deleted.
+    fn is_deleted(&self, doc: DocId) -> bool {
+        self.alive_bitset()
+            .map(|alive_bitset| alive_bitset.is_deleted(doc))
+            .unwrap_or(false)
+    }
+
+    /// Returns an iterator that will iterate over the alive document ids
+    fn doc_ids_alive(&self) -> Box<dyn Iterator<Item = DocId> + Send + '_> {
+        if let Some(alive_bitset) = &self.alive_bitset() {
+            Box::new(alive_bitset.iter_alive())
+        } else {
+            Box::new(0u32..self.max_doc())
+        }
+    }
+
+    /// Summarize total space usage of this segment.
+    fn space_usage(&self) -> io::Result<SegmentSpaceUsage>;
+}
+
+/// Convenient alias for an atomically reference counted segment reader handle.
+pub type ArcSegmentReader = Arc<dyn SegmentReader>;
+
 /// Entry point to access all of the datastructures of the `Segment`
 ///
 /// - term dictionary
@@ -29,8 +118,8 @@ use crate::{DocId, Opstamp};
 /// The segment reader has a very low memory footprint,
 /// as close to all of the memory data is mmapped.
 #[derive(Clone)]
-pub struct SegmentReader {
-    inv_idx_reader_cache: Arc<RwLock<HashMap<Field, Arc<InvertedIndexReader>>>>,
+pub struct TantivySegmentReader {
+    inv_idx_reader_cache: Arc<RwLock<HashMap<Field, ArcInvertedIndexReader>>>,
 
     segment_id: SegmentId,
     delete_opstamp: Option<Opstamp>,
@@ -49,98 +138,9 @@ pub struct SegmentReader {
     schema: Schema,
 }
 
-impl SegmentReader {
-    /// Returns the highest document id ever attributed in
-    /// this segment + 1.
-    pub fn max_doc(&self) -> DocId {
-        self.max_doc
-    }
-
-    /// Returns the number of alive documents.
-    /// Deleted documents are not counted.
-    pub fn num_docs(&self) -> DocId {
-        self.num_docs
-    }
-
-    /// Returns the schema of the index this segment belongs to.
-    pub fn schema(&self) -> &Schema {
-        &self.schema
-    }
-
-    /// Return the number of documents that have been
-    /// deleted in the segment.
-    pub fn num_deleted_docs(&self) -> DocId {
-        self.max_doc - self.num_docs
-    }
-
-    /// Returns true if some of the documents of the segment have been deleted.
-    pub fn has_deletes(&self) -> bool {
-        self.num_deleted_docs() > 0
-    }
-
-    /// Accessor to a segment's fast field reader given a field.
-    ///
-    /// Returns the u64 fast value reader if the field
-    /// is a u64 field indexed as "fast".
-    ///
-    /// Return a FastFieldNotAvailableError if the field is not
-    /// declared as a fast field in the schema.
-    ///
-    /// # Panics
-    /// May panic if the index is corrupted.
-    pub fn fast_fields(&self) -> &FastFieldReaders {
-        &self.fast_fields_readers
-    }
-
-    /// Accessor to the `FacetReader` associated with a given `Field`.
-    pub fn facet_reader(&self, field_name: &str) -> crate::Result<FacetReader> {
-        let schema = self.schema();
-        let field = schema.get_field(field_name)?;
-        let field_entry = schema.get_field_entry(field);
-        if field_entry.field_type().value_type() != Type::Facet {
-            return Err(crate::TantivyError::SchemaError(format!(
-                "`{field_name}` is not a facet field.`"
-            )));
-        }
-        let Some(facet_column) = self.fast_fields().str(field_name)? else {
-            panic!("Facet Field `{field_name}` is missing. This should not happen");
-        };
-        Ok(FacetReader::new(facet_column))
-    }
-
-    /// Accessor to the segment's `Field norms`'s reader.
-    ///
-    /// Field norms are the length (in tokens) of the fields.
-    /// It is used in the computation of the [TfIdf](https://fulmicoton.gitbooks.io/tantivy-doc/content/tfidf.html).
-    ///
-    /// They are simply stored as a fast field, serialized in
-    /// the `.fieldnorm` file of the segment.
-    pub fn get_fieldnorms_reader(&self, field: Field) -> crate::Result<FieldNormReader> {
-        self.fieldnorm_readers.get_field(field)?.ok_or_else(|| {
-            let field_name = self.schema.get_field_name(field);
-            let err_msg = format!(
-                "Field norm not found for field {field_name:?}. Was the field set to record norm \
-                 during indexing?"
-            );
-            crate::TantivyError::SchemaError(err_msg)
-        })
-    }
-
-    #[doc(hidden)]
-    pub fn fieldnorms_readers(&self) -> &FieldNormReaders {
-        &self.fieldnorm_readers
-    }
-
-    /// Accessor to the segment's [`StoreReader`](crate::store::StoreReader).
-    ///
-    /// `cache_num_blocks` sets the number of decompressed blocks to be cached in an LRU.
-    /// The size of blocks is configurable, this should be reflexted in the
-    pub fn get_store_reader(&self, cache_num_blocks: usize) -> io::Result<StoreReader> {
-        StoreReader::open(self.store_file.clone(), cache_num_blocks)
-    }
-
+impl TantivySegmentReader {
     /// Open a new segment for reading.
-    pub fn open(segment: &Segment) -> crate::Result<SegmentReader> {
+    pub fn open(segment: &Segment) -> crate::Result<TantivySegmentReader> {
         Self::open_with_custom_alive_set(segment, None)
     }
 
@@ -148,7 +148,7 @@ impl SegmentReader {
     pub fn open_with_custom_alive_set(
         segment: &Segment,
         custom_bitset: Option<AliveBitSet>,
-    ) -> crate::Result<SegmentReader> {
+    ) -> crate::Result<TantivySegmentReader> {
         let termdict_file = segment.open_read(SegmentComponent::Terms)?;
         let termdict_composite = CompositeFile::open(&termdict_file)?;
 
@@ -190,7 +190,7 @@ impl SegmentReader {
             .map(|alive_bitset| alive_bitset.num_alive_docs() as u32)
             .unwrap_or(max_doc);
 
-        Ok(SegmentReader {
+        Ok(TantivySegmentReader {
             inv_idx_reader_cache: Default::default(),
             num_docs,
             max_doc,
@@ -206,6 +206,52 @@ impl SegmentReader {
             schema,
         })
     }
+}
+
+impl SegmentReader for TantivySegmentReader {
+    /// Returns the highest document id ever attributed in
+    /// this segment + 1.
+    fn max_doc(&self) -> DocId {
+        self.max_doc
+    }
+
+    /// Returns the number of alive documents.
+    /// Deleted documents are not counted.
+    fn num_docs(&self) -> DocId {
+        self.num_docs
+    }
+
+    /// Returns the schema of the index this segment belongs to.
+    fn schema(&self) -> &Schema {
+        &self.schema
+    }
+
+    /// Accessor to a segment's fast field reader given a field.
+    ///
+    /// Returns the u64 fast value reader if the field
+    /// is a u64 field indexed as "fast".
+    ///
+    /// Return a FastFieldNotAvailableError if the field is not
+    /// declared as a fast field in the schema.
+    ///
+    /// # Panics
+    /// May panic if the index is corrupted.
+    fn fast_fields(&self) -> &FastFieldReaders {
+        &self.fast_fields_readers
+    }
+
+    #[doc(hidden)]
+    fn fieldnorms_readers(&self) -> &FieldNormReaders {
+        &self.fieldnorm_readers
+    }
+
+    /// Accessor to the segment's [`StoreReader`](crate::store::StoreReader).
+    ///
+    /// `cache_num_blocks` sets the number of decompressed blocks to be cached in an LRU.
+    /// The size of blocks is configurable, this should be reflexted in the
+    fn get_store_reader(&self, cache_num_blocks: usize) -> io::Result<StoreReader> {
+        StoreReader::open(self.store_file.clone(), cache_num_blocks)
+    }
 
     /// Returns a field reader associated with the field given in argument.
     /// If the field was not present in the index during indexing time,
@@ -219,7 +265,7 @@ impl SegmentReader {
     /// is returned.
     /// Similarly, if the field is marked as indexed but no term has been indexed for the given
     /// index, an empty `InvertedIndexReader` is returned (but no warning is logged).
-    pub fn inverted_index(&self, field: Field) -> crate::Result<Arc<InvertedIndexReader>> {
+    fn inverted_index(&self, field: Field) -> crate::Result<ArcInvertedIndexReader> {
         if let Some(inv_idx_reader) = self
             .inv_idx_reader_cache
             .read()
@@ -244,7 +290,7 @@ impl SegmentReader {
             //
             // Returns an empty inverted index.
             let record_option = record_option_opt.unwrap_or(IndexRecordOption::Basic);
-            return Ok(Arc::new(InvertedIndexReader::empty(record_option)));
+            return Ok(Arc::new(TantivyInvertedIndexReader::empty(record_option)));
         }
 
         let record_option = record_option_opt.unwrap();
@@ -268,7 +314,7 @@ impl SegmentReader {
             DataCorruption::comment_only(error_msg)
         })?;
 
-        let inv_idx_reader = Arc::new(InvertedIndexReader::new(
+        let inv_idx_reader: ArcInvertedIndexReader = Arc::new(TantivyInvertedIndexReader::new(
             TermDictionary::open(termdict_file)?,
             postings_file,
             positions_file,
@@ -298,7 +344,7 @@ impl SegmentReader {
     /// Disclaimer: Some fields may not be listed here. For instance, if the schema contains a json
     /// field that is not indexed nor a fast field but is stored, it is possible for the field
     /// to not be listed.
-    pub fn fields_metadata(&self) -> crate::Result<Vec<FieldMetadata>> {
+    fn fields_metadata(&self) -> crate::Result<Vec<FieldMetadata>> {
         let mut indexed_fields: Vec<FieldMetadata> = Vec::new();
         let mut map_to_canonical = FnvHashMap::default();
         for (field, field_entry) in self.schema().fields() {
@@ -420,39 +466,22 @@ impl SegmentReader {
     }
 
     /// Returns the segment id
-    pub fn segment_id(&self) -> SegmentId {
+    fn segment_id(&self) -> SegmentId {
         self.segment_id
     }
 
     /// Returns the delete opstamp
-    pub fn delete_opstamp(&self) -> Option<Opstamp> {
+    fn delete_opstamp(&self) -> Option<Opstamp> {
         self.delete_opstamp
     }
 
     /// Returns the bitset representing the alive `DocId`s.
-    pub fn alive_bitset(&self) -> Option<&AliveBitSet> {
+    fn alive_bitset(&self) -> Option<&AliveBitSet> {
         self.alive_bitset_opt.as_ref()
     }
 
-    /// Returns true if the `doc` is marked
-    /// as deleted.
-    pub fn is_deleted(&self, doc: DocId) -> bool {
-        self.alive_bitset()
-            .map(|alive_bitset| alive_bitset.is_deleted(doc))
-            .unwrap_or(false)
-    }
-
-    /// Returns an iterator that will iterate over the alive document ids
-    pub fn doc_ids_alive(&self) -> Box<dyn Iterator<Item = DocId> + Send + '_> {
-        if let Some(alive_bitset) = &self.alive_bitset_opt {
-            Box::new(alive_bitset.iter_alive())
-        } else {
-            Box::new(0u32..self.max_doc)
-        }
-    }
-
     /// Summarize total space usage of this segment.
-    pub fn space_usage(&self) -> io::Result<SegmentSpaceUsage> {
+    fn space_usage(&self) -> io::Result<SegmentSpaceUsage> {
         Ok(SegmentSpaceUsage::new(
             self.num_docs(),
             self.termdict_composite.space_usage(self.schema()),
@@ -576,7 +605,7 @@ fn intersect_alive_bitset(
     }
 }
 
-impl fmt::Debug for SegmentReader {
+impl fmt::Debug for TantivySegmentReader {
     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
         write!(f, "SegmentReader({:?})", self.segment_id)
     }

src/indexer/delete_queue.rs

@@ -250,11 +250,15 @@ mod tests {
 
     struct DummyWeight;
     impl Weight for DummyWeight {
-        fn scorer(&self, _reader: &SegmentReader, _boost: Score) -> crate::Result<Box<dyn Scorer>> {
+        fn scorer(
+            &self,
+            _reader: &dyn SegmentReader,
+            _boost: Score,
+        ) -> crate::Result<Box<dyn Scorer>> {
             Err(crate::TantivyError::InternalError("dummy impl".to_owned()))
         }
 
-        fn explain(&self, _reader: &SegmentReader, _doc: DocId) -> crate::Result<Explanation> {
+        fn explain(&self, _reader: &dyn SegmentReader, _doc: DocId) -> crate::Result<Explanation> {
             Err(crate::TantivyError::InternalError("dummy impl".to_owned()))
         }
     }

src/indexer/index_writer.rs

@@ -12,7 +12,9 @@ use super::{AddBatch, AddBatchReceiver, AddBatchSender, PreparedCommit};
 use crate::directory::{DirectoryLock, GarbageCollectionResult, TerminatingWrite};
 use crate::error::TantivyError;
 use crate::fastfield::write_alive_bitset;
-use crate::index::{Index, Segment, SegmentComponent, SegmentId, SegmentMeta, SegmentReader};
+use crate::index::{
+    Index, Segment, SegmentComponent, SegmentId, SegmentMeta, SegmentReader, TantivySegmentReader,
+};
 use crate::indexer::delete_queue::{DeleteCursor, DeleteQueue};
 use crate::indexer::doc_opstamp_mapping::DocToOpstampMapping;
 use crate::indexer::index_writer_status::IndexWriterStatus;
@@ -94,7 +96,7 @@ pub struct IndexWriter<D: Document = TantivyDocument> {
 
 fn compute_deleted_bitset(
     alive_bitset: &mut BitSet,
-    segment_reader: &SegmentReader,
+    segment_reader: &dyn SegmentReader,
     delete_cursor: &mut DeleteCursor,
     doc_opstamps: &DocToOpstampMapping,
     target_opstamp: Opstamp,
@@ -143,7 +145,7 @@ pub fn advance_deletes(
         return Ok(());
     }
 
-    let segment_reader = SegmentReader::open(&segment)?;
+    let segment_reader = TantivySegmentReader::open(&segment)?;
 
     let max_doc = segment_reader.max_doc();
     let mut alive_bitset: BitSet = match segment_entry.alive_bitset() {
@@ -218,7 +220,7 @@ fn index_documents<D: Document>(
     let alive_bitset_opt = apply_deletes(&segment_with_max_doc, &mut delete_cursor, &doc_opstamps)?;
 
     let meta = segment_with_max_doc.meta().clone();
-
+    meta.untrack_temp_docstore();
     // update segment_updater inventory to remove tempstore
     let segment_entry = SegmentEntry::new(meta, delete_cursor, alive_bitset_opt);
     segment_updater.schedule_add_segment(segment_entry).wait()?;
@@ -243,7 +245,7 @@ fn apply_deletes(
         .max()
         .expect("Empty DocOpstamp is forbidden");
 
-    let segment_reader = SegmentReader::open(segment)?;
+    let segment_reader = TantivySegmentReader::open(segment)?;
     let doc_to_opstamps = DocToOpstampMapping::WithMap(doc_opstamps);
 
     let max_doc = segment.meta().max_doc();

src/indexer/merger.rs

@@ -1,5 +1,3 @@
-use std::sync::Arc;
-
 use columnar::{
     ColumnType, ColumnarReader, MergeRowOrder, RowAddr, ShuffleMergeOrder, StackMergeOrder,
 };
@@ -12,14 +10,14 @@ use crate::docset::{DocSet, TERMINATED};
 use crate::error::DataCorruption;
 use crate::fastfield::AliveBitSet;
 use crate::fieldnorm::{FieldNormReader, FieldNormReaders, FieldNormsSerializer, FieldNormsWriter};
-use crate::index::{Segment, SegmentComponent, SegmentReader};
+use crate::index::{Segment, SegmentComponent, SegmentReader, TantivySegmentReader};
 use crate::indexer::doc_id_mapping::{MappingType, SegmentDocIdMapping};
 use crate::indexer::SegmentSerializer;
 use crate::postings::{InvertedIndexSerializer, Postings, SegmentPostings};
 use crate::schema::{value_type_to_column_type, Field, FieldType, Schema};
 use crate::store::StoreWriter;
 use crate::termdict::{TermMerger, TermOrdinal};
-use crate::{DocAddress, DocId, InvertedIndexReader};
+use crate::{ArcInvertedIndexReader, DocAddress, DocId};
 
 /// Segment's max doc must be `< MAX_DOC_LIMIT`.
 ///
@@ -27,7 +25,7 @@ use crate::{DocAddress, DocId, InvertedIndexReader};
 pub const MAX_DOC_LIMIT: u32 = 1 << 31;
 
 fn estimate_total_num_tokens_in_single_segment(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field: Field,
 ) -> crate::Result<u64> {
     // There are no deletes. We can simply use the exact value saved into the posting list.
@@ -68,7 +66,7 @@ fn estimate_total_num_tokens_in_single_segment(
     Ok((segment_num_tokens as f64 * ratio) as u64)
 }
 
-fn estimate_total_num_tokens(readers: &[SegmentReader], field: Field) -> crate::Result<u64> {
+fn estimate_total_num_tokens(readers: &[TantivySegmentReader], field: Field) -> crate::Result<u64> {
     let mut total_num_tokens: u64 = 0;
     for reader in readers {
         total_num_tokens += estimate_total_num_tokens_in_single_segment(reader, field)?;
@@ -78,7 +76,7 @@ fn estimate_total_num_tokens(readers: &[SegmentReader], field: Field) -> crate::
 
 pub struct IndexMerger {
     schema: Schema,
-    pub(crate) readers: Vec<SegmentReader>,
+    pub(crate) readers: Vec<TantivySegmentReader>,
     max_doc: u32,
 }
 
@@ -170,8 +168,10 @@ impl IndexMerger {
         let mut readers = vec![];
         for (segment, new_alive_bitset_opt) in segments.iter().zip(alive_bitset_opt) {
             if segment.meta().num_docs() > 0 {
-                let reader =
-                    SegmentReader::open_with_custom_alive_set(segment, new_alive_bitset_opt)?;
+                let reader = TantivySegmentReader::open_with_custom_alive_set(
+                    segment,
+                    new_alive_bitset_opt,
+                )?;
                 readers.push(reader);
             }
         }
@@ -204,8 +204,20 @@ impl IndexMerger {
             let fieldnorms_readers: Vec<FieldNormReader> = self
                 .readers
                 .iter()
-                .map(|reader| reader.get_fieldnorms_reader(field))
-                .collect::<Result<_, _>>()?;
+                .map(|reader| {
+                    reader
+                        .fieldnorms_readers()
+                        .get_field(field)?
+                        .ok_or_else(|| {
+                            let field_name = self.schema.get_field_name(field);
+                            let err_msg = format!(
+                                "Field norm not found for field {field_name:?}. Was the field set \
+                                 to record norm during indexing?"
+                            );
+                            crate::TantivyError::SchemaError(err_msg)
+                        })
+                })
+                .collect::<crate::Result<_>>()?;
             for old_doc_addr in doc_id_mapping.iter_old_doc_addrs() {
                 let fieldnorms_reader = &fieldnorms_readers[old_doc_addr.segment_ord as usize];
                 let fieldnorm_id = fieldnorms_reader.fieldnorm_id(old_doc_addr.doc_id);
@@ -262,7 +274,7 @@ impl IndexMerger {
                 }),
         );
 
-        let has_deletes: bool = self.readers.iter().any(SegmentReader::has_deletes);
+        let has_deletes: bool = self.readers.iter().any(|reader| reader.has_deletes());
         let mapping_type = if has_deletes {
             MappingType::StackedWithDeletes
         } else {
@@ -297,7 +309,7 @@ impl IndexMerger {
 
         let mut max_term_ords: Vec<TermOrdinal> = Vec::new();
 
-        let field_readers: Vec<Arc<InvertedIndexReader>> = self
+        let field_readers: Vec<ArcInvertedIndexReader> = self
             .readers
             .iter()
             .map(|reader| reader.inverted_index(indexed_field))
@@ -366,7 +378,7 @@ impl IndexMerger {
             // Let's compute the list of non-empty posting lists
             for (segment_ord, term_info) in merged_terms.current_segment_ords_and_term_infos() {
                 let segment_reader = &self.readers[segment_ord];
-                let inverted_index: &InvertedIndexReader = &field_readers[segment_ord];
+                let inverted_index = field_readers[segment_ord].as_ref();
                 let segment_postings = inverted_index
                     .read_postings_from_terminfo(&term_info, segment_postings_option)?;
                 let alive_bitset_opt = segment_reader.alive_bitset();
@@ -1534,7 +1546,7 @@ mod tests {
         for segment_reader in searcher.segment_readers() {
             let mut term_scorer = term_query
                 .specialized_weight(EnableScoring::enabled_from_searcher(&searcher))?
-                .term_scorer_for_test(segment_reader, 1.0)?
+                .term_scorer_for_test(segment_reader.as_ref(), 1.0)?
                 .unwrap();
             // the difference compared to before is intrinsic to the bm25 formula. no worries
             // there.

src/indexer/segment_updater.rs

@@ -710,7 +710,7 @@ mod tests {
     use crate::indexer::segment_updater::merge_filtered_segments;
     use crate::query::QueryParser;
     use crate::schema::*;
-    use crate::{Directory, DocAddress, Index, Segment};
+    use crate::{Directory, DocAddress, Index, Segment, SegmentReader};
 
     #[test]
     fn test_delete_during_merge() -> crate::Result<()> {

src/indexer/segment_writer.rs

@@ -871,7 +871,7 @@ mod tests {
         let searcher = reader.searcher();
         let segment_reader = searcher.segment_reader(0u32);
 
-        fn assert_type(reader: &SegmentReader, field: &str, typ: ColumnType) {
+        fn assert_type(reader: &dyn SegmentReader, field: &str, typ: ColumnType) {
             let cols = reader.fast_fields().dynamic_column_handles(field).unwrap();
             assert_eq!(cols.len(), 1, "{field}");
             assert_eq!(cols[0].column_type(), typ, "{field}");
@@ -890,7 +890,7 @@ mod tests {
         assert_type(segment_reader, "json.my_arr", ColumnType::I64);
         assert_type(segment_reader, "json.my_arr.my_key", ColumnType::Str);
 
-        fn assert_empty(reader: &SegmentReader, field: &str) {
+        fn assert_empty(reader: &dyn SegmentReader, field: &str) {
             let cols = reader.fast_fields().dynamic_column_handles(field).unwrap();
             assert_eq!(cols.len(), 0);
         }

src/lib.rs

@@ -224,8 +224,9 @@ pub use self::docset::{DocSet, COLLECT_BLOCK_BUFFER_LEN, TERMINATED};
 pub use crate::core::{json_utils, Executor, Searcher, SearcherGeneration};
 pub use crate::directory::Directory;
 pub use crate::index::{
-    Index, IndexBuilder, IndexMeta, IndexSettings, InvertedIndexReader, Order, Segment,
-    SegmentMeta, SegmentReader,
+    ArcInvertedIndexReader, ArcSegmentReader, Index, IndexBuilder, IndexMeta, IndexSettings,
+    InvertedIndexReader, Order, Segment, SegmentMeta, SegmentReader, TantivyInvertedIndexReader,
+    TantivySegmentReader,
 };
 pub use crate::indexer::{IndexWriter, SingleSegmentIndexWriter};
 pub use crate::schema::{Document, TantivyDocument, Term};
@@ -523,11 +524,11 @@ pub mod tests {
         let searcher = index_reader.searcher();
         let reader = searcher.segment_reader(0);
         {
-            let fieldnorm_reader = reader.get_fieldnorms_reader(text_field)?;
+            let fieldnorm_reader = reader.fieldnorms_readers().get_field(text_field)?.unwrap();
             assert_eq!(fieldnorm_reader.fieldnorm(0), 3);
         }
         {
-            let fieldnorm_reader = reader.get_fieldnorms_reader(title_field)?;
+            let fieldnorm_reader = reader.fieldnorms_readers().get_field(title_field)?.unwrap();
             assert_eq!(fieldnorm_reader.fieldnorm_id(0), 0);
         }
         Ok(())
@@ -545,15 +546,18 @@ pub mod tests {
         index_writer.commit()?;
         let reader = index.reader()?;
         let searcher = reader.searcher();
-        let segment_reader: &SegmentReader = searcher.segment_reader(0);
-        let fieldnorms_reader = segment_reader.get_fieldnorms_reader(text_field)?;
+        let segment_reader: &dyn SegmentReader = searcher.segment_reader(0);
+        let fieldnorms_reader = segment_reader
+            .fieldnorms_readers()
+            .get_field(text_field)?
+            .unwrap();
         assert_eq!(fieldnorms_reader.fieldnorm(0), 3);
         assert_eq!(fieldnorms_reader.fieldnorm(1), 0);
         assert_eq!(fieldnorms_reader.fieldnorm(2), 2);
         Ok(())
     }
 
-    fn advance_undeleted(docset: &mut dyn DocSet, reader: &SegmentReader) -> bool {
+    fn advance_undeleted(docset: &mut dyn DocSet, reader: &dyn SegmentReader) -> bool {
         let mut doc = docset.advance();
         while doc != TERMINATED {
             if !reader.is_deleted(doc) {
@@ -1070,7 +1074,7 @@ pub mod tests {
         }
         let reader = index.reader()?;
         let searcher = reader.searcher();
-        let segment_reader: &SegmentReader = searcher.segment_reader(0);
+        let segment_reader: &dyn SegmentReader = searcher.segment_reader(0);
         {
             let fast_field_reader_res = segment_reader.fast_fields().u64("text");
             assert!(fast_field_reader_res.is_err());

src/postings/block_segment_postings.rs

@@ -182,32 +182,6 @@ impl BlockSegmentPostings {
         self.freq_reading_option
     }
 
-    // Resets the block segment postings on another position
-    // in the postings file.
-    //
-    // This is useful for enumerating through a list of terms,
-    // and consuming the associated posting lists while avoiding
-    // reallocating a `BlockSegmentPostings`.
-    //
-    // # Warning
-    //
-    // This does not reset the positions list.
-    pub(crate) fn reset(&mut self, doc_freq: u32, postings_data: OwnedBytes) -> io::Result<()> {
-        let (skip_data_opt, postings_data) =
-            split_into_skips_and_postings(doc_freq, postings_data)?;
-        self.data = postings_data;
-        self.block_max_score_cache = None;
-        self.block_loaded = false;
-        if let Some(skip_data) = skip_data_opt {
-            self.skip_reader.reset(skip_data, doc_freq);
-        } else {
-            self.skip_reader.reset(OwnedBytes::empty(), doc_freq);
-        }
-        self.doc_freq = doc_freq;
-        self.load_block();
-        Ok(())
-    }
-
     /// Returns the overall number of documents in the block postings.
     /// It does not take in account whether documents are deleted or not.
     ///
@@ -303,10 +277,10 @@ impl BlockSegmentPostings {
     }
 
     pub(crate) fn load_block(&mut self) {
+        let offset = self.skip_reader.byte_offset();
         if self.block_is_loaded() {
             return;
         }
-        let offset = self.skip_reader.byte_offset();
         match self.skip_reader.block_info() {
             BlockInfo::BitPacked {
                 doc_num_bits,
@@ -521,40 +495,4 @@ mod tests {
         assert_eq!(block_postings.doc(COMPRESSION_BLOCK_SIZE - 1), TERMINATED);
         Ok(())
     }
-
-    #[test]
-    fn test_reset_block_segment_postings() -> crate::Result<()> {
-        let mut schema_builder = Schema::builder();
-        let int_field = schema_builder.add_u64_field("id", INDEXED);
-        let schema = schema_builder.build();
-        let index = Index::create_in_ram(schema);
-        let mut index_writer = index.writer_for_tests()?;
-        // create two postings list, one containing even number,
-        // the other containing odd numbers.
-        for i in 0..6 {
-            let doc = doc!(int_field=> (i % 2) as u64);
-            index_writer.add_document(doc)?;
-        }
-        index_writer.commit()?;
-        let searcher = index.reader()?.searcher();
-        let segment_reader = searcher.segment_reader(0);
-
-        let mut block_segments;
-        {
-            let term = Term::from_field_u64(int_field, 0u64);
-            let inverted_index = segment_reader.inverted_index(int_field)?;
-            let term_info = inverted_index.get_term_info(&term)?.unwrap();
-            block_segments = inverted_index
-                .read_block_postings_from_terminfo(&term_info, IndexRecordOption::Basic)?;
-        }
-        assert_eq!(block_segments.docs(), &[0, 2, 4]);
-        {
-            let term = Term::from_field_u64(int_field, 1u64);
-            let inverted_index = segment_reader.inverted_index(int_field)?;
-            let term_info = inverted_index.get_term_info(&term)?.unwrap();
-            inverted_index.reset_block_postings_from_terminfo(&term_info, &mut block_segments)?;
-        }
-        assert_eq!(block_segments.docs(), &[1, 3, 5]);
-        Ok(())
-    }
 }

src/postings/mod.rs

@@ -46,7 +46,7 @@ pub(crate) mod tests {
     use super::{InvertedIndexSerializer, Postings};
     use crate::docset::{DocSet, TERMINATED};
     use crate::fieldnorm::FieldNormReader;
-    use crate::index::{Index, SegmentComponent, SegmentReader};
+    use crate::index::{Index, SegmentComponent, SegmentReader, TantivySegmentReader};
     use crate::indexer::operation::AddOperation;
     use crate::indexer::SegmentWriter;
     use crate::query::Scorer;
@@ -258,9 +258,12 @@ pub(crate) mod tests {
             segment_writer.finalize()?;
         }
         {
-            let segment_reader = SegmentReader::open(&segment)?;
+            let segment_reader = TantivySegmentReader::open(&segment)?;
             {
-                let fieldnorm_reader = segment_reader.get_fieldnorms_reader(text_field)?;
+                let fieldnorm_reader = segment_reader
+                    .fieldnorms_readers()
+                    .get_field(text_field)?
+                    .unwrap();
                 assert_eq!(fieldnorm_reader.fieldnorm(0), 8 + 5);
                 assert_eq!(fieldnorm_reader.fieldnorm(1), 2);
                 for i in 2..1000 {

src/postings/segment_postings.rs

@@ -168,20 +168,12 @@ impl DocSet for SegmentPostings {
         self.doc()
     }
 
-    #[inline]
     fn seek(&mut self, target: DocId) -> DocId {
         debug_assert!(self.doc() <= target);
         if self.doc() >= target {
             return self.doc();
         }
 
-        // As an optimization, if the block is already loaded, we can
-        // cheaply check the next doc.
-        self.cur = (self.cur + 1).min(COMPRESSION_BLOCK_SIZE - 1);
-        if self.doc() >= target {
-            return self.doc();
-        }
-
         // Delegate block-local search to BlockSegmentPostings::seek, which returns
         // the in-block index of the first doc >= target.
         self.cur = self.block_cursor.seek(target);

src/postings/skip.rs

@@ -142,23 +142,6 @@ impl SkipReader {
         skip_reader
     }
 
-    pub fn reset(&mut self, data: OwnedBytes, doc_freq: u32) {
-        self.last_doc_in_block = if doc_freq >= COMPRESSION_BLOCK_SIZE as u32 {
-            0
-        } else {
-            TERMINATED
-        };
-        self.last_doc_in_previous_block = 0u32;
-        self.owned_read = data;
-        self.block_info = BlockInfo::VInt { num_docs: doc_freq };
-        self.byte_offset = 0;
-        self.remaining_docs = doc_freq;
-        self.position_offset = 0u64;
-        if doc_freq >= COMPRESSION_BLOCK_SIZE as u32 {
-            self.read_block_info();
-        }
-    }
-
     // Returns the block max score for this block if available.
     //
     // The block max score is available for all full bitpacked block,

src/query/all_query.rs

@@ -21,7 +21,7 @@ impl Query for AllQuery {
 pub struct AllWeight;
 
 impl Weight for AllWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let all_scorer = AllScorer::new(reader.max_doc());
         if boost != 1.0 {
             Ok(Box::new(BoostScorer::new(all_scorer, boost)))
@@ -30,7 +30,7 @@ impl Weight for AllWeight {
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         if doc >= reader.max_doc() {
             return Err(does_not_match(doc));
         }

src/query/automaton_weight.rs

@@ -67,7 +67,7 @@ where
     }
 
     /// Returns the term infos that match the automaton
-    pub fn get_match_term_infos(&self, reader: &SegmentReader) -> crate::Result<Vec<TermInfo>> {
+    pub fn get_match_term_infos(&self, reader: &dyn SegmentReader) -> crate::Result<Vec<TermInfo>> {
         let inverted_index = reader.inverted_index(self.field)?;
         let term_dict = inverted_index.terms();
         let mut term_stream = self.automaton_stream(term_dict)?;
@@ -84,7 +84,7 @@ where
     A: Automaton + Send + Sync + 'static,
     A::State: Clone,
 {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let max_doc = reader.max_doc();
         let mut doc_bitset = BitSet::with_max_value(max_doc);
         let inverted_index = reader.inverted_index(self.field)?;
@@ -110,7 +110,7 @@ where
         Ok(Box::new(const_scorer))
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) == doc {
             Ok(Explanation::new("AutomatonScorer", 1.0))

src/query/boolean_query/boolean_weight.rs

@@ -205,7 +205,7 @@ impl<TScoreCombiner: ScoreCombiner> BooleanWeight<TScoreCombiner> {
 
     fn per_occur_scorers(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<HashMap<Occur, Vec<Box<dyn Scorer>>>> {
         let mut per_occur_scorers: HashMap<Occur, Vec<Box<dyn Scorer>>> = HashMap::new();
@@ -221,7 +221,7 @@ impl<TScoreCombiner: ScoreCombiner> BooleanWeight<TScoreCombiner> {
 
     fn complex_scorer<TComplexScoreCombiner: ScoreCombiner>(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
         score_combiner_fn: impl Fn() -> TComplexScoreCombiner,
     ) -> crate::Result<SpecializedScorer> {
@@ -291,6 +291,18 @@ impl<TScoreCombiner: ScoreCombiner> BooleanWeight<TScoreCombiner> {
             }
         };
 
+        let exclude_scorer_opt: Option<Box<dyn Scorer>> = if exclude_scorers.is_empty() {
+            None
+        } else {
+            let exclude_specialized_scorer: SpecializedScorer =
+                scorer_union(exclude_scorers, DoNothingCombiner::default, num_docs);
+            Some(into_box_scorer(
+                exclude_specialized_scorer,
+                DoNothingCombiner::default,
+                num_docs,
+            ))
+        };
+
         let include_scorer = match (should_scorers, must_scorers) {
             (ShouldScorersCombinationMethod::Ignored, must_scorers) => {
                 // No SHOULD clauses (or they were absorbed into MUST).
@@ -368,23 +380,16 @@ impl<TScoreCombiner: ScoreCombiner> BooleanWeight<TScoreCombiner> {
                 }
             }
         };
-        if exclude_scorers.is_empty() {
-            return Ok(include_scorer);
-        }
-
-        let include_scorer_boxed = into_box_scorer(include_scorer, &score_combiner_fn, num_docs);
-        let scorer: Box<dyn Scorer> = if exclude_scorers.len() == 1 {
-            let exclude_scorer = exclude_scorers.pop().unwrap();
-            match exclude_scorer.downcast::<TermScorer>() {
-                // Cast to TermScorer succeeded
-                Ok(exclude_scorer) => Box::new(Exclude::new(include_scorer_boxed, *exclude_scorer)),
-                // We get back the original Box<dyn Scorer>
-                Err(exclude_scorer) => Box::new(Exclude::new(include_scorer_boxed, exclude_scorer)),
-            }
+        if let Some(exclude_scorer) = exclude_scorer_opt {
+            let include_scorer_boxed =
+                into_box_scorer(include_scorer, &score_combiner_fn, num_docs);
+            Ok(SpecializedScorer::Other(Box::new(Exclude::new(
+                include_scorer_boxed,
+                exclude_scorer,
+            ))))
         } else {
-            Box::new(Exclude::new(include_scorer_boxed, exclude_scorers))
-        };
-        Ok(SpecializedScorer::Other(scorer))
+            Ok(include_scorer)
+        }
     }
 }
 
@@ -413,7 +418,7 @@ fn remove_and_count_all_and_empty_scorers(
 }
 
 impl<TScoreCombiner: ScoreCombiner + Sync> Weight for BooleanWeight<TScoreCombiner> {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let num_docs = reader.num_docs();
         if self.weights.is_empty() {
             Ok(Box::new(EmptyScorer))
@@ -437,7 +442,7 @@ impl<TScoreCombiner: ScoreCombiner + Sync> Weight for BooleanWeight<TScoreCombin
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) != doc {
             return Err(does_not_match(doc));
@@ -459,7 +464,7 @@ impl<TScoreCombiner: ScoreCombiner + Sync> Weight for BooleanWeight<TScoreCombin
 
     fn for_each(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score),
     ) -> crate::Result<()> {
         let scorer = self.complex_scorer(reader, 1.0, &self.score_combiner_fn)?;
@@ -481,7 +486,7 @@ impl<TScoreCombiner: ScoreCombiner + Sync> Weight for BooleanWeight<TScoreCombin
 
     fn for_each_no_score(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(&[DocId]),
     ) -> crate::Result<()> {
         let scorer = self.complex_scorer(reader, 1.0, || DoNothingCombiner)?;
@@ -516,7 +521,7 @@ impl<TScoreCombiner: ScoreCombiner + Sync> Weight for BooleanWeight<TScoreCombin
     fn for_each_pruning(
         &self,
         threshold: Score,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score) -> Score,
     ) -> crate::Result<()> {
         let scorer = self.complex_scorer(reader, 1.0, &self.score_combiner_fn)?;

src/query/boost_query.rs

@@ -67,11 +67,11 @@ impl BoostWeight {
 }
 
 impl Weight for BoostWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         self.weight.scorer(reader, boost * self.boost)
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: u32) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: u32) -> crate::Result<Explanation> {
         let underlying_explanation = self.weight.explain(reader, doc)?;
         let score = underlying_explanation.value() * self.boost;
         let mut explanation =
@@ -80,7 +80,7 @@ impl Weight for BoostWeight {
         Ok(explanation)
     }
 
-    fn count(&self, reader: &SegmentReader) -> crate::Result<u32> {
+    fn count(&self, reader: &dyn SegmentReader) -> crate::Result<u32> {
         self.weight.count(reader)
     }
 }

src/query/const_score_query.rs

@@ -63,12 +63,12 @@ impl ConstWeight {
 }
 
 impl Weight for ConstWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let inner_scorer = self.weight.scorer(reader, boost)?;
         Ok(Box::new(ConstScorer::new(inner_scorer, boost * self.score)))
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: u32) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: u32) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) != doc {
             return Err(TantivyError::InvalidArgument(format!(
@@ -81,7 +81,7 @@ impl Weight for ConstWeight {
         Ok(explanation)
     }
 
-    fn count(&self, reader: &SegmentReader) -> crate::Result<u32> {
+    fn count(&self, reader: &dyn SegmentReader) -> crate::Result<u32> {
         self.weight.count(reader)
     }
 }

src/query/empty_query.rs

@@ -26,11 +26,11 @@ impl Query for EmptyQuery {
 /// It is useful for tests and handling edge cases.
 pub struct EmptyWeight;
 impl Weight for EmptyWeight {
-    fn scorer(&self, _reader: &SegmentReader, _boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, _reader: &dyn SegmentReader, _boost: Score) -> crate::Result<Box<dyn Scorer>> {
         Ok(Box::new(EmptyScorer))
     }
 
-    fn explain(&self, _reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, _reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         Err(does_not_match(doc))
     }
 }

src/query/exclude.rs

@@ -1,71 +1,48 @@
-use crate::docset::{DocSet, SeekDangerResult, TERMINATED};
+use crate::docset::{DocSet, TERMINATED};
 use crate::query::Scorer;
 use crate::{DocId, Score};
 
-/// An exclusion set is a set of documents
-/// that should be excluded from a given DocSet.
+#[inline]
+fn is_within<TDocSetExclude: DocSet>(docset: &mut TDocSetExclude, doc: DocId) -> bool {
+    docset.doc() <= doc && docset.seek(doc) == doc
+}
+
+/// Filters a given `DocSet` by removing the docs from a given `DocSet`.
 ///
-/// It can be a single DocSet, or a Vec of DocSets.
-pub trait ExclusionSet: Send {
-    /// Returns `true` if the given `doc` is in the exclusion set.
-    fn contains(&mut self, doc: DocId) -> bool;
-}
-
-impl<TDocSet: DocSet> ExclusionSet for TDocSet {
-    #[inline]
-    fn contains(&mut self, doc: DocId) -> bool {
-        self.seek_danger(doc) == SeekDangerResult::Found
-    }
-}
-
-impl<TDocSet: DocSet> ExclusionSet for Vec<TDocSet> {
-    #[inline]
-    fn contains(&mut self, doc: DocId) -> bool {
-        for docset in self.iter_mut() {
-            if docset.seek_danger(doc) == SeekDangerResult::Found {
-                return true;
-            }
-        }
-        false
-    }
-}
-
-/// Filters a given `DocSet` by removing the docs from an exclusion set.
-///
-/// The excluding docsets have no impact on scoring.
-pub struct Exclude<TDocSet, TExclusionSet> {
+/// The excluding docset has no impact on scoring.
+pub struct Exclude<TDocSet, TDocSetExclude> {
     underlying_docset: TDocSet,
-    exclusion_set: TExclusionSet,
+    excluding_docset: TDocSetExclude,
 }
 
-impl<TDocSet, TExclusionSet> Exclude<TDocSet, TExclusionSet>
+impl<TDocSet, TDocSetExclude> Exclude<TDocSet, TDocSetExclude>
 where
     TDocSet: DocSet,
-    TExclusionSet: ExclusionSet,
+    TDocSetExclude: DocSet,
 {
     /// Creates a new `ExcludeScorer`
     pub fn new(
         mut underlying_docset: TDocSet,
-        mut exclusion_set: TExclusionSet,
-    ) -> Exclude<TDocSet, TExclusionSet> {
+        mut excluding_docset: TDocSetExclude,
+    ) -> Exclude<TDocSet, TDocSetExclude> {
         while underlying_docset.doc() != TERMINATED {
             let target = underlying_docset.doc();
-            if !exclusion_set.contains(target) {
+            if !is_within(&mut excluding_docset, target) {
                 break;
             }
             underlying_docset.advance();
         }
         Exclude {
             underlying_docset,
-            exclusion_set,
+            excluding_docset,
         }
     }
 }
 
-impl<TDocSet, TExclusionSet> DocSet for Exclude<TDocSet, TExclusionSet>
+impl<TDocSet, TDocSetExclude> DocSet for Exclude<TDocSet, TDocSetExclude>
 where
     TDocSet: DocSet,
-    TExclusionSet: ExclusionSet,
+    TDocSetExclude: DocSet,
 {
     fn advance(&mut self) -> DocId {
         loop {
@@ -73,7 +50,7 @@ where
             if candidate == TERMINATED {
                 return TERMINATED;
             }
-            if !self.exclusion_set.contains(candidate) {
+            if !is_within(&mut self.excluding_docset, candidate) {
                 return candidate;
             }
         }
@@ -84,7 +61,7 @@ where
         if candidate == TERMINATED {
             return TERMINATED;
         }
-        if !self.exclusion_set.contains(candidate) {
+        if !is_within(&mut self.excluding_docset, candidate) {
             return candidate;
         }
         self.advance()
@@ -102,10 +79,10 @@ where
     }
 }
 
-impl<TScorer, TExclusionSet> Scorer for Exclude<TScorer, TExclusionSet>
+impl<TScorer, TDocSetExclude> Scorer for Exclude<TScorer, TDocSetExclude>
 where
     TScorer: Scorer,
-    TExclusionSet: ExclusionSet + 'static,
+    TDocSetExclude: DocSet + 'static,
 {
     #[inline]
     fn score(&mut self) -> Score {

src/query/exist_query.rs

@@ -98,7 +98,7 @@ pub struct ExistsWeight {
 }
 
 impl Weight for ExistsWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let fast_field_reader = reader.fast_fields();
         let mut column_handles = fast_field_reader.dynamic_column_handles(&self.field_name)?;
         if self.field_type == Type::Json && self.json_subpaths {
@@ -165,7 +165,7 @@ impl Weight for ExistsWeight {
         Ok(Box::new(ConstScorer::new(docset, boost)))
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) != doc {
             return Err(does_not_match(doc));

src/query/intersection.rs

@@ -84,14 +84,6 @@ impl<TDocSet: DocSet> Intersection<TDocSet, TDocSet> {
         docsets.sort_by_key(|docset| docset.cost());
         go_to_first_doc(&mut docsets);
         let left = docsets.remove(0);
-        debug_assert!({
-            let doc = left.doc();
-            if doc == TERMINATED {
-                true
-            } else {
-                docsets.iter().all(|docset| docset.doc() == doc)
-            }
-        });
         let right = docsets.remove(0);
         Intersection {
             left,
@@ -120,24 +112,30 @@ impl<TDocSet: DocSet, TOtherDocSet: DocSet> DocSet for Intersection<TDocSet, TOt
         // Invariant:
         // - candidate is always <= to the next document in the intersection.
         // - candidate strictly increases at every occurence of the loop.
-        let mut candidate = left.doc() + 1;
+        let mut candidate = 0;
 
         // Termination: candidate strictly increases.
         'outer: while candidate < TERMINATED {
             // As we enter the loop, we should always have candidate < next_doc.
 
-            candidate = left.seek(candidate);
+            // This step always increases candidate.
+            //
+            // TODO: Think about which value would make sense here
+            // It depends on the DocSet implementation, when a seek would outweigh an advance.
+            candidate = if candidate > left.doc().wrapping_add(100) {
+                left.seek(candidate)
+            } else {
+                left.advance()
+            };
 
             // Left is positionned on `candidate`.
             debug_assert_eq!(left.doc(), candidate);
 
             if let SeekDangerResult::SeekLowerBound(seek_lower_bound) = right.seek_danger(candidate)
             {
-                debug_assert!(
-                    seek_lower_bound == TERMINATED || seek_lower_bound > candidate,
-                    "seek_lower_bound {seek_lower_bound} must be greater than candidate \
-                     {candidate}"
-                );
+                // The max is technically useless but it makes the invariant
+                // easier to proofread.
+                debug_assert!(seek_lower_bound >= candidate);
                 candidate = seek_lower_bound;
                 continue;
             }
@@ -150,11 +148,7 @@ impl<TDocSet: DocSet, TOtherDocSet: DocSet> DocSet for Intersection<TDocSet, TOt
                     other.seek_danger(candidate)
                 {
                     // One of the scorer does not match, let's restart at the top of the loop.
-                    debug_assert!(
-                        seek_lower_bound == TERMINATED || seek_lower_bound > candidate,
-                        "seek_lower_bound {seek_lower_bound} must be greater than candidate \
-                         {candidate}"
-                    );
+                    debug_assert!(seek_lower_bound >= candidate);
                     candidate = seek_lower_bound;
                     continue 'outer;
                 }
@@ -244,12 +238,9 @@ mod tests {
     use proptest::prelude::*;
 
     use super::Intersection;
-    use crate::collector::Count;
     use crate::docset::{DocSet, TERMINATED};
     use crate::postings::tests::test_skip_against_unoptimized;
-    use crate::query::{QueryParser, VecDocSet};
-    use crate::schema::{Schema, TEXT};
-    use crate::Index;
+    use crate::query::VecDocSet;
 
     #[test]
     fn test_intersection() {
@@ -420,29 +411,4 @@ mod tests {
             assert_eq!(intersection.doc(), TERMINATED);
         }
     }
-
-    #[test]
-    fn test_bug_2811_intersection_candidate_should_increase() {
-        let mut schema_builder = Schema::builder();
-        let text_field = schema_builder.add_text_field("text", TEXT);
-        let schema = schema_builder.build();
-
-        let index = Index::create_in_ram(schema);
-        let mut writer = index.writer_for_tests().unwrap();
-        writer
-            .add_document(doc!(text_field=>"hello happy tax"))
-            .unwrap();
-        writer.add_document(doc!(text_field=>"hello")).unwrap();
-        writer.add_document(doc!(text_field=>"hello")).unwrap();
-        writer.add_document(doc!(text_field=>"happy tax")).unwrap();
-
-        writer.commit().unwrap();
-        let query_parser = QueryParser::for_index(&index, Vec::new());
-        let query = query_parser
-            .parse_query(r#"+text:hello +text:"happy tax""#)
-            .unwrap();
-        let searcher = index.reader().unwrap().searcher();
-        let c = searcher.search(&*query, &Count).unwrap();
-        assert_eq!(c, 1);
-    }
 }

src/query/mod.rs

@@ -43,7 +43,7 @@ pub use self::boost_query::{BoostQuery, BoostWeight};
 pub use self::const_score_query::{ConstScoreQuery, ConstScorer};
 pub use self::disjunction_max_query::DisjunctionMaxQuery;
 pub use self::empty_query::{EmptyQuery, EmptyScorer, EmptyWeight};
-pub use self::exclude::{Exclude, ExclusionSet};
+pub use self::exclude::Exclude;
 pub use self::exist_query::ExistsQuery;
 pub use self::explanation::Explanation;
 #[cfg(test)]

src/query/phrase_prefix_query/phrase_prefix_weight.rs

@@ -32,7 +32,7 @@ impl PhrasePrefixWeight {
         }
     }
 
-    fn fieldnorm_reader(&self, reader: &SegmentReader) -> crate::Result<FieldNormReader> {
+    fn fieldnorm_reader(&self, reader: &dyn SegmentReader) -> crate::Result<FieldNormReader> {
         let field = self.phrase_terms[0].1.field();
         if self.similarity_weight_opt.is_some() {
             if let Some(fieldnorm_reader) = reader.fieldnorms_readers().get_field(field)? {
@@ -44,7 +44,7 @@ impl PhrasePrefixWeight {
 
     pub(crate) fn phrase_scorer(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<Option<PhrasePrefixScorer<SegmentPostings>>> {
         let similarity_weight_opt = self
@@ -114,7 +114,7 @@ impl PhrasePrefixWeight {
 }
 
 impl Weight for PhrasePrefixWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         if let Some(scorer) = self.phrase_scorer(reader, boost)? {
             Ok(Box::new(scorer))
         } else {
@@ -122,7 +122,7 @@ impl Weight for PhrasePrefixWeight {
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let scorer_opt = self.phrase_scorer(reader, 1.0)?;
         if scorer_opt.is_none() {
             return Err(does_not_match(doc));

src/query/phrase_query/phrase_scorer.rs

@@ -531,12 +531,7 @@ impl<TPostings: Postings> DocSet for PhraseScorer<TPostings> {
     }
 
     fn seek_danger(&mut self, target: DocId) -> SeekDangerResult {
-        debug_assert!(
-            target >= self.doc(),
-            "target ({}) should be greater than or equal to doc ({})",
-            target,
-            self.doc()
-        );
+        debug_assert!(target >= self.doc());
         let seek_res = self.intersection_docset.seek_danger(target);
         if seek_res != SeekDangerResult::Found {
             return seek_res;

src/query/phrase_query/phrase_weight.rs

@@ -29,7 +29,7 @@ impl PhraseWeight {
         }
     }
 
-    fn fieldnorm_reader(&self, reader: &SegmentReader) -> crate::Result<FieldNormReader> {
+    fn fieldnorm_reader(&self, reader: &dyn SegmentReader) -> crate::Result<FieldNormReader> {
         let field = self.phrase_terms[0].1.field();
         if self.similarity_weight_opt.is_some() {
             if let Some(fieldnorm_reader) = reader.fieldnorms_readers().get_field(field)? {
@@ -41,7 +41,7 @@ impl PhraseWeight {
 
     pub(crate) fn phrase_scorer(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<Option<PhraseScorer<SegmentPostings>>> {
         let similarity_weight_opt = self
@@ -74,7 +74,7 @@ impl PhraseWeight {
 }
 
 impl Weight for PhraseWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         if let Some(scorer) = self.phrase_scorer(reader, boost)? {
             Ok(Box::new(scorer))
         } else {
@@ -82,7 +82,7 @@ impl Weight for PhraseWeight {
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let scorer_opt = self.phrase_scorer(reader, 1.0)?;
         if scorer_opt.is_none() {
             return Err(does_not_match(doc));

src/query/phrase_query/regex_phrase_weight.rs

@@ -45,7 +45,7 @@ impl RegexPhraseWeight {
         }
     }
 
-    fn fieldnorm_reader(&self, reader: &SegmentReader) -> crate::Result<FieldNormReader> {
+    fn fieldnorm_reader(&self, reader: &dyn SegmentReader) -> crate::Result<FieldNormReader> {
         if self.similarity_weight_opt.is_some() {
             if let Some(fieldnorm_reader) = reader.fieldnorms_readers().get_field(self.field)? {
                 return Ok(fieldnorm_reader);
@@ -56,7 +56,7 @@ impl RegexPhraseWeight {
 
     pub(crate) fn phrase_scorer(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<Option<PhraseScorer<UnionType>>> {
         let similarity_weight_opt = self
@@ -84,7 +84,8 @@ impl RegexPhraseWeight {
                     "Phrase query exceeded max expansions {num_terms}"
                 )));
             }
-            let union = Self::get_union_from_term_infos(&term_infos, reader, &inverted_index)?;
+            let union =
+                Self::get_union_from_term_infos(&term_infos, reader, inverted_index.as_ref())?;
 
             posting_lists.push((offset, union));
         }
@@ -99,7 +100,7 @@ impl RegexPhraseWeight {
 
     /// Add all docs of the term to the docset
     fn add_to_bitset(
-        inverted_index: &InvertedIndexReader,
+        inverted_index: &dyn InvertedIndexReader,
         term_info: &TermInfo,
         doc_bitset: &mut BitSet,
     ) -> crate::Result<()> {
@@ -174,8 +175,8 @@ impl RegexPhraseWeight {
     /// Use Roaring Bitmaps for sparse terms. The full bitvec is main memory consumer currently.
     pub(crate) fn get_union_from_term_infos(
         term_infos: &[TermInfo],
-        reader: &SegmentReader,
-        inverted_index: &InvertedIndexReader,
+        reader: &dyn SegmentReader,
+        inverted_index: &dyn InvertedIndexReader,
     ) -> crate::Result<UnionType> {
         let max_doc = reader.max_doc();
 
@@ -269,7 +270,7 @@ impl RegexPhraseWeight {
 }
 
 impl Weight for RegexPhraseWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         if let Some(scorer) = self.phrase_scorer(reader, boost)? {
             Ok(Box::new(scorer))
         } else {
@@ -277,7 +278,7 @@ impl Weight for RegexPhraseWeight {
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let scorer_opt = self.phrase_scorer(reader, 1.0)?;
         if scorer_opt.is_none() {
             return Err(does_not_match(doc));

src/query/query.rs

@@ -146,7 +146,7 @@ pub trait Query: QueryClone + Send + Sync + downcast_rs::Downcast + fmt::Debug {
         let weight = self.weight(EnableScoring::disabled_from_searcher(searcher))?;
         let mut result = 0;
         for reader in searcher.segment_readers() {
-            result += weight.count(reader)? as usize;
+            result += weight.count(reader.as_ref())? as usize;
         }
         Ok(result)
     }

src/query/range_query/range_query.rs

@@ -212,7 +212,7 @@ impl InvertedIndexRangeWeight {
 }
 
 impl Weight for InvertedIndexRangeWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         let max_doc = reader.max_doc();
         let mut doc_bitset = BitSet::with_max_value(max_doc);
 
@@ -245,7 +245,7 @@ impl Weight for InvertedIndexRangeWeight {
         Ok(Box::new(ConstScorer::new(doc_bitset, boost)))
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) != doc {
             return Err(does_not_match(doc));
@@ -686,7 +686,7 @@ mod tests {
                 .weight(EnableScoring::disabled_from_schema(&schema))
                 .unwrap();
             let range_scorer = range_weight
-                .scorer(&searcher.segment_readers()[0], 1.0f32)
+                .scorer(searcher.segment_readers()[0].as_ref(), 1.0f32)
                 .unwrap();
             range_scorer
         };

src/query/range_query/range_query_fastfield.rs

@@ -52,7 +52,7 @@ impl FastFieldRangeWeight {
 }
 
 impl Weight for FastFieldRangeWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         // Check if both bounds are Bound::Unbounded
         if self.bounds.is_unbounded() {
             return Ok(Box::new(AllScorer::new(reader.max_doc())));
@@ -219,7 +219,7 @@ impl Weight for FastFieldRangeWeight {
         }
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if scorer.seek(doc) != doc {
             return Err(TantivyError::InvalidArgument(format!(
@@ -236,7 +236,7 @@ impl Weight for FastFieldRangeWeight {
 ///
 /// Convert into fast field value space and search.
 fn search_on_json_numerical_field(
-    reader: &SegmentReader,
+    reader: &dyn SegmentReader,
     field_name: &str,
     typ: Type,
     bounds: BoundsRange<ValueBytes<Vec<u8>>>,

src/query/term_query/term_scorer.rs

@@ -105,7 +105,6 @@ impl DocSet for TermScorer {
 
     #[inline]
     fn seek(&mut self, target: DocId) -> DocId {
-        debug_assert!(target >= self.doc());
         self.postings.seek(target)
     }
 
@@ -264,7 +263,9 @@ mod tests {
             let mut block_max_scores_b = vec![];
             let mut docs = vec![];
             {
-                let mut term_scorer = term_weight.term_scorer_for_test(reader, 1.0)?.unwrap();
+                let mut term_scorer = term_weight
+                    .term_scorer_for_test(reader.as_ref(), 1.0)?
+                    .unwrap();
                 while term_scorer.doc() != TERMINATED {
                     let mut score = term_scorer.score();
                     docs.push(term_scorer.doc());
@@ -278,7 +279,9 @@ mod tests {
                 }
             }
             {
-                let mut term_scorer = term_weight.term_scorer_for_test(reader, 1.0)?.unwrap();
+                let mut term_scorer = term_weight
+                    .term_scorer_for_test(reader.as_ref(), 1.0)?
+                    .unwrap();
                 for d in docs {
                     term_scorer.seek_block(d);
                     block_max_scores_b.push(term_scorer.block_max_score());

src/query/term_query/term_weight.rs

@@ -34,11 +34,11 @@ impl TermOrEmptyOrAllScorer {
 }
 
 impl Weight for TermWeight {
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>> {
         Ok(self.specialized_scorer(reader, boost)?.into_boxed_scorer())
     }
 
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation> {
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation> {
         match self.specialized_scorer(reader, 1.0)? {
             TermOrEmptyOrAllScorer::TermScorer(mut term_scorer) => {
                 if term_scorer.doc() > doc || term_scorer.seek(doc) != doc {
@@ -53,7 +53,7 @@ impl Weight for TermWeight {
         }
     }
 
-    fn count(&self, reader: &SegmentReader) -> crate::Result<u32> {
+    fn count(&self, reader: &dyn SegmentReader) -> crate::Result<u32> {
         if let Some(alive_bitset) = reader.alive_bitset() {
             Ok(self.scorer(reader, 1.0)?.count(alive_bitset))
         } else {
@@ -68,7 +68,7 @@ impl Weight for TermWeight {
     /// `DocSet` and push the scored documents to the collector.
     fn for_each(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score),
     ) -> crate::Result<()> {
         match self.specialized_scorer(reader, 1.0)? {
@@ -87,7 +87,7 @@ impl Weight for TermWeight {
     /// `DocSet` and push the scored documents to the collector.
     fn for_each_no_score(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(&[DocId]),
     ) -> crate::Result<()> {
         match self.specialized_scorer(reader, 1.0)? {
@@ -118,7 +118,7 @@ impl Weight for TermWeight {
     fn for_each_pruning(
         &self,
         threshold: Score,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score) -> Score,
     ) -> crate::Result<()> {
         let specialized_scorer = self.specialized_scorer(reader, 1.0)?;
@@ -166,7 +166,7 @@ impl TermWeight {
     #[cfg(test)]
     pub(crate) fn term_scorer_for_test(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<Option<TermScorer>> {
         let scorer = self.specialized_scorer(reader, boost)?;
@@ -178,7 +178,7 @@ impl TermWeight {
 
     fn specialized_scorer(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         boost: Score,
     ) -> crate::Result<TermOrEmptyOrAllScorer> {
         let field = self.term.field();
@@ -206,7 +206,10 @@ impl TermWeight {
         )))
     }
 
-    fn fieldnorm_reader(&self, segment_reader: &SegmentReader) -> crate::Result<FieldNormReader> {
+    fn fieldnorm_reader(
+        &self,
+        segment_reader: &dyn SegmentReader,
+    ) -> crate::Result<FieldNormReader> {
         if self.scoring_enabled {
             if let Some(field_norm_reader) = segment_reader
                 .fieldnorms_readers()

src/query/weight.rs

@@ -69,13 +69,13 @@ pub trait Weight: Send + Sync + 'static {
     /// `boost` is a multiplier to apply to the score.
     ///
     /// See [`Query`](crate::query::Query).
-    fn scorer(&self, reader: &SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>>;
+    fn scorer(&self, reader: &dyn SegmentReader, boost: Score) -> crate::Result<Box<dyn Scorer>>;
 
     /// Returns an [`Explanation`] for the given document.
-    fn explain(&self, reader: &SegmentReader, doc: DocId) -> crate::Result<Explanation>;
+    fn explain(&self, reader: &dyn SegmentReader, doc: DocId) -> crate::Result<Explanation>;
 
     /// Returns the number documents within the given [`SegmentReader`].
-    fn count(&self, reader: &SegmentReader) -> crate::Result<u32> {
+    fn count(&self, reader: &dyn SegmentReader) -> crate::Result<u32> {
         let mut scorer = self.scorer(reader, 1.0)?;
         if let Some(alive_bitset) = reader.alive_bitset() {
             Ok(scorer.count(alive_bitset))
@@ -88,7 +88,7 @@ pub trait Weight: Send + Sync + 'static {
     /// `DocSet` and push the scored documents to the collector.
     fn for_each(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score),
     ) -> crate::Result<()> {
         let mut scorer = self.scorer(reader, 1.0)?;
@@ -100,7 +100,7 @@ pub trait Weight: Send + Sync + 'static {
     /// `DocSet` and push the scored documents to the collector.
     fn for_each_no_score(
         &self,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(&[DocId]),
     ) -> crate::Result<()> {
         let mut docset = self.scorer(reader, 1.0)?;
@@ -123,7 +123,7 @@ pub trait Weight: Send + Sync + 'static {
     fn for_each_pruning(
         &self,
         threshold: Score,
-        reader: &SegmentReader,
+        reader: &dyn SegmentReader,
         callback: &mut dyn FnMut(DocId, Score) -> Score,
     ) -> crate::Result<()> {
         let mut scorer = self.scorer(reader, 1.0)?;

src/reader/mod.rs

@@ -10,7 +10,7 @@ use self::warming::WarmingState;
 use crate::core::searcher::{SearcherGeneration, SearcherInner};
 use crate::directory::{Directory, WatchCallback, WatchHandle, META_LOCK};
 use crate::store::DOCSTORE_CACHE_CAPACITY;
-use crate::{Index, Inventory, Searcher, SegmentReader, TrackedObject};
+use crate::{ArcSegmentReader, Index, Inventory, Searcher, TantivySegmentReader, TrackedObject};
 
 /// Defines when a new version of the index should be reloaded.
 ///
@@ -189,19 +189,22 @@ impl InnerIndexReader {
     ///
     /// This function acquires a lock to prevent GC from removing files
     /// as we are opening our index.
-    fn open_segment_readers(index: &Index) -> crate::Result<Vec<SegmentReader>> {
+    fn open_segment_readers(index: &Index) -> crate::Result<Vec<ArcSegmentReader>> {
         // Prevents segment files from getting deleted while we are in the process of opening them
         let _meta_lock = index.directory().acquire_lock(&META_LOCK)?;
         let searchable_segments = index.searchable_segments()?;
         let segment_readers = searchable_segments
             .iter()
-            .map(SegmentReader::open)
+            .map(|segment| {
+                TantivySegmentReader::open(segment)
+                    .map(|reader| Arc::new(reader) as ArcSegmentReader)
+            })
             .collect::<crate::Result<_>>()?;
         Ok(segment_readers)
     }
 
     fn track_segment_readers_in_inventory(
-        segment_readers: &[SegmentReader],
+        segment_readers: &[ArcSegmentReader],
         searcher_generation_counter: &Arc<AtomicU64>,
         searcher_generation_inventory: &Inventory<SearcherGeneration>,
     ) -> TrackedObject<SearcherGeneration> {

src/schema/field_entry.rs

@@ -210,8 +210,11 @@ mod tests {
         index_writer.add_document(doc!(text=>"abc"))?;
         index_writer.commit()?;
         let searcher = index.reader()?.searcher();
-        let err = searcher.segment_reader(0u32).get_fieldnorms_reader(text);
-        assert!(matches!(err, Err(crate::TantivyError::SchemaError(_))));
+        let fieldnorm_opt = searcher
+            .segment_reader(0u32)
+            .fieldnorms_readers()
+            .get_field(text)?;
+        assert!(fieldnorm_opt.is_none());
         Ok(())
     }
 }

src/space_usage/mod.rs

@@ -124,6 +124,7 @@ impl SegmentSpaceUsage {
             FieldNorms => PerField(self.fieldnorms().clone()),
             Terms => PerField(self.termdict().clone()),
             SegmentComponent::Store => ComponentSpaceUsage::Store(self.store().clone()),
+            SegmentComponent::TempStore => ComponentSpaceUsage::Store(self.store().clone()),
             Delete => Basic(self.deletes()),
         }
     }

src/store/mod.rs

@@ -26,7 +26,7 @@
 //! and should rely on either
 //!
 //! - at the segment level, the [`SegmentReader`'s `doc`
-//!   method](../struct.SegmentReader.html#method.doc)
+//!   method](../trait.SegmentReader.html#method.doc)
 //! - at the index level, the [`Searcher::doc()`](crate::Searcher::doc) method
 
 mod compressors;