lancedb/docs/web/search/explain-analyze.mdx

---
title: "Explain analyze a query plan"
sidebarTitle: Explain Analyze
description: "Execute a query and read the runtime metrics attached to every operator."
icon: "gauge"
---

`analyze_plan` executes the query and returns the physical plan annotated with
runtime metrics -- elapsed time, rows produced, bytes read, index comparisons.
Where [`explain_plan`](/search/explain) tells you what a query intends to do,
this tells you what it cost.

## What it measures {#analyze-a-query-plan}

Executes the query and provides detailed runtime metrics, including:

- Operation duration (`_elapsed_compute_`)
- Data processing statistics (`_output_rows_`, `_bytes_read_`)
- Index effectiveness (`_index_comparisons_`, `_indices_loaded_`)
- Resource utilization (`_iops_`, `_requests_`)

## Performance Analysis {#performance-analysis}

Let's use `analyze_plan` to run the query and analyze the query performance, which will help us identify potential bottlenecks:

<CodeGroup>
```python Python icon="python"
# analyze_plan
query_analyze_plan = (
    table.search(query_embed)
    .where("identifier > 0 AND identifier < 1000000")
    .select(["chunk_index", "title", "identifier"])
    .limit(100)
    .analyze_plan()
)
```

```typescript TypeScript icon="square-js"
// analyze_plan
const analyzePlan = await table
    .search(queryEmbed)
    .where("identifier > 0 AND identifier < 1000000")
    .select(["chunk_index", "title", "identifier"])
    .limit(100)
    .analyzePlan();
```
</CodeGroup>

### Performance Metrics Analysis {#performance-metrics-analysis}

```
ProjectionExec: expr=[chunk_index@4 as chunk_index, title@5 as title, identifier@1 as identifier, _distance@3 as _distance], metrics=[output_rows=100, elapsed_compute=1.424µs]
    RemoteTake: columns="vector, identifier, _rowid, _distance, chunk_index, title", metrics=[output_rows=100, elapsed_compute=175.53097ms, output_batches=1, remote_takes=100]
      CoalesceBatchesExec: target_batch_size=1024, metrics=[output_rows=100, elapsed_compute=2.748µs]
        GlobalLimitExec: skip=0, fetch=100, metrics=[output_rows=100, elapsed_compute=1.819µs]
          FilterExec: _distance@3 IS NOT NULL, metrics=[output_rows=100, elapsed_compute=10.275µs]
            SortExec: TopK(fetch=100), expr=[_distance@3 ASC NULLS LAST], preserve_partitioning=[false], metrics=[output_rows=100, elapsed_compute=39.259451ms, row_replacements=546]
              KNNVectorDistance: metric=l2, metrics=[output_rows=1099508, elapsed_compute=56.783526ms, output_batches=1076]
                FilterExec: identifier@1 > 0 AND identifier@1 < 1000000, metrics=[output_rows=1099508, elapsed_compute=17.136819ms]
                  LanceScan: uri=***, projection=[vector, identifier], row_id=true, row_addr=false, ordered=false, metrics=[output_rows=1200000, elapsed_compute=21.348178ms, bytes_read=1852931072, iops=78, requests=78]
```

#### 1\. Data Loading (LanceScan) {#1-data-loading-lancescan}

- Scanned 1,200,000 rows from the LanceDB table
- Read 1.86GB of data in 78 I/O operations
- Only loaded necessary columns (`vector` and `identifier`)
- Unordered scan for parallel processing

#### 2\. Filtering and Search {#2-filtering-and-search}

- Applied prefilter condition (`identifier > 0 AND identifier < 1000000`)
- Reduced dataset from 1.2M to 1,099,508 rows
- KNN search used L2 (Euclidean) distance metric
- Vector comparisons processed in 1076 batches

#### 3\. Results Processing {#3-results-processing}

- KNN results sorted by distance (TopK with fetch=100)
- Null distances filtered out
- Batches coalesced to target size of 1024 rows
- Additional columns fetched for final results
- Remote take operation for 100 results
- Final projection of required columns

### Distributed metrics on remote tables {#distributed-metrics-on-remote-tables}

<Badge color="red">Enterprise</Badge>

When you call `analyze_plan` against a LanceDB Enterprise table, the query runs across a pool of workers. By default the returned plan aggregates each operator's metrics into a single value, matching the local single-node output. Pass a `distributed_metrics` mode when you need to see how work was split across workers:

| Mode          | What it shows                                                                       | When to use it                                                                                          |
| :------------ | :---------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------ |
| `aggregate`   | One summary value per operator (default). Same shape as local `analyze_plan`.       | Everyday performance checks and comparisons against a local plan.                                       |
| `per_worker`  | Metrics reported separately for each worker that participated in the query.         | Diagnosing stragglers or uneven work distribution across the cluster.                                   |
| `full`        | Both the aggregate summary and the per-worker breakdown.                            | Deep investigations where you need the totals *and* the per-worker detail in a single output.           |

The parameter only affects remote plans — local queries always return the aggregate output regardless of what you pass.

<CodeGroup>
```python Python icon="python"
# Show a per-worker breakdown for a remote query
plan = (
    table.search(query_embed)
    .where("identifier > 0 AND identifier < 1000000")
    .limit(100)
    .analyze_plan(distributed_metrics="per_worker")
)
print(plan)
```

```typescript TypeScript icon="square-js"
// Show a per-worker breakdown for a remote query
const plan = await table
    .search(queryEmbed)
    .where("identifier > 0 AND identifier < 1000000")
    .limit(100)
    .analyzePlan("per_worker");
```
</CodeGroup>

<Note>
Use `per_worker` or `full` sparingly — the plan output grows with the size of your worker pool. Stick with the default `aggregate` mode for routine tuning.
</Note>

### Key Observations {#key-observations}

- Vector search is the primary bottleneck (1,099,508 vector comparisons)
- Significant I/O overhead (1.86GB data read)
- Full table scan due to lack of indices
- Substantial optimization potential through proper index implementation

## Optimized Query Execution {#optimized-query-execution}

After creating vector and scalar indices, the execution plan shows:

```
ProjectionExec: expr=[chunk_index@3 as chunk_index, title@4 as title, identifier@2 as identifier, _distance@0 as _distance]
  RemoteTake: columns="_distance, _rowid, identifier, chunk_index, title"
    CoalesceBatchesExec: target_batch_size=1024
      GlobalLimitExec: skip=0, fetch=100
        SortExec: TopK(fetch=100), expr=[_distance@0 ASC NULLS LAST], preserve_partitioning=[false]
          ANNSubIndex: name=vector_idx, k=100, deltas=1
            ANNIvfPartition: uuid=83916fd5-fc45-4977-bad9-1f0737539bb9, nprobes=20, deltas=1
            ScalarIndexQuery: query=AND(identifier > 0,identifier < 1000000)
```

### Optimized Plan Analysis {#optimized-plan-analysis}

#### 1\. Scalar Index Query {#1-scalar-index-query}

```
ScalarIndexQuery: query=AND(identifier > 0,identifier < 1000000)
metrics=[
    output_rows=2
    index_comparisons=2,301,824
    indices_loaded=2
    output_batches=1
    parts_loaded=562
    elapsed_compute=86.979354ms
]
```

- Range filter using scalar index
- Only 2 index files and 562 scalar index parts loaded
- 2.3M index comparisons for matches

#### 2\. Vector Search {#2-vector-search}

```
ANNSubIndex: name=vector_idx, k=100, deltas=1
metrics=[
    output_rows=2,000
    index_comparisons=25,893
    indices_loaded=0
    output_batches=20
    parts_loaded=20
    elapsed_compute=111.849043ms
]
```

- IVF index with 20 probes
- Only 20 index parts loaded
- 25,893 vector comparisons
- 2,000 matching vectors

#### 3\. Results Processing {#3-results-processing-2}

```
SortExec: TopK(fetch=100), expr=[_distance@0 ASC NULLS LAST], preserve_partitioning=[false]
GlobalLimitExec: skip=0, fetch=100
CoalesceBatchesExec: target_batch_size=1024
```

- Sorts by distance
- Limits to top 100 results
- Batches into groups of 1024

#### 4\. Data Fetching {#4-data-fetching}

```
RemoteTake: columns="_distance, _rowid, identifier, chunk_index, title"
metrics=[output_rows=100, elapsed_compute=113.491859ms, output_batches=1, remote_takes=100]
```

- Single output batch
- One remote take per row

#### 5\. Final Projection {#5-final-projection}

```
ProjectionExec: expr=[chunk_index@3 as chunk_index, title@4 as title, identifier@2 as identifier, _distance@0 as _distance]
```

- Returns specified columns: chunk_index, title, identifier, and distance

### Performance Improvements {#performance-improvements}

#### 1\. Initial Data Access {#1-initial-data-access}

```
ScalarIndexQuery metrics:
- indices_loaded=2
- parts_loaded=562
- output_batches=1
```

- Before: Full table scan of 1.2M rows, 1.86GB data
- After: Only 2 indices and 562 scalar index parts loaded
- Benefit: Eliminated table scans for prefilter

#### 2\. Vector Search Efficiency {#2-vector-search-efficiency}

```
ANNSubIndex:
- index_comparisons=25,893
- indices_loaded=0
- parts_loaded=20
- output_batches=20
```

- Before: L2 calculations on 1,099,508 vectors
- After: 
  - 99.8% reduction in vector comparisons
  - Decreased output batches from 1,076 to 20

#### 3\. Data Retrieval Optimization {#3-data-retrieval-optimization}

```
RemoteTake:
- remote_takes=100
- output_batches=1
```

- RemoteTake operation remains consistent

## Performance Optimization Guide {#performance-optimization-guide}

### 1\. Index Implementation {#1-index-implementation}

#### When to Create Indices {#when-to-create-indices}

- Columns used in WHERE clauses
- Vector columns for similarity searches
- Join columns used in `merge_insert`

#### Index Type Selection {#index-type-selection}

| Data Type   | Recommended Index  | Use Case                                 |
| ----------- | ------------------ | ---------------------------------------- |
| Vector      | IVF_PQ/IVF_HNSW_SQ/IVF_HNSW_FLAT | Approximate nearest neighbor search      |
| Scalar      | B-Tree             | Range queries and sorting                |
| Categorical | Bitmap             | Multi-value filters and set operations   |
| `List`      | Label_list         | Multi-label classification and filtering |

<Card title="Index Coverage Monitoring" icon="lightbulb">
Use `table.index_stats()` to monitor index coverage.
A well-optimized table should have `num_unindexed_rows ~ 0`.
</Card>

### 2\. Query Plan Optimization {#2-query-plan-optimization}

#### Common Patterns and Fixes {#common-patterns-and-fixes}

| Plan Pattern                               | Optimization                                 |
| ------------------------------------------ | -------------------------------------------- |
| LanceScan with high _bytes_read_ or _iops_ | Add missing index                            |
|                                            | Use `select()` to limit returned columns     |
|                                            | Check whether the dataset has been compacted |
| Multiple sequential filters                | Reorder filter conditions                    |

<Note>
Regularly analyze your query plans to identify and address performance
bottlenecks. The `analyze_plan` output provides detailed metrics to guide
optimization efforts.
</Note>

### 3\. Getting Started with Optimization {#3-getting-started-with-optimization}

For vector search performance:

- Create ANN index on your vector column(s) as described in the [index guide](/indexing/vector-index/)
- If you often filter by metadata, create [scalar indices](/indexing/scalar-index/) on those columns

### Hybrid queries {#hybrid-queries}

For a hybrid query, `analyze_plan` executes both sub-queries and labels them
`Vector Search Plan:` and `FTS Search Plan:` in the output. This is the easiest
way to see whether the filter pushed into both halves uniformly, and which half
is dominating latency.