rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-12 14:52:26 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: declare where indexing and maintenance apply

Browse Source 
Indexes are built in the embedded library and on a deployment, which
builds them in the background, keeps most of them current, and builds
only IVF_FLAT, IVF_PQ, IVF_RQ and IVF_HNSW_SQ vector indexes. Compaction
runs on both, by optimize() locally and in the background on a
deployment; cleanup on a deployment runs only where its configuration
enables it, so it varies; background jobs exist only on a deployment.

Sections that described one offering as if it were both now name it:
optimize() is embedded only, the brute-force limit and automatic IVF_PQ
are deployment settings, FM indexes get no background updates, a
remainder below the deployment's threshold can keep wait_for_index from
finishing, GPU builds on a deployment are IVF_PQ with a configured GPU
count, list_indices on a deployment omits num_unindexed_rows, and the
background jobs page no longer says every other job is one you start.
This commit is contained in:
Brendan Clement
2026-10-08 21:31:25 -07:00
parent e85f1b4434
commit 027597d36d
8 changed files with 121 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/web/indexing/fts-index.mdx
+8 -1
View File
@@ -3,6 +3,13 @@ title: "Full-Text Search (FTS) Index"
sidebarTitle: "FTS Index"
description: "Create and tune BM25-based full-text search indexes in LanceDB."
icon: "book"
availability:
oss: available
enterprise: available
summary: >-
Full-text indexes are built in the embedded library and on a deployment.
The embedded library builds the index before the call returns; a
deployment builds it in the background.
---
import { PyFtsIndexAsync as FtsIndexAsync, PyFtsIndexCreate as FtsIndexCreate, PyFtsIndexNested as FtsIndexNested, PyFtsIndexWait as FtsIndexWait } from '/snippets/indexing.mdx';
@@ -114,7 +121,7 @@ await async_table.create_index("payload.text", config=FTS(with_position=True))
- `jieba/*` is for Chinese word segmentation with Jieba.
- `lindera/*` loads a compiled Lindera dictionary, such as `lindera/ipadic` for Japanese or `lindera/ko-dic` for Korean.
Model-backed tokenizers such as `jieba/default`, `lindera/ipadic`, and `lindera/ko-dic` require tokenizer model files in Lance's language model home. Lance looks under the default platform data directory for `lance/language_models`, or you can set `LANCE_LANGUAGE_MODEL_HOME` to point to another model root. For example, `jieba/default` is resolved under `<model-home>/jieba/default/...`.
Model-backed tokenizers such as `jieba/default`, `lindera/ipadic`, and `lindera/ko-dic` require tokenizer model files in Lance's language model home. Lance looks under the default platform data directory for `lance/language_models`, or you can set `LANCE_LANGUAGE_MODEL_HOME` to point to another model root. For example, `jieba/default` is resolved under `<model-home>/jieba/default/...`. For a table on a deployment, the files must be present where the deployment builds and queries the index, so ask its administrator whether they are installed.
`language` is used by token filters, not by the base tokenizer. Stemming supports Arabic, Danish, Dutch, English, Finnish, French, German, Greek, Hungarian, Italian, Norwegian, Portuguese, Romanian, Russian, Spanish, Swedish, Tamil, and Turkish. Built-in stop-word removal supports Danish, Dutch, English, Finnish, French, German, Hungarian, Italian, Norwegian, Portuguese, Russian, Spanish, and Swedish. For other stemming languages, set `remove_stop_words=False` or pass `custom_stop_words`.
docs/web/indexing/index.mdx
+13 -2
View File
@@ -3,6 +3,13 @@ title: "Indexing Data"
sidebarTitle: "Overview"
description: "Optimize search performance with LanceDB using vector indexes, full-text search, scalar indexes, and more."
icon: "list"
availability:
oss: available
enterprise: available
summary: >-
Indexes are built in the embedded library and on a deployment. A deployment
builds them in the background, and builds fewer vector index types than the
embedded library.
---
An **index** is a data structure that facilitates efficient scans and lookups on an embedded dataset.
@@ -24,6 +31,10 @@ LanceDB provides a comprehensive suite of indexes to optimize performance across
| `LabelList` (Scalar) | List columns (e.g., tags, categories, keywords) requiring `array_contains_all` or `array_contains_any` filters. | Scalar index for `List<T>` and `LargeList<T>` columns of primitive values, using an underlying bitmap index structure to enable fast array membership lookups. |
| `FM` (Scalar) | String or binary columns that need raw substring search. | FM-Index over `Utf8`, `LargeUtf8`, `Binary`, or `LargeBinary` data for filters such as `contains(path, 'needle')`. Use FTS instead for tokenized word search and BM25 ranking. |
A deployment builds the vector indexes `IVF_FLAT`, `IVF_PQ`, `IVF_RQ` and
`IVF_HNSW_SQ`. `IVF_SQ`, `IVF_HNSW_FLAT` and `IVF_HNSW_PQ` are built only by the
embedded library.
## Quantization
LanceDB also supports several [quantization](/indexing/vector-index) methods, used by vector indexes to compress vectors and reduce storage requirements:
@@ -42,12 +53,12 @@ If vector search latency climbs with table size (i.e., queries that ran in milli
| Index | When to use |
|-------|-------------|
| `IVF_PQ` | General-purpose default; what Enterprise builds automatically. Higher accuracy than RQ at small dimensions (≤ 256). |
| `IVF_PQ` | General-purpose default, and what a deployment builds when it is set to index a column automatically. Higher accuracy than RQ at small dimensions (≤ 256). |
| `IVF_RQ` | Maximum compression on high-dim vectors, faster builds than PQ. |
| `IVF_HNSW_SQ` | Best recall/latency for unfiltered search; higher latency variance under selective filters. |
| `IVF_FLAT` | Required for binary vectors with `hamming`. |
The distance metric is fixed once the index is built. Pick the distance metric based on how the embedding model was trained: `cosine` (unnormalized), `dot` (already-normalized, best performance), `l2` (general-purpose, default), `hamming` (binary). If a query explicitly requests a different metric, the index cannot be used and the query falls back to a brute-force scan; LanceDB Enterprise may reject that scan on a large table. For parameter tuning, see [Vector Indexing](/indexing/vector-index).
The distance metric is fixed once the index is built. Pick the distance metric based on how the embedding model was trained: `cosine` (unnormalized), `dot` (already-normalized, best performance), `l2` (general-purpose, default), `hamming` (binary). If a query explicitly requests a different metric, the index cannot be used and the query falls back to a brute-force scan; a deployment can be configured to reject that scan on a large table. For parameter tuning, see [Vector Indexing](/indexing/vector-index).
#### Scalar indexes {#scalar-indexes}
docs/web/indexing/reindexing.mdx
+18 -9
View File
@@ -4,6 +4,14 @@ sidebarTitle: "Build & Repair"
description: "Learn how to keep your indexes up-to-date in LanceDB using incremental indexing, including best practices for adding new records without full reindexing."
icon: "refresh"
keywords: ["reindexing", "re-index", "incremental indexing", "reindex"]
availability:
oss: available
enterprise: available
summary: >-
Indexes are kept up to date in the embedded library and on a deployment. In
the embedded library you add new rows to them with `optimize()`; a
deployment does it in the background, and `optimize()` is not available
there.
---
import {
@@ -28,8 +36,9 @@ matching `reindex_*` type instead.
## Incremental Reindexing {#incremental-reindexing}
You can manually trigger an incremental indexing operation on updated data
using the `optimize()` method on a table.
In the embedded library, you trigger an incremental indexing operation on
updated data with the `optimize()` method on a table. A deployment does this
for you and does not support `optimize()`, as described below.
Table optimization performs three maintenance operations:
@@ -45,10 +54,11 @@ Table optimization performs three maintenance operations:
<Badge color="red">Enterprise</Badge>
LanceDB Enterprise support incremental reindexing through an automated background process. When new data is added to a table, the system automatically triggers a new index build. As the dataset grows, indexes are asynchronously updated in the background.
LanceDB Enterprise support incremental reindexing through an automated background process. Once enough new rows have been added to a table, the system automatically adds them to its indexes. As the dataset grows, indexes are asynchronously updated in the background.
- While indexes are being rebuilt, queries use brute force methods on unindexed rows, which may temporarily increase latency. To avoid this, set `fast_search=True` to search only indexed data.
- Use `index_stats()` to view the number of unindexed rows. This will be zero when indexes are fully up-to-date. If you call `wait_for_index(...)`, it polls the same status and can time out while continuous writes keep adding unindexed rows.
- Rows below the deployment's threshold stay unindexed until more arrive, so the count can stay above zero without further writes. `FM` indexes are not updated in the background.
The benefit of using LanceDB Enterprise is that it automates the reindexing process
and operates continuously in the background, minimizing the impact on latency under high loads.
@@ -82,9 +92,8 @@ is capable of indexing billions of rows in under four hours on a 1-8 GPU cluster
Automatic GPU Indexing is currently only available in [LanceDB Enterprise](/enterprise/).
Please [contact us](https://www.lancedb.com/contact) to enable this feature for your deployment.
The vector index is created when you call `create_index`. The backend will use GPU resources
to build either the IVF or HNSW indexes. The system automatically selects the optimal GPU
configuration based on your data size and available hardware.
The vector index is created when you call `create_index`. The backend uses GPU resources
to build `IVF_PQ` indexes, with the number of GPUs set by the deployment's configuration.
This process is also asynchronous by default, but you can use `wait_for_index` to convert it
into a synchronous process by waiting until the index is built.
@@ -93,9 +102,9 @@ into a synchronous process by waiting until the index is built.
`wait_for_index(...)` waits for the index to exist and for `index_stats(...)` to report `num_unindexed_rows == 0`. It can time out if the table is receiving continuous writes while the build is trying to catch up.
GPU acceleration changes how the vector index is built, not the lifecycle after the build. Rows
appended later still need `optimize()` before they are part of the index, and `fast_search()` only
searches indexed rows.
GPU acceleration changes how the vector index is built, not the lifecycle after the build. In the
embedded library, rows appended later still need `optimize()` before they are part of the index; a
deployment adds them in the background. `fast_search()` only searches indexed rows.
### Manual GPU indexing in LanceDB OSS {#gpu-manual-gpu-indexing-in-lancedb-oss}
docs/web/indexing/scalar-index.mdx
+12 -3
View File
@@ -3,6 +3,13 @@ title: "Scalar Indexes"
sidebarTitle: "Scalar Index"
description: "Learn how to use scalar indexes in LanceDB for efficient metadata filtering and query optimization."
icon: "tree"
availability:
oss: available
enterprise: available
summary: >-
Scalar indexes are built in the embedded library and on a deployment. A
deployment builds them in the background and keeps most of them up to date
itself; in the embedded library you update them with `optimize()`.
---
import {
@@ -70,11 +77,11 @@ If you are using LanceDB Enterprise, the `create_scalar_index` API returns immed
</CodeBlock>
</CodeGroup>
`wait_for_index(...)` waits until the named scalar indexes exist and `index_stats(...)` reports `num_unindexed_rows == 0`. If a table is receiving steady writes, that fully indexed state may not stabilize before the timeout.
`wait_for_index(...)` waits until the named scalar indexes exist and `index_stats(...)` reports `num_unindexed_rows == 0`. If a table is receiving steady writes, that fully indexed state may not stabilize before the timeout. On a deployment, rows added after the index was built are indexed only once enough have accumulated, so a small remainder can also keep the wait from finishing.
### 3. Update the Index
Updating the table data (adding, deleting, or modifying records) requires that you also update the scalar index. This can be done by calling `optimize`, which will trigger an update to the existing scalar index.
Updating the table data (adding, deleting, or modifying records) requires that you also update the scalar index. In the embedded library, call `optimize`, which will trigger an update to the existing scalar index.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
@@ -83,7 +90,7 @@ Updating the table data (adding, deleting, or modifying records) requires that y
</CodeGroup>
<Note title="LanceDB Enterprise">
New data added after creating the scalar index will still appear in search results if optimize is not used, but with increased latency due to a flat search on the unindexed portion. LanceDB Enterprise automates the optimize process, minimizing the impact on search speed.
New data added after creating the scalar index will still appear in search results if optimize is not used, but with increased latency due to a flat search on the unindexed portion. LanceDB Enterprise does not support `optimize()`; instead it adds new rows to `BTREE`, `BITMAP` and `LABEL_LIST` indexes in the background once enough have accumulated. It does not update `FM` indexes automatically.
</Note>
### 4. Run Indexed Searches
@@ -143,6 +150,8 @@ or language-aware tokenization.
Build an FM-Index with the async `create_index` API by passing the `Fm` config in
Python or `Index.fm()` in TypeScript. In Rust, use `Index::Fm(FmIndexBuilder::default())`.
A deployment builds FM-Indexes too, but does not add new rows to them in the background as it
does for the other scalar indexes.
<CodeGroup>
```python Python icon="python"
docs/web/indexing/vector-index.mdx
+22 -10
View File
@@ -3,6 +3,14 @@ title: "Vector Indexes"
sidebarTitle: "Vector Index"
description: "Build and manage LanceDB vector indexes."
icon: "arrow-up-right-dots"
availability:
oss: available
enterprise: available
summary: >-
Vector indexes are built in the embedded library and on a deployment. The
embedded library builds an index before `create_index()` returns; a
deployment builds it in the background, keeps it up to date, and builds
only `IVF_FLAT`, `IVF_PQ`, `IVF_RQ` and `IVF_HNSW_SQ`.
---
import {
@@ -53,7 +61,7 @@ Lance with `index_type`.
| Maximum compression | `IVF_RQ` | IVF indexing with RaBitQ quantization | Around `1/32` of raw size | `IvfRq` |
| Higher accuracy at small dimensions (`dimension <= 256`) | `IVF_PQ` | IVF indexing with product quantization | Usually `1/64` to `1/16` of raw size (depends on `num_sub_vectors`) | `IvfPq` |
| Best recall/latency trade-off | `IVF_HNSW_SQ` | IVF-HNSW indexing with scalar | Typically a little larger than `1/4` of raw size | `IvfHnswSq` |
| Highest recall / no quantization | `IVF_HNSW_FLAT` | IVF-HNSW indexing with no quantization | Around raw vector size plus HNSW graph overhead | `IvfHnswFlat` |
| Highest recall / no quantization | `IVF_HNSW_FLAT` | IVF-HNSW indexing with no quantization; embedded library only | Around raw vector size plus HNSW graph overhead | `IvfHnswFlat` |
<Warning>
If your vector search frequently includes metadata filters (`where(...)`), use `IVF_RQ` or `IVF_PQ`. In filtered workloads, HNSW-backed IVF indexes such as `IVF_HNSW_FLAT` and `IVF_HNSW_SQ` can show higher latency variance.
@@ -114,7 +122,7 @@ your destination. This iterative series of road networks mimics the layered grap
In LanceDB, HNSW is not exposed as a top-level vector index. Instead, it's available as a substructure which
further indexes the selected vectors inside each IVF partition.
This combines the scalability of IVF with the high recall of HNSW.
LanceDB supports IVF-HNSW-based quantized indexes `IVF_HNSW_FLAT`, `IVF_HNSW_PQ`, and `IVF_HNSW_SQ`.
LanceDB supports IVF-HNSW-based quantized indexes `IVF_HNSW_FLAT`, `IVF_HNSW_PQ`, and `IVF_HNSW_SQ`; a deployment builds only `IVF_HNSW_SQ`.
</Info>
@@ -152,8 +160,8 @@ The `distance_type` can be specified at search time in addition to build time. I
you do this, set it to the same value used to build the vector index. If the search
metric does not match the index metric, LanceDB cannot use the index and falls
back to a costly flat search. On LanceDB Cloud, the brute-force KNN safety limit
can reject that search on a large table instead of executing it. LanceDB
Enterprise doesn't have the limit.
can reject that search on a large table instead of executing it. An Enterprise
deployment applies the same limit only when it is configured to.
When the search metric is omitted, current SDKs use the index metric if an index
is available. Some older SDK versions sent `l2` when it was omitted, so explicitly
@@ -173,6 +181,8 @@ older client.
**Filtered queries and adaptive `nprobes`.** When a `where(...)` filter is active, LanceDB initially scans `minimum_nprobes`
partitions and uses a wider scan if too few rows are found.
Set `minimum_nprobes == maximum_nprobes` or call `nprobes(n)` to instead fix the partition count.
Whether a deployment widens the scan this way depends on its version; one that does not scans exactly
`minimum_nprobes` partitions.
</Note>
Here is an example of a vector search exercising several of the above parameters.
@@ -265,17 +275,18 @@ The snippet below builds and queries an `IVF_HNSW_SQ` index.
<Badge color="red">Enterprise-only</Badge>
In LanceDB Enterprise, vector indexes are managed **automatically**. The system asynchronously updates and optimizes indexes as a background process:
- Automatically manages indexing parameters and storage
- Infers vector columns from the schema
- GPU-based indexing is available on request -- [contact support](/deployment/support) to enable it for your deployment. A few billion vectors index in under 4 hours on GPU.
- Can build an `IVF_PQ` index automatically on a vector column the deployment is configured to index
- GPU-based `IVF_PQ` builds are available on request -- [contact support](/deployment/support) to enable them for your deployment. A few billion vectors index in under 4 hours on GPU.
<Badge color="gray">Open-Source</Badge>
LanceDB OSS users can manually create vector indexes by calling `table.create_index()`.
See the above sections for guidance on manually tuning index parameters as data changes.
<Note>
`create_index()` returns immediately, but the vector index builds asynchronously.
On a deployment, `create_index()` returns immediately and the vector index builds asynchronously.
To wait until all data is indexed, specify the `wait_timeout` parameter, or call `wait_for_index(...)` afterward —
it waits for the named index to exist and for `index_stats(...)` to report `num_unindexed_rows == 0`.
In the embedded library the index is built before `create_index()` returns.
</Note>
<Note>
@@ -287,11 +298,12 @@ fallback and searches only indexed rows.
<Info>
**Operational checks**
After appends or other writes, use `optimize()` to fold new rows into existing indexes.
In the embedded library, use `optimize()` after appends or other writes to fold new rows into existing indexes.
A deployment does this in the background, and `optimize()` is not available there.
</Info>
#### Check Index Status
Vector index creation runs in the background and may take some time to complete.
On a deployment, vector index creation runs in the background and may take some time to complete.
While it is ongoing, you can check its status through the API or the **LanceDB Enterprise UI**.
In the LanceDB Enterprise UI, navigate to your table page - the "Index" column reflects each column's index status: it is blank when no index exists, shows an "in progress" label while the index is being built, and shows the index type once the build completes.
@@ -318,7 +330,7 @@ Each entry returned by `list_indices()` also carries detailed per-index metadata
<Note>
These fields are populated for local and embedded tables. On LanceDB Enterprise remote tables they are returned as `None` / `undefined` until the server response surfaces them.
These fields are populated for local tables. On a deployment, `list_indices()` returns them except `num_unindexed_rows`; use `index_stats()` for index coverage.
</Note>
#### Custom Index Names
docs/web/maintenance/background-jobs.mdx
+17 -11
View File
@@ -4,19 +4,25 @@ sidebarTitle: "Background Jobs"
description: "Jobs a deployment runs on its own — visible in SHOW JOBS, not started by a call you make."
icon: gears
keywords: ["background jobs", "job type", "wal", "spfresh", "prewarm", "archive"]
availability:
oss: unavailable
enterprise: available
summary: >-
Background jobs run only on a deployment, which starts them itself; its
configuration decides which of them run.
---
<Badge color="red">Enterprise</Badge>
Every job documented elsewhere in these docs -- [Build & Repair](/indexing/reindexing),
The jobs documented elsewhere in these docs -- [Build & Repair](/indexing/reindexing),
[Compaction](/maintenance/compaction), [Cleanup](/maintenance/cleanup),
[Column Refresh](/feature-engineering/refresh-column) -- is one you cause,
directly or by a schedule you set. This page is the rest: jobs the deployment
runs to keep itself correct and fast, which you may see named in
[`SHOW JOBS`](/sql/ddl/job#show) without having asked for any of them.
[Column Refresh](/feature-engineering/refresh-column) -- start from a call you
make or, for compaction, cleanup and index updates, from the deployment's own
maintenance. This page is the rest: jobs the deployment runs to keep itself
correct and fast, which you may see named in [`SHOW JOBS`](/sql/ddl/job#show)
without having asked for any of them.
There is no action to take on these beyond recognizing the name. None of them
has a parameter you set or a call you make.
has a parameter you set from an SDK; the deployment's operators configure them,
and start the cache-warming jobs through its admin API.
## Write-ahead log maintenance {#wal}
@@ -41,7 +47,7 @@ log from growing without bound:
| --- | --- |
| `prewarm_page_cache` | Loads table data into a query node's page cache ahead of a request needing it. |
| `prewarm_index_cache` | The same, for index structures. |
| `startup_prewarm_idx` | Runs cache prewarming as part of a query node's own startup, rather than on a schedule. |
| `startup_prewarm_idx` | Runs cache prewarming as part of a plan executor's own startup, rather than on a schedule, when the deployment enables it. |
## Registry housekeeping {#registry}
@@ -51,8 +57,8 @@ log from growing without bound:
| `job_events_export` | Copies terminal jobs' event histories into a curated system table for metering. Registry-scoped, like the archive sweep. |
These two are why an old job you completed weeks ago no longer shows in
`SHOW JOBS` -- it was archived, not deleted; its events survive in the exported
table.
`SHOW JOBS` -- it was archived, not deleted. Where the deployment enables the
export, its events also survive in the exported table.
## Object teardown {#teardown}
docs/web/maintenance/cleanup.mdx
+18 -4
View File
@@ -4,6 +4,15 @@ sidebarTitle: "Cleanup"
description: "Pruning old versions and the files only they referenced."
icon: broom
keywords: ["cleanup", "retention", "prune", "delete_unverified", "vacuum"]
availability:
oss: available
enterprise: varies
summary: >-
In the embedded library, `optimize()` removes versions older than a
retention window, 7 days by default. A deployment removes old versions in
the background only when its configuration enables cleanup, and that
configuration sets the retention period; `optimize()` is not available
there.
---
Every write keeps the version before it, which is what makes
@@ -14,7 +23,8 @@ those versions referenced.
<Note>
**Enterprise job type:** `cleanup`. Distinct from `compact` -- compaction
merges files, cleanup removes them. `optimize()` in OSS runs both in one call;
Enterprise schedules them as separate jobs.
Enterprise schedules them as separate jobs, and runs cleanup only when the
deployment's configuration enables it.
</Note>
## Retention {#retention}
@@ -26,7 +36,8 @@ table.optimize(cleanup_older_than=timedelta(days=30))
```
`cleanup_older_than` defaults to **7 days**. Versions older than that are
removed, along with the files only they referenced.
removed, along with the files only they referenced. On a deployment, the
retention period comes from its configuration instead.
The latest version is never removed, whatever you pass -- `timedelta(0)` prunes
everything except it.
@@ -46,8 +57,8 @@ Compaction writes new, merged files before the old ones are removed, so disk
usage can rise before it falls. Space is reclaimed only when cleanup prunes the
versions that referenced the superseded files.
If you need to reclaim space aggressively, shorten the retention window rather
than waiting for the default:
If you need to reclaim space aggressively in the embedded library, shorten the
retention window rather than waiting for the default:
```python
table.optimize(cleanup_older_than=timedelta(days=1))
@@ -63,6 +74,9 @@ old, because it cannot tell them apart from a live write.
table.optimize(delete_unverified=True)
```
This option belongs to `optimize()` in the embedded library. A deployment's
cleanup jobs never set it.
<Warning>
`delete_unverified=True` removes them regardless of age. Only set it when you
can guarantee **no other process is writing to the table** -- otherwise it can
docs/web/maintenance/compaction.mdx
+13 -2
View File
@@ -3,13 +3,22 @@ title: "Compaction and cleanup"
sidebarTitle: "Compaction"
description: "Merge small files, prune old versions, and fold new rows into existing indexes."
icon: broom
availability:
oss: available
enterprise: available
summary: >-
Tables are compacted in the embedded library and on a deployment. In the
embedded library you call `optimize()`; a deployment compacts tables in the
background, and `optimize()` is not available there.
---
import { PyStorageRebuildVectorIndex } from '/snippets/doc_storage_examples.mdx';
Every write commits a new table version, and versions accumulate: small files
from frequent appends, superseded files from updates and deletes, and rows added
since an index was built. `optimize()` addresses all three.
since an index was built. In the embedded library, `optimize()` addresses all
three; a deployment handles them in the background and does not support
`optimize()`.
It is modelled after `VACUUM` in PostgreSQL — routine housekeeping that a table
needs periodically rather than a repair for something broken.
@@ -60,7 +69,9 @@ and can help after a large ingest that changed the data distribution.
## When to run it {#when}
There is no background compactor: `optimize()` runs when you call it.
In the embedded library there is no background compactor: `optimize()` runs
when you call it. A deployment compacts a table automatically once enough small
fragments accumulate, unless its operator turns this off.
Frequent small appends are what make it necessary, because each one commits
files. A table written in a few large batches needs it rarely; one taking