From f866e0ad69d29f8875ffb359ebc3613f32cf58d0 Mon Sep 17 00:00:00 2001 From: Chang She <759245+changhiskhan@users.noreply.github.com> Date: Fri, 19 Jul 2024 21:26:39 -0700 Subject: [PATCH] doc: add snippet on incremental reindexing --- docs/src/concepts/data_management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/concepts/data_management.md b/docs/src/concepts/data_management.md index 7ac01781..791182b1 100644 --- a/docs/src/concepts/data_management.md +++ b/docs/src/concepts/data_management.md @@ -55,7 +55,7 @@ When a reindex job is triggered in the background, the entire data is reindexed, ### Vector reindex * LanceDB Cloud supports incremental reindexing, where a background process will trigger a new index build for you automatically when new data is added to a dataset -* LanceDB OSS requires you to manually trigger a reindex operation -- we are working on adding incremental reindexing to LanceDB OSS as well +* LanceDB OSS requires you to manually trigger a reindex operation -- incremental indexing is available via the Lance API `lance_table.to_lance().optimize.optimize_indices()`. Incremental indexing means that any unindexed rows are added to the existing index. This is much faster than a full reindex because it does not involve kmeans training or reconstructing the graph from scratch (depending on your index type). ### FTS reindex