docs: describe tag and branch retention as Lance cleanup implements it

The cleanup and guarantees pages said a tag or branch pointing at a pruned
version stops resolving, while the table page said tagged versions are
exempt and quietly kept. The pinned Lance (ae2903729f02, the commit in
Cargo.lock and in Sophon's Lance submodule) keeps a tagged version
whatever its age, but with its default policy refuses to clean up while a
tag points at a version older than the window, raising an error that
names the tag. optimize() passes that default, so it raises after
compacting, prunes nothing and skips its index update until the tag is
deleted. Sophon's cleanup job uses the same default.

Branches are retained separately: cleaning the parent keeps the version a
branch was forked from and the files the branch reads, while the
parent's other old versions are pruned. The pages now say both, checked
locally against the baseline SDK.
This commit is contained in:
Brendan Clement
2026-10-09 09:11:56 -07:00
parent 43f982f1d3
commit 058dca70b0
3 changed files with 27 additions and 13 deletions
+5 -3
View File
@@ -74,9 +74,11 @@ exists.
<Warning>
Retention is finite. [Cleanup](/maintenance/cleanup) prunes versions older
than the retention window — **7 days by default** for `optimize()` in the
embedded library, while a deployment's configuration sets its own — and a tag or branch
pointing at a pruned version no longer resolves. If you rely on reading old
versions, widen the window before running it, not after.
embedded library, while a deployment's configuration sets its own. A tag keeps
its version, and while a tag points at a version older than the window, cleanup
refuses to run until the tag is deleted; a branch keeps the version it was
forked from. If you rely on reading old versions, widen the window before
running it, not after.
</Warning>
**Dropping a table ends its history.** Versioning does not survive
+15 -5
View File
@@ -43,12 +43,22 @@ The latest version is never removed, whatever you pass -- `timedelta(0)` prunes
everything except it.
<Warning>
Pruning is what makes [time travel](/management/table#versioning) finite. A
version outside the retention window is gone, and a tag or branch pointing at
it no longer resolves.
Pruning is what makes [time travel](/management/table#versioning) finite: a
version outside the retention window is gone, unless something still refers to
it.
If you rely on reading old versions, set the window to cover the range you
need before running this, not after.
- A **tag** keeps its version, whatever its age. While a tag points at a
version older than the window, cleanup refuses to run and raises an error
naming the tag; `optimize()` raises it after compacting, so nothing is
pruned and its index update does not run. Delete the tag to let cleanup
proceed.
- A **branch** keeps the version it was forked from, and the files it reads,
when its parent is cleaned up; the parent's other old versions are pruned as
usual.
A deployment's background cleanup follows the same rules. If you rely on
reading old versions, set the window to cover the range you need before
running this, not after.
</Warning>
## Disk usage does not drop immediately {#disk}
+7 -5
View File
@@ -1057,8 +1057,8 @@ Numeric table versions like `v3` or `v17` are precise but hard to remember. Tags
let you attach human-readable labels (e.g., `"prod"`, `"baseline"`,
`"q3-evaluation"`) to specific versions and check those out by name. They are
conceptually similar to git tags, and unlike numeric versions, **tagged versions
are preserved when old versions are pruned** (see the cleanup note at the bottom
of this page).
are never pruned** -- cleanup stops instead while a tag points at a version older
than its retention window (see the cleanup note at the bottom of this page).
The tags API supports the standard CRUD operations — create, list, update, delete —
plus checking out by tag name.
@@ -1158,7 +1158,9 @@ In LanceDB OSS, `optimize()` can prune older versions based on its retention set
which is when old-version files are removed and disk space is reclaimed. A deployment prunes them in the background
when its configuration enables [cleanup](/maintenance/cleanup); `optimize()` is not available there.
**Tagged versions are exempt from cleanup.** A version with a tag pointing at it is
retained regardless of age, and its files are not removed by `optimize()`. To make
a tagged version eligible for pruning, [delete the tag](#versioning-tag-based-versioning) first.
**Tagged versions are never pruned.** A version with a tag pointing at it is
retained regardless of age. Cleanup does not prune around it, though: while a tag points
at a version older than the retention window, `optimize()` raises an error naming the
tag, after compacting and before pruning anything or updating indexes. To let cleanup
proceed, [delete the tag](#versioning-tag-based-versioning) first.
</Note>