mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 23:02:28 +00:00
docs: describe tag and branch retention as Lance cleanup implements it
The cleanup and guarantees pages said a tag or branch pointing at a pruned version stops resolving, while the table page said tagged versions are exempt and quietly kept. The pinned Lance (ae2903729f02, the commit in Cargo.lock and in Sophon's Lance submodule) keeps a tagged version whatever its age, but with its default policy refuses to clean up while a tag points at a version older than the window, raising an error that names the tag. optimize() passes that default, so it raises after compacting, prunes nothing and skips its index update until the tag is deleted. Sophon's cleanup job uses the same default. Branches are retained separately: cleaning the parent keeps the version a branch was forked from and the files the branch reads, while the parent's other old versions are pruned. The pages now say both, checked locally against the baseline SDK.
This commit is contained in:
@@ -74,9 +74,11 @@ exists.
|
||||
<Warning>
|
||||
Retention is finite. [Cleanup](/maintenance/cleanup) prunes versions older
|
||||
than the retention window — **7 days by default** for `optimize()` in the
|
||||
embedded library, while a deployment's configuration sets its own — and a tag or branch
|
||||
pointing at a pruned version no longer resolves. If you rely on reading old
|
||||
versions, widen the window before running it, not after.
|
||||
embedded library, while a deployment's configuration sets its own. A tag keeps
|
||||
its version, and while a tag points at a version older than the window, cleanup
|
||||
refuses to run until the tag is deleted; a branch keeps the version it was
|
||||
forked from. If you rely on reading old versions, widen the window before
|
||||
running it, not after.
|
||||
</Warning>
|
||||
|
||||
**Dropping a table ends its history.** Versioning does not survive
|
||||
|
||||
@@ -43,12 +43,22 @@ The latest version is never removed, whatever you pass -- `timedelta(0)` prunes
|
||||
everything except it.
|
||||
|
||||
<Warning>
|
||||
Pruning is what makes [time travel](/management/table#versioning) finite. A
|
||||
version outside the retention window is gone, and a tag or branch pointing at
|
||||
it no longer resolves.
|
||||
Pruning is what makes [time travel](/management/table#versioning) finite: a
|
||||
version outside the retention window is gone, unless something still refers to
|
||||
it.
|
||||
|
||||
If you rely on reading old versions, set the window to cover the range you
|
||||
need before running this, not after.
|
||||
- A **tag** keeps its version, whatever its age. While a tag points at a
|
||||
version older than the window, cleanup refuses to run and raises an error
|
||||
naming the tag; `optimize()` raises it after compacting, so nothing is
|
||||
pruned and its index update does not run. Delete the tag to let cleanup
|
||||
proceed.
|
||||
- A **branch** keeps the version it was forked from, and the files it reads,
|
||||
when its parent is cleaned up; the parent's other old versions are pruned as
|
||||
usual.
|
||||
|
||||
A deployment's background cleanup follows the same rules. If you rely on
|
||||
reading old versions, set the window to cover the range you need before
|
||||
running this, not after.
|
||||
</Warning>
|
||||
|
||||
## Disk usage does not drop immediately {#disk}
|
||||
|
||||
@@ -1057,8 +1057,8 @@ Numeric table versions like `v3` or `v17` are precise but hard to remember. Tags
|
||||
let you attach human-readable labels (e.g., `"prod"`, `"baseline"`,
|
||||
`"q3-evaluation"`) to specific versions and check those out by name. They are
|
||||
conceptually similar to git tags, and unlike numeric versions, **tagged versions
|
||||
are preserved when old versions are pruned** (see the cleanup note at the bottom
|
||||
of this page).
|
||||
are never pruned** -- cleanup stops instead while a tag points at a version older
|
||||
than its retention window (see the cleanup note at the bottom of this page).
|
||||
|
||||
The tags API supports the standard CRUD operations — create, list, update, delete —
|
||||
plus checking out by tag name.
|
||||
@@ -1158,7 +1158,9 @@ In LanceDB OSS, `optimize()` can prune older versions based on its retention set
|
||||
which is when old-version files are removed and disk space is reclaimed. A deployment prunes them in the background
|
||||
when its configuration enables [cleanup](/maintenance/cleanup); `optimize()` is not available there.
|
||||
|
||||
**Tagged versions are exempt from cleanup.** A version with a tag pointing at it is
|
||||
retained regardless of age, and its files are not removed by `optimize()`. To make
|
||||
a tagged version eligible for pruning, [delete the tag](#versioning-tag-based-versioning) first.
|
||||
**Tagged versions are never pruned.** A version with a tag pointing at it is
|
||||
retained regardless of age. Cleanup does not prune around it, though: while a tag points
|
||||
at a version older than the retention window, `optimize()` raises an error naming the
|
||||
tag, after compacting and before pruning anything or updating indexes. To let cleanup
|
||||
proceed, [delete the tag](#versioning-tag-based-versioning) first.
|
||||
</Note>
|
||||
|
||||
Reference in New Issue
Block a user