mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 23:02:28 +00:00
docs: say which tagged versions stop cleanup, and what zero retention keeps
The cleanup page said timedelta(0) prunes everything except the latest version, and the three pages said any tag on a version older than the window stops cleanup. In the pinned Lance, cleanup also keeps the versions branches were forked from, and only a tagged version that is older than the window and is not the latest stops it; a tag on the latest version is kept without an error. optimize() can still turn such a tag into a blocking one by committing a compaction first. The pages now say so, with moving the tag inside the window or widening the window as remedies alongside deleting it. Each case was checked locally against the baseline SDK.
This commit is contained in:
@@ -76,11 +76,12 @@ exists.
|
||||
<Warning>
|
||||
Retention is finite. [Cleanup](/maintenance/cleanup) prunes versions older
|
||||
than the retention window — **7 days by default** for `optimize()` in the
|
||||
embedded library, while a deployment's configuration sets its own. A tag keeps
|
||||
its version, and while a tag points at a version older than the window, cleanup
|
||||
refuses to run until the tag is deleted; a branch keeps the version it was
|
||||
forked from. If you rely on reading old versions, widen the window before
|
||||
running it, not after.
|
||||
embedded library, while a deployment's configuration sets its own. It always
|
||||
keeps the latest version, the versions branches were forked from and tagged
|
||||
versions. A tagged version that is older than the window and is not the latest
|
||||
stops cleanup altogether, until the tag is moved inside the window or deleted,
|
||||
or the window is widened. If you rely on reading old versions, widen the window
|
||||
before running it, not after.
|
||||
</Warning>
|
||||
|
||||
**Dropping a table ends its history.** Versioning does not survive
|
||||
|
||||
@@ -39,22 +39,30 @@ table.optimize(cleanup_older_than=timedelta(days=30))
|
||||
removed, along with the files only they referenced. On a deployment, the
|
||||
retention period comes from its configuration instead.
|
||||
|
||||
The latest version is never removed, whatever you pass -- `timedelta(0)` prunes
|
||||
everything except it.
|
||||
The latest version is never removed, whatever you pass. `timedelta(0)` makes
|
||||
every older version eligible for removal, but cleanup still keeps the versions
|
||||
that branches and tags protect, as the warning below describes.
|
||||
|
||||
<Warning>
|
||||
Pruning is what makes [time travel](/management/table#versioning) finite: a
|
||||
version outside the retention window is gone, unless something still refers to
|
||||
it.
|
||||
Pruning is what makes [time travel](/management/table#versioning) finite: an
|
||||
eligible version outside the retention window is gone. Cleanup keeps a version,
|
||||
whatever its age, when it is:
|
||||
|
||||
- A **tag** keeps its version, whatever its age. While a tag points at a
|
||||
version older than the window, cleanup refuses to run and raises an error
|
||||
naming the tag; `optimize()` raises it after compacting, so nothing is
|
||||
pruned and its index update does not run. Delete the tag to let cleanup
|
||||
proceed.
|
||||
- A **branch** keeps the version it was forked from, and the files it reads,
|
||||
when its parent is cleaned up; the parent's other old versions are pruned as
|
||||
usual.
|
||||
- the **latest** version;
|
||||
- a version a **branch** was forked from, along with the files the branch
|
||||
reads; the parent's other old versions are pruned as usual;
|
||||
- a **tagged** version.
|
||||
|
||||
Under the default policy, a tag can also stop cleanup. When a tagged version is
|
||||
older than the window and is not the latest, cleanup refuses to run: it raises
|
||||
an error naming the tag and prunes nothing. To let it proceed, move the tag to a
|
||||
version inside the window, delete the tag, or widen the window.
|
||||
|
||||
A tag on the latest version does not stop cleanup. But `optimize()` compacts
|
||||
before it prunes, and compacting commits a new version whenever it merges
|
||||
files, so a tag on what was the latest version can become an older one by the
|
||||
time cleanup runs. `optimize()` then raises after compacting, and its index
|
||||
update does not run.
|
||||
|
||||
A deployment's background cleanup follows the same rules. If you rely on
|
||||
reading old versions, set the window to cover the range you need before
|
||||
|
||||
@@ -1057,8 +1057,9 @@ Numeric table versions like `v3` or `v17` are precise but hard to remember. Tags
|
||||
let you attach human-readable labels (e.g., `"prod"`, `"baseline"`,
|
||||
`"q3-evaluation"`) to specific versions and check those out by name. They are
|
||||
conceptually similar to git tags, and unlike numeric versions, **tagged versions
|
||||
are never pruned** -- cleanup stops instead while a tag points at a version older
|
||||
than its retention window (see the cleanup note at the bottom of this page).
|
||||
are never pruned** -- a tagged version that is older than the retention window and
|
||||
is not the latest stops cleanup instead (see the cleanup note at the bottom of this
|
||||
page).
|
||||
|
||||
The tags API supports the standard CRUD operations — create, list, update, delete —
|
||||
plus checking out by tag name.
|
||||
@@ -1159,8 +1160,12 @@ which is when old-version files are removed and disk space is reclaimed. A deplo
|
||||
when its configuration enables [cleanup](/maintenance/cleanup); `optimize()` is not available there.
|
||||
|
||||
**Tagged versions are never pruned.** A version with a tag pointing at it is
|
||||
retained regardless of age. Cleanup does not prune around it, though: while a tag points
|
||||
at a version older than the retention window, `optimize()` raises an error naming the
|
||||
tag, after compacting and before pruning anything or updating indexes. To let cleanup
|
||||
proceed, [delete the tag](#versioning-tag-based-versioning) first.
|
||||
retained regardless of age. When a tagged version is older than the retention window and
|
||||
is not the latest, cleanup refuses to run instead of pruning around it: `optimize()` raises
|
||||
an error naming the tag and prunes nothing. A tag on the latest version does not cause
|
||||
this, but `optimize()` compacts before it prunes, and compacting commits a new version
|
||||
whenever it merges files, so a tag on what was the latest version can become an older
|
||||
one by the time cleanup runs; the error then comes after compacting, and the index
|
||||
update does not run. To let cleanup proceed, [move the tag](#versioning-tag-based-versioning)
|
||||
to a version inside the retention window, delete it, or widen the window.
|
||||
</Note>
|
||||
|
||||
Reference in New Issue
Block a user