Files
lancedb/docs/web/tables/branching.mdx
Brendan Clement 30c973e64c docs: make the TypeScript and Rust quickstarts complete programs
The TypeScript connection example only defined a helper that nothing
called, so every later step failed with `db is not defined`. It now
connects, as the Python one does. The Rust one kept its connection
inside a helper, the Enterprise one read its settings without
connecting, and the later steps relied on imports and a record-batch
helper the page never showed.

The quickstart now says how the steps make one program: an ES module
run with Node in TypeScript, and the body of an async `main` in Rust,
whose dependencies the install step lists. Each Rust step brings its own
imports, the data step shows the helper that turns the records into a
reader, and Arrow comes from lancedb's re-exports, so no Arrow crate is
needed. The row structs no longer derive serde traits nothing used.

The connection examples are shared, so the connect, create-table and
branching pages gain the import line their connection code now needs.
2026-10-09 17:28:57 -07:00

440 lines
17 KiB
Plaintext

---
title: "Branches"
sidebarTitle: "Branch"
description: "Fork isolated, writable lines of table history in LanceDB. Run experiments, backfills, and index rebuilds without disturbing production reads on main."
icon: "code-branch"
availability:
oss: available
enterprise: available
summary: >-
Branches work in the embedded library and on a deployment. Comparing a
branch with `main` and promoting its new columns onto `main` is available
only on a deployment.
---
import {
PyConnect,
TsConnect,
RsConnect,
PyConnectEnterpriseQuickstart,
TsConnectEnterpriseQuickstart,
RsConnectEnterpriseQuickstart,
} from '/snippets/connection.mdx';
import {
PyBranchCreate as BranchCreate,
TsBranchCreate,
RsBranchCreate,
PyBranchWrite as BranchWrite,
TsBranchWrite,
RsBranchWrite,
PyBranchReopen as BranchReopen,
TsBranchReopen,
RsBranchReopen,
PyBranchDelete as BranchDelete,
TsBranchDelete,
RsBranchDelete,
PyBranchUpsertToMain as BranchUpsertToMain,
TsBranchUpsertToMain,
RsBranchUpsertToMain,
PyBranchIndex as BranchIndex,
TsBranchIndex,
RsBranchIndex,
} from '/snippets/tables.mdx';
A branch is an isolated, writable line of history forked from `main` (or from any
other branch). Anything you do on a branch (e.g., adding rows, changing the schema,
building an index) stays on that branch, so `main` keeps serving production
reads exactly as before. Branches are a natural fit when you want to:
- Experiment with a new index, schema change, or reprocessing step without
affecting live queries on `main`.
- Run a backfill or migration you'd like to validate before applying it to
`main`.
- Hand a collaborator a frozen point-in-time fork while you keep writing to `main`.
## How branches relate to versions and tags {#how-branches-relate-to-versions-and-tags}
Every LanceDB table already tracks a linear history of [versions](/management/table#versioning),
and you can [tag](/management/table#versioning-tag-based-versioning) a version or `checkout`
one to read it. Branches add the missing piece: a *separate, writable* line of
history. Where a tag is a read-only label and `checkout` is a read-only view, a
branch forks from a point in history and then evolves on its own. Creating or
checking out a branch hands you a new table handle whose reads and writes are
scoped to that branch. The [comparison table](#branches-vs-tags-vs-versions) at
the end of this page lays out when to reach for each.
<Note>
Branches are supported on local and namespace-backed tables in LanceDB OSS, as
well as on LanceDB Enterprise (remote) tables.
</Note>
## Connect to a table {#connect-to-a-table}
The branch API is identical no matter how you connect — only the connection
itself differs between OSS and Enterprise. Establish a connection (`db`) and open
a `table` (see [Ingesting data](/ingestion)), then use the same branch calls
in every example that follows.
### LanceDB OSS {#lancedb-oss}
Point LanceDB at a local directory (or an object-storage URI) to use it as an
embedded library.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{PyConnect}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{ "import * as lancedb from \"@lancedb/lancedb\";\n\n" }
{TsConnect}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{ "use lancedb::connect;\n\n" }
{RsConnect}
</CodeBlock>
</CodeGroup>
### LanceDB Enterprise {#lancedb-enterprise}
<Badge color="red">Enterprise</Badge>
Branching on LanceDB Enterprise works the same way as on OSS, apart from the differences
noted below, but you connect to your
Enterprise deployment with a `db://` URI, an API key, and your
region. Once you have a connection, open a table and use the same branch calls in
every example that follows.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{PyConnectEnterpriseQuickstart}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsConnectEnterpriseQuickstart}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{ "use lancedb::connect;\n\n" }
{RsConnectEnterpriseQuickstart}
</CodeBlock>
</CodeGroup>
## Work with branches {#work-with-branches}
The lifecycle of a branch is short and predictable: fork it, write to it, reopen
it whenever you need it, and delete it once you're done. The examples below use a
small `quotes` table with three rows on `main`.
### Create a branch {#create-a-branch}
Forking from `main` returns a table handle scoped to the new branch. `main` is
the reserved default source, so `create` needs only a name; to fork from
somewhere else, pass a branch name, a specific version, or both.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchCreate}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchCreate}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchCreate}
</CodeBlock>
</CodeGroup>
### Write to a branch {#write-to-a-branch}
Writes go through the branch handle and stay there — the `main` handle keeps
reporting its original row count. Listing branches returns a mapping of each
branch name to its metadata, including the version it was forked from.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchWrite}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchWrite}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchWrite}
</CodeBlock>
</CodeGroup>
### Reopen a branch {#reopen-a-branch}
A branch outlives the handle that created it. Reopen it later by name — either
from an existing table handle or straight from the connection when you open the
table. Both routes give you a writable handle tracking the branch's latest state.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchReopen}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchReopen}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchReopen}
</CodeBlock>
</CodeGroup>
### Delete a branch {#delete-a-branch}
Deleting a branch removes it and its branch-local history; `main` is untouched.
A deployment refuses to delete a branch whose columns have been promoted onto
`main`, because `main` still depends on it.
Before deleting a branch, make sure you've retained any results you need — see
[Apply branch-tested changes to `main`](#apply-branch-tested-changes-to-main)
below.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchDelete}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchDelete}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchDelete}
</CodeBlock>
</CodeGroup>
## Apply branch-tested changes to `main` {#apply-branch-tested-changes-to-main}
<Warning>
A branch has its own writable history. Outside of the [diff and cherry-pick
APIs](#compare-and-merge-a-branch-into-main) — which are available on
LanceDB Enterprise and promote added columns only — LanceDB does not reconcile
one branch's history with another or detect conflicts between them. To carry
other accepted work forward, rerun the validated operation against `main` or
explicitly write selected results to it.
</Warning>
How you apply a validated change depends on the type of work:
- **Added columns (Enterprise):** review the branch with `diff`, then promote
the new columns onto `main` with `cherry_pick` (`cherryPick` in TypeScript). See
[Compare and promote a branch into `main`](#compare-and-merge-a-branch-into-main).
- **Backfill or transformation:** rerun the validated job against `main`.
- **Schema change:** apply the same reviewed schema operation to `main`.
- **Index change:** build the index on `main` using the configuration validated
on the branch.
- **Selected row results:** upsert those rows into `main` using a stable unique
key.
### Upsert selected branch rows into `main` {#upsert-selected-branch-rows-into-main}
If the result you want to retain is a set of inserted or updated rows, use
[`merge_insert`](/ingestion/merge-insert#merge-incoming-rows-by-key) to write them to
`main`. Despite its name, `merge_insert` is a row-ingestion operation: it
matches incoming rows by key and does not merge branch histories.
Read the rows you want from the reviewed branch, then upsert them into `main` on
your key column — `id` in this example. Here, `candidate` is the branch handle:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchUpsertToMain}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchUpsertToMain}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchUpsertToMain}
</CodeBlock>
</CodeGroup>
Against a deployment, the synchronous Python table raises for `to_arrow()`; read
the branch's rows with a query instead -- see
[Avoid materializing the whole table](/search#avoid-materializing-the-whole-table).
This operation transfers only the inserted or updated rows in its input. It does
not transfer branch history, schema changes, indexes, or branch-local deletions,
and it does not determine which rows changed after the branch was created.
Reapplying an identical payload is idempotent on the key, but the operation is
not conflict-aware. If `main` has diverged, incoming branch rows can overwrite
newer values with the same key. Read and upsert the whole branch only when that
overwrite is intentional; otherwise, filter the branch read to the rows you
intend to apply.
## Compare and promote a branch into `main` {#compare-and-merge-a-branch-into-main}
<Badge color="red">Enterprise</Badge>
On LanceDB Enterprise, branches expose two review-and-land calls: `diff`
inspects what a branch has changed relative to `main`, and `cherry_pick`
(`cherryPick` in TypeScript) promotes the branch's new columns onto `main` in
place, without reissuing the branch's writes.
<Info>
`diff` and `cherry_pick` work on Enterprise (remote) tables only. On a local
table they fail as not supported: Python raises `NotImplementedError`, and
TypeScript and Rust return an error. A cherry-pick promotes added columns; use
the [upsert](#upsert-selected-branch-rows-into-main) or rerun patterns above for
row and index changes.
</Info>
### Diff a branch {#diff-a-branch}
`diff` reads the branch and `main` without modifying either, and returns what
has changed: columns added, removed or changed, indexes added or removed, row
counts, and a list of `errors` -- the reasons a cherry-pick of the branch could
not land now. An empty list means it could.
<CodeGroup>
```python Python icon="python"
diff = table.branches.diff("exp")
print([c["name"] for c in diff["addedColumns"]]) # columns the branch introduced
print(diff["baseMoved"]) # True if main has moved on since the branch's fork
for error in diff["errors"]: # empty when a cherry-pick could land
print(error["code"], error["message"])
```
```typescript TypeScript icon="square-js"
const branches = await table.branches();
const diff = await branches.diff("exp");
console.log(diff.addedColumns.map((c) => c.name)); // columns the branch introduced
console.log(diff.baseMoved); // true if main has moved on since the branch's fork
for (const error of diff.errors) { // empty when a cherry-pick could land
console.log(error.code, error.message);
}
```
</CodeGroup>
The result also carries the versions it compared (`parentVersion`,
`mainVersion`, `branchVersion`), `rowCountMain` and `rowCountBranch`, a
`rowSummary`, and `removedColumns`, `changedColumns`, `addedIndexes` and
`removedIndexes`. In Python the result is a dictionary with the same camelCase
keys.
The error codes are `baseMoved` (`main` has moved on since the version the
branch was forked from), `rowCountMismatch`, `rowsChanged`, `columnRemoved`,
`columnChanged`, `nothingToApply`, `noColumnChanges`, `inputColumnDependency`
and `parentNotMain`. A code the client does not know surfaces as `unknown`, so
older clients keep working.
### Promote a branch's columns {#merge-a-branch}
`cherry_pick` promotes the columns a branch added onto `main`. The server
re-evaluates the diff when the request arrives and either lands the change or
refuses it for the reasons `diff` would report. A refusal is not an exception:
the call returns a result whose `status` is `"failed"`, with the reasons in
`diff.errors`, so you can decide what to do next. Cherry-picks are not retried.
Pass `dry_run=True` in Python, or `true` as the second argument in TypeScript,
to check without landing anything. `preview.promotedColumns` lists the columns
the cherry-pick would promote, or did.
<CodeGroup>
```python Python icon="python"
# Check first; this does not modify main.
preview = table.branches.cherry_pick("exp", dry_run=True)
print(preview["status"]) # "ready" when it would land
print(preview["preview"]["promotedColumns"])
# Land it.
result = table.branches.cherry_pick("exp")
if result["status"] == "cherryPicked":
print("landed at main version", result["mainVersionAfter"])
elif result["status"] == "failed":
for error in result["diff"]["errors"]:
print(error["code"], error["message"])
```
```typescript TypeScript icon="square-js"
const branches = await table.branches();
// Check first; this does not modify main.
const preview = await branches.cherryPick("exp", true);
console.log(preview.status); // "ready" when it would land
console.log(preview.preview.promotedColumns);
// Land it.
const result = await branches.cherryPick("exp");
if (result.status === "cherryPicked") {
console.log("landed at main version", result.mainVersionAfter);
} else if (result.status === "failed") {
for (const error of result.diff.errors) {
console.log(error.code, error.message);
}
}
```
</CodeGroup>
`status` is `ready` (a dry run that would land), `cherryPicked` (it landed, and
`mainVersionAfter` is the new version of `main`), `failed` (refused, or for a
dry run, would be refused; see `diff.errors`), `notImplemented`, or `unknown`
for a status the client does not know. A branch that does not exist raises a
not-found error instead of returning a status.
In async Python, await the same calls on an `AsyncTable`:
`await table.branches.diff("exp")` and
`await table.branches.cherry_pick("exp", dry_run=True)`. In Rust they are
`Table::diff_branch` and `Table::cherry_pick`.
## Build indexes on a branch {#build-indexes-on-a-branch}
One of the most useful things a branch buys you is a safe place to build and
validate an index without affecting what's in production on the `main` branch.
Fork a branch, create your vector (ANN) and full-text search (FTS)
indexes on it, and check recall and latency. Once you have selected a
configuration, build the corresponding index on `main` through your normal
deployment workflow. Because the indexes live on the branch, queries against
`main` never see a half-built index and are never slowed down by the branch's
build.
This pattern is especially valuable on Enterprise deployments, where `main` is
typically serving production traffic while you tune an index configuration on
the side. A deployment builds the branch's indexes in the background, so wait
for them before measuring.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BranchIndex}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBranchIndex}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBranchIndex}
</CodeBlock>
</CodeGroup>
Schema changes such as adding, altering, or dropping columns are branch-scoped in
the same way, so you can stage and review a larger reshaping of a table before
applying the same schema operations to `main`.
## Branches vs. tags vs. versions {#branches-vs-tags-vs-versions}
Now that you're familiar with branches, you can see how they complement the other ways LanceDB
give you to work with table history. Choose these approaches based on whether you need to
*label*, *read*, or *write* data at a point in history:
| Feature | Writable? | Purpose |
| --- | --- | --- |
| **[Branch](#work-with-branches)** | ✅ Yes | Write on top of a point in history without touching `main`. |
| **[Tag](/management/table#versioning-tag-based-versioning)** | ❌ No | Attach a human-readable label to an existing version; protects it from cleanup. |
| **[`checkout(version)`](/management/table#versioning-rollback-to-previous-versions)** | ❌ No | Read a historical version of `main` without forking. Read-only until you `restore`. |
<Note>
For linear version history — creating versions, listing them, rolling back, and
tagging — see the [Versioning and Reproducibility](/management/table#versioning) guide.
</Note>