mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 15:26:52 +00:00
Will JonesandClaude Opus 5.5
4d7a64712f
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
101 lines
4.9 KiB
Plaintext
101 lines
4.9 KiB
Plaintext
---
|
|
title: "Run experiments on branches"
|
|
sidebarTitle: "Branch Experiments"
|
|
description: "Use LanceDB branches to isolate agent-driven experiments from main, evaluate them on a fixed test set, and promote only the winner."
|
|
icon: "flask"
|
|
keywords: ["AI agents", "branches", "experiments", "embeddings", "evaluation"]
|
|
---
|
|
|
|
LanceDB supports [branching](/tables/branching) to isolate experiments from the main table when working with agents.
|
|
Branches are useful when you instruct an agent to try several approaches without
|
|
affecting the production data on the `main` branch.
|
|
Each experiment gets its own writable table history.
|
|
|
|
For example, you can generate and compare embeddings from two text embedding
|
|
models on the given table:
|
|
|
|
| Branch | Model | Added column |
|
|
| --- | --- | --- |
|
|
| `embed-minilm` | `sentence-transformers/all-MiniLM-L6-v2` | `vector_minilm` |
|
|
| `embed-nomic` | `nomic-embed-text` from Ollama | `vector_nomic` |
|
|
|
|
The branches share a logical table, but each has its own schema and history.
|
|
The original table handle still points to `main`. LanceDB does not have a
|
|
process-wide current branch.
|
|
|
|
Install the model packages, then download the `nomic-embed-text` model from Ollama
|
|
(or any other model you prefer):
|
|
|
|
```bash
|
|
uv add sentence-transformers ollama
|
|
ollama pull nomic-embed-text
|
|
```
|
|
|
|
It's always recommended to measure the results of an experiment on a fixed evaluation set.
|
|
Here's an example of how you could ask the agent to run both experiments and compare the results:
|
|
|
|
```text Agent prompt
|
|
Use the lancedb plugin and the LanceDB branching documentation to compare two
|
|
embedding experiments on the `camelot_multimodal` table.
|
|
|
|
1. Fork `embed-minilm` from `main`. Build normalized embeddings from
|
|
`"{role}. {description}"` with
|
|
`sentence-transformers/all-MiniLM-L6-v2` and add them as `vector_minilm`.
|
|
2. Fork `embed-nomic` from `main`. Embed the identical text with Ollama
|
|
`nomic-embed-text` and add the results as `vector_nomic`.
|
|
3. Process rows in batches and write only through each branch-scoped table
|
|
handle. Verify that neither new column appears on `main`.
|
|
4. Evaluate both branches with the queries "wise magical advisor",
|
|
"treacherous rebel", and "virtuous Grail knight". Report the top three names,
|
|
latency, and whether the expected character ranks first.
|
|
5. Do not modify `main` and do not delete either branch. Recommend a winner
|
|
based on the results.
|
|
```
|
|
|
|
<Note>
|
|
The vector dimensions may differ because the columns live on separate branches.
|
|
Use a different column name for each model. Within an experiment, use the same
|
|
model for the stored text and the search queries.
|
|
</Note>
|
|
|
|
## Apply the winning experiment to `main` {#apply-the-winning-experiment-to-main}
|
|
|
|
The branch experiments leave you with the results side by side, and they
|
|
deliberately never touch `main`. Once you've picked a winner, apply it to
|
|
`main` yourself by rerunning that experiment's validated transformation
|
|
directly against the `main` table. Rerunning the reviewed operation is the
|
|
reliable path on both LanceDB Embedded and LanceDB Enterprise: it replays exactly the transformation
|
|
you validated on the branch, and it works the same regardless of how the branch
|
|
evolved.
|
|
|
|
Suppose `nomic-embed-text` wins. Rerun the same Nomic embedding transformation
|
|
you validated on `embed-nomic` against `main` in batches, then call
|
|
`table.optimize()` on LanceDB Embedded. Verify a bounded vector search on `main`, and keep
|
|
the losing branch around until you're confident in the result.
|
|
|
|
See [Branches](/tables/branching) for more on how branches, versions, and tags
|
|
relate.
|
|
|
|
## More experiments you can run {#more-experiments-you-can-run}
|
|
|
|
Swapping embedding models on a branch while working with agents is only one example
|
|
of what you can do with the LanceDB plugin. The table below shows other experiments
|
|
you can run with the plugin.
|
|
|
|
| Hypothesis | Change on the branch | What to evaluate |
|
|
| --- | --- | --- |
|
|
| A different embedding model improves retrieval | Add a new vector column | Recall, ranking quality, latency, and cost |
|
|
| A new parser or OCR model improves source data | Reprocess files and backfill metadata | Validation failures and a reviewed sample |
|
|
| A new search setup works better | Build an index or change hybrid-search and reranking settings | Relevance and latency on a fixed query set |
|
|
| A curation rule improves the corpus | Deduplicate, classify, or filter records | False positives, false negatives, and row counts |
|
|
| A migration is safe to deploy | Add columns or run a backfill | Schema compatibility and application checks |
|
|
|
|
For each experiment, ask the agent to state the hypothesis, use a fixed
|
|
evaluation set, report the comparison, and, if you want, work on an alternate
|
|
branch that's isolated from `main` until you decide you want the new derived
|
|
column in `main`.
|
|
|
|
Once you've chosen a winner, rerun that validated transformation to ingest it into `main`
|
|
yourself, on either LanceDB Embedded or LanceDB Enterprise. See our documentation on [branches](/tables/branching)
|
|
for more information.
|