mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
Jack YeandClaude Sonnet 5
de95ed3f6d
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
103 lines
5.0 KiB
Plaintext
103 lines
5.0 KiB
Plaintext
---
|
|
title: "Run experiments on branches"
|
|
sidebarTitle: "Branch Experiments"
|
|
description: "Use LanceDB branches to isolate agent-driven experiments from main, evaluate them on a fixed test set, and promote only the winner."
|
|
icon: "flask"
|
|
keywords: ["AI agents", "branches", "experiments", "embeddings", "evaluation"]
|
|
---
|
|
|
|
LanceDB supports [branching](/tables/branching) to isolate experiments from the main table when working with agents.
|
|
Branches are useful when you instruct an agent to try several approaches without
|
|
affecting the production data on the `main` branch.
|
|
Each experiment gets its own writable table history.
|
|
|
|
For example, you can generate and compare embeddings from two text embedding
|
|
models on the given table:
|
|
|
|
| Branch | Model | Added column |
|
|
| --- | --- | --- |
|
|
| `embed-minilm` | `sentence-transformers/all-MiniLM-L6-v2` | `vector_minilm` |
|
|
| `embed-nomic` | `nomic-embed-text` from Ollama | `vector_nomic` |
|
|
|
|
The branches share a logical table, but each has its own schema and history.
|
|
The original table handle still points to `main`. LanceDB does not have a
|
|
process-wide current branch.
|
|
|
|
Install the model packages, then download the `nomic-embed-text` model from Ollama
|
|
(or any other model you prefer):
|
|
|
|
<CodeGroup>
|
|
```bash Bash icon="terminal"
|
|
uv add sentence-transformers ollama
|
|
ollama pull nomic-embed-text
|
|
```
|
|
</CodeGroup>
|
|
|
|
It's always recommended to measure the results of an experiment on a fixed evaluation set.
|
|
Here's an example of how you could ask the agent to run both experiments and compare the results:
|
|
|
|
```text Agent prompt
|
|
Use the lancedb plugin and the LanceDB branching documentation to compare two
|
|
embedding experiments on the `frames` table from the quickstart.
|
|
|
|
1. Fork `embed-minilm` from `main`. Build normalized embeddings from
|
|
`caption` with `sentence-transformers/all-MiniLM-L6-v2` and add them as
|
|
`vector_minilm`.
|
|
2. Fork `embed-nomic` from `main`. Embed the identical text with Ollama
|
|
`nomic-embed-text` and add the results as `vector_nomic`.
|
|
3. Process rows in batches and write only through each branch-scoped table
|
|
handle. Verify that neither new column appears on `main`.
|
|
4. Evaluate both branches with the queries "a gripper closing on an object",
|
|
"a robot arm holding steady", and "lowering an object onto a surface".
|
|
Report the top match, latency, and whether the expected frame ranks first.
|
|
5. Do not modify `main` and do not delete either branch. Recommend a winner
|
|
based on the results.
|
|
```
|
|
|
|
<Note>
|
|
The vector dimensions may differ because the columns live on separate branches.
|
|
Use a different column name for each model. Within an experiment, use the same
|
|
model for the stored text and the search queries.
|
|
</Note>
|
|
|
|
## Apply the winning experiment to `main` {#apply-the-winning-experiment-to-main}
|
|
|
|
The branch experiments leave you with the results side by side, and they
|
|
deliberately never touch `main`. Once you've picked a winner, apply it to
|
|
`main` yourself by rerunning that experiment's validated transformation
|
|
directly against the `main` table. Rerunning the reviewed operation is the
|
|
reliable path on both LanceDB Embedded and LanceDB Enterprise: it replays exactly the transformation
|
|
you validated on the branch, and it works the same regardless of how the branch
|
|
evolved.
|
|
|
|
Suppose `nomic-embed-text` wins. Rerun the same Nomic embedding transformation
|
|
you validated on `embed-nomic` against `main` in batches, then call
|
|
`table.optimize()` on LanceDB Embedded. Verify a bounded vector search on `main`, and keep
|
|
the losing branch around until you're confident in the result.
|
|
|
|
See [Branches](/tables/branching) for more on how branches, versions, and tags
|
|
relate.
|
|
|
|
## More experiments you can run {#more-experiments-you-can-run}
|
|
|
|
Swapping embedding models on a branch while working with agents is only one example
|
|
of what you can do with the LanceDB plugin. The table below shows other experiments
|
|
you can run with the plugin.
|
|
|
|
| Hypothesis | Change on the branch | What to evaluate |
|
|
| --- | --- | --- |
|
|
| A different embedding model improves retrieval | Add a new vector column | Recall, ranking quality, latency, and cost |
|
|
| A new parser or OCR model improves source data | Reprocess files and backfill metadata | Validation failures and a reviewed sample |
|
|
| A new search setup works better | Build an index or change hybrid-search and reranking settings | Relevance and latency on a fixed query set |
|
|
| A curation rule improves the corpus | Deduplicate, classify, or filter records | False positives, false negatives, and row counts |
|
|
| A migration is safe to deploy | Add columns or run a backfill | Schema compatibility and application checks |
|
|
|
|
For each experiment, ask the agent to state the hypothesis, use a fixed
|
|
evaluation set, report the comparison, and, if you want, work on an alternate
|
|
branch that's isolated from `main` until you decide you want the new derived
|
|
column in `main`.
|
|
|
|
Once you've chosen a winner, rerun that validated transformation to ingest it into `main`
|
|
yourself, on either LanceDB Embedded or LanceDB Enterprise. See our documentation on [branches](/tables/branching)
|
|
for more information.
|