Files
lancedb/docs/web/embedding/quickstart.mdx
T
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00

156 lines
5.5 KiB
Plaintext

---
title: "Embeddings: Quickstart"
sidebarTitle: "Quickstart"
description: "Quickstart guide for generating and working with embeddings."
icon: "rocket"
---
import {
TsQuickstartConnect,
TsQuickstartCreateTable,
TsQuickstartImports,
TsQuickstartInitModel,
TsQuickstartQuery,
TsQuickstartSchema,
} from '/snippets/sentence-transformers.mdx';
LanceDB will automatically vectorize the data both at ingestion and query time. All you need to do is specify which model to use.
Popular embedding models like OpenAI, Hugging Face, Sentence Transformers, CLIP, and more, are supported.
## Step 1: Import Required Libraries {#step-1-import-required-libraries}
First, import the necessary LanceDB components:
<CodeGroup>
```python Python icon="python"
import lancedb
from lancedb.pydantic import LanceModel, Vector
from lancedb.embeddings import get_registry
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartImports}
</CodeBlock>
</CodeGroup>
- `lancedb`: The main database connection and operations
- `LanceModel`: Pydantic model for defining table schemas
- `Vector`: Field type for storing vector embeddings
- `get_registry()`: Access to the embedding function registry. It has all the supported as well as custom embedding functions registered by the user
- TypeScript uses `lancedb.embedding.getRegistry()` and `lancedb.embedding.LanceSchema()` for the same registry/schema workflow
- In TypeScript, import the provider module before calling `getRegistry().get(...)`; the provider import is what registers names such as `"huggingface"` or `"openai"`
## Step 2: Connect to LanceDB {#step-2-connect-to-lancedb}
Establish a connection to your LanceDB OSS directory or Enterprise cluster:
<CodeGroup>
```python Python icon="python"
# Enter your LanceDB connection URI for OSS or Enterprise here
db = lancedb.connect(...)
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartConnect}
</CodeBlock>
</CodeGroup>
## Step 3: Initialize the Embedding Function {#step-3-initialize-the-embedding-function}
Choose and configure your embedding model:
<CodeGroup>
```python Python icon="python"
model = get_registry().get("sentence-transformers").create(name="BAAI/bge-small-en-v1.5", )
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartInitModel}
</CodeBlock>
</CodeGroup>
This creates an embedding function from the local embedding registry. The Python snippet uses the
`sentence-transformers` provider with the BGE model; the TypeScript snippet uses the Transformers-backed
`huggingface` provider. You can:
- Change `"sentence-transformers"` to other providers like `"openai"`, `"cohere"`, etc.
- Modify the model name for different embedding models
- Set `device="cuda"` for GPU acceleration if available
## Step 4: Define Your Schema {#step-4-define-your-schema}
Create a Pydantic model that defines your table structure:
<CodeGroup>
```python Python icon="python"
class Words(LanceModel):
text: str = model.SourceField()
vector: Vector(model.ndims()) = model.VectorField()
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartSchema}
</CodeBlock>
</CodeGroup>
- `SourceField()`: This field will be embedded
- `VectorField()`: This stores the embeddings
- `model.ndims()`: Sets vector dimensions for your model
- In TypeScript, use `model.sourceField(...)` and `model.vectorField()` inside `LanceSchema(...)`
## Step 5: Create Table and Ingest Data {#step-5-create-table-and-ingest-data}
Create a table with your schema and add data:
<CodeGroup>
```python Python icon="python"
table = db.create_table("words", schema=Words)
table.add([
{"text": "hello world"},
{"text": "goodbye world"}
])
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartCreateTable}
</CodeBlock>
</CodeGroup>
The `table.add()` call automatically:
- Takes the text from each document
- Generates embeddings using your chosen model
- Stores both the original text and the vector embeddings
If your input already includes the vector column, automatic embedding only runs when that column is
absent or entirely null for the batch. Partially supplied vectors are treated as manual data, so
LanceDB preserves them instead of filling only the missing rows.
## Step 6: Query with Automatic Embedding {#step-6-query-with-automatic-embedding}
Note: On LanceDB Enterprise, the server does not generate embeddings from query text. In the Python remote
client, `table.search("greetings")` can still work when the table schema includes embedding metadata, because
the client computes the query embedding before sending the vector search. If there is no embedding metadata for
that search, `search("greetings")` in `auto` mode is treated as FTS instead.
Search your data using natural language queries:
<CodeGroup>
```python Python icon="python"
query = "greetings"
actual = table.search(query).limit(1).to_pydantic(Words)[0]
print(actual.text)
```
<CodeBlock filename="TypeScript" language="typescript" icon="square-js">
{TsQuickstartQuery}
</CodeBlock>
</CodeGroup>
The search process:
1. Automatically converts your query text to embeddings
2. Finds the most similar vectors in your table
3. Returns the matching documents
Automatic text search depends on the table's embedding metadata. If the client cannot reconstruct an
embedding function from that metadata, compute the query embedding yourself and search with the
vector directly.