mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 14:52:26 +00:00
The TypeScript connection example only defined a helper that nothing called, so every later step failed with `db is not defined`. It now connects, as the Python one does. The Rust one kept its connection inside a helper, the Enterprise one read its settings without connecting, and the later steps relied on imports and a record-batch helper the page never showed. The quickstart now says how the steps make one program: an ES module run with Node in TypeScript, and the body of an async `main` in Rust, whose dependencies the install step lists. Each Rust step brings its own imports, the data step shows the helper that turns the records into a reader, and Arrow comes from lancedb's re-exports, so no Arrow crate is needed. The row structs no longer derive serde traits nothing used. The connection examples are shared, so the connect, create-table and branching pages gain the import line their connection code now needs.
598 lines
20 KiB
Plaintext
598 lines
20 KiB
Plaintext
---
|
|
title: Quickstart
|
|
sidebarTitle: "Quickstart"
|
|
description: "Get started with LanceDB in minutes."
|
|
icon: rocket
|
|
---
|
|
import {
|
|
PyConnect,
|
|
PyConnectAsync,
|
|
PyConnectEnterpriseQuickstart,
|
|
PyConnectObjectStorage,
|
|
PyConnectObjectStorageAsync,
|
|
RsConnect,
|
|
RsConnectEnterpriseQuickstart,
|
|
RsConnectObjectStorage,
|
|
TsConnect,
|
|
TsConnectEnterpriseQuickstart,
|
|
TsConnectObjectStorage,
|
|
} from '/snippets/connection.mdx';
|
|
import {
|
|
PyQuickstartData,
|
|
PyQuickstartAddData,
|
|
PyQuickstartComputedFeature,
|
|
PyQuickstartComputedFeatureEnterprise,
|
|
PyQuickstartTraining,
|
|
PyQuickstartCreateTable,
|
|
PyQuickstartCreateTableAsync,
|
|
PyQuickstartAddFeature,
|
|
PyQuickstartCurateWithMetadata,
|
|
PyQuickstartMultimodalBytes,
|
|
PyQuickstartQueryFeature,
|
|
PyQuickstartVectorSearch1,
|
|
PyQuickstartVectorSearch1Async,
|
|
PyQuickstartOutputPandas,
|
|
RsQuickstartAddData,
|
|
RsQuickstartAddFeature,
|
|
RsQuickstartCurateWithMetadata,
|
|
RsQuickstartCreateTable,
|
|
RsQuickstartData,
|
|
RsQuickstartDefineStruct,
|
|
RsQuickstartMultimodalBytes,
|
|
RsQuickstartQueryFeature,
|
|
RsQuickstartVectorSearch1,
|
|
TsQuickstartAddData,
|
|
TsQuickstartAddFeature,
|
|
TsQuickstartCurateWithMetadata,
|
|
TsQuickstartCreateTable,
|
|
TsQuickstartData,
|
|
TsQuickstartMultimodalBytes,
|
|
TsQuickstartQueryFeature,
|
|
TsQuickstartVectorSearch1,
|
|
} from '/snippets/quickstart.mdx';
|
|
|
|
As described in [the landing page](/), LanceDB provides one data layer for
|
|
curation, feature engineering, search and retrieval, and model training. Whether you are preparing
|
|
training data, building a RAG or agentic retrieval system, reviewing examples, or adding model-generated
|
|
features, you'll work with the same underlying table and search primitives.
|
|
|
|
Let's get started in just a few steps!
|
|
|
|
## 1\. Install LanceDB {#1-install-lancedb}
|
|
|
|
Install LanceDB in your client SDK.
|
|
|
|
<CodeGroup>
|
|
```bash pip icon="terminal"
|
|
pip install lancedb
|
|
```
|
|
|
|
```bash uv icon="terminal"
|
|
uv add lancedb
|
|
|
|
# Or, in an existing virtual environment:
|
|
uv pip install lancedb
|
|
```
|
|
|
|
```bash TypeScript icon=js
|
|
npm install @lancedb/lancedb
|
|
```
|
|
|
|
```bash Rust icon=Rust
|
|
cargo add lancedb --features polars,remote
|
|
cargo add tokio --features macros,rt-multi-thread
|
|
```
|
|
</CodeGroup>
|
|
|
|
The TypeScript and Rust code of steps 2 to 5 makes one program. In TypeScript,
|
|
put it, in order, in an ES module such as `quickstart.mts` and run
|
|
`node quickstart.mts`; Node.js 22.18 and later run TypeScript directly. In Rust,
|
|
it goes, in order, inside `main` in `src/main.rs`, and `cargo run` runs it. The
|
|
`polars` feature returns results as DataFrames, and `remote` connects to
|
|
LanceDB Enterprise.
|
|
|
|
```rust src/main.rs icon="rust"
|
|
#[tokio::main]
|
|
async fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// The Rust code of steps 2 to 5 goes here.
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
### Python pre-release builds {#python-pre-release-builds}
|
|
|
|
To pick up the latest features and bug fixes
|
|
before the next stable release, install a pre-release from LanceDB's Fury index.
|
|
|
|
<CodeGroup>
|
|
```bash pip icon="terminal"
|
|
pip install --pre --extra-index-url https://pypi.fury.io/lancedb/ lancedb
|
|
```
|
|
|
|
```bash uv icon="terminal"
|
|
uv venv
|
|
uv pip install --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb
|
|
|
|
# To add to pyproject.toml, use:
|
|
uv add --prerelease allow --index https://pypi.fury.io/lancedb/ lancedb
|
|
```
|
|
</CodeGroup>
|
|
|
|
<Note>
|
|
Pre-release builds receive the same level of testing as stable releases, but their availability is not guaranteed
|
|
for more than 6 months after release. For real-world workloads, we recommend you use the latest stable release
|
|
as far as possible.
|
|
</Note>
|
|
|
|
<Accordion title="Running on older x86_64 CPUs without AVX2?">
|
|
The default `lancedb` wheel targets `x86-64-haswell` (AVX2 + FMA + F16C). On older x86_64 CPUs without AVX2
|
|
(Intel Sandy Bridge, Ivy Bridge, Westmere; AMD Bulldozer, Piledriver, Steamroller), `import lancedb` crashes
|
|
with `Illegal instruction`.
|
|
|
|
Install `lancedb-compat` instead. It exposes the same API (`import lancedb` still works), is built at the
|
|
`x86-64-v2` baseline, and uses runtime SIMD dispatch to still leverage AVX2, FMA, or AVX-512 when available:
|
|
|
|
```bash pip icon="terminal"
|
|
pip install lancedb-compat
|
|
```
|
|
|
|
The two packages share the same `lancedb/` namespace and conflict at install time, so pick one. To switch,
|
|
uninstall the other first (`pip uninstall lancedb && pip install lancedb-compat`).
|
|
</Accordion>
|
|
|
|
## 2\. Connect to a LanceDB database {#2-connect-to-a-lancedb-database}
|
|
|
|
LanceDB supports several URI patterns to connect to a database.
|
|
|
|
- A local filesystem path (when using it as an embedded library)
|
|
- A `db://...` URI (when using LanceDB Enterprise)
|
|
- An object storage URI: `s3://...`, `gs://...`, or `az://...` (when connecting directly from the client SDK)
|
|
|
|
Pick one. Steps 3 to 5 then run unchanged, whichever you picked.
|
|
|
|
### Connect via local directory path {#connect-via-local-directory-path}
|
|
|
|
The simplest way to begin is to use LanceDB as an embedded library. Import LanceDB in your
|
|
client SDK of choice and point to a local directory path.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (sync)" language="Python" icon="python">
|
|
{PyConnect}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Python (async)" language="Python" icon="python">
|
|
{PyConnectAsync}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{ "import * as lancedb from \"@lancedb/lancedb\";\n\n" }
|
|
{TsConnect}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{ "use lancedb::connect;\n" }
|
|
{ "\n" }
|
|
{RsConnect}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
|
|
### Connect via object storage URIs {#connect-via-object-storage-uris}
|
|
|
|
You can also connect directly to object storage from the client SDK:
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (sync)" language="Python" icon="python">
|
|
{PyConnectObjectStorage}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Python (async)" language="Python" icon="python">
|
|
{PyConnectObjectStorageAsync}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsConnectObjectStorage}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsConnectObjectStorage}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
For credentials, endpoints, and provider-specific options, see
|
|
[Configuring storage](/deployment/storage).
|
|
|
|
### Connect to LanceDB Enterprise {#connect-to-lancedb-enterprise}
|
|
|
|
Connecting to a deployment takes four settings, which its administrator gives
|
|
you -- or the LanceDB team, for a deployment they run for you. If you don't have
|
|
them, [contact the LanceDB team](https://www.lancedb.com/contact). Put them in
|
|
your environment:
|
|
|
|
| Variable | Holds |
|
|
| --- | --- |
|
|
| `LANCEDB_URI` | the database's `db://` URI |
|
|
| `LANCEDB_HOST_OVERRIDE` | the deployment's endpoint: its full URL, with `https://` and any non-default port |
|
|
| `LANCEDB_API_KEY` | an API key for the deployment |
|
|
| `LANCEDB_REGION` | its region, if it needs one; `us-east-1` otherwise |
|
|
|
|
The client reads them from there, so no credential appears in your code. The
|
|
endpoint is required: without it the code stops rather than guessing one.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyConnectEnterpriseQuickstart}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{ "import * as lancedb from \"@lancedb/lancedb\";\n\n" }
|
|
{TsConnectEnterpriseQuickstart}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{ "use lancedb::connect;\n\n" }
|
|
{RsConnectEnterpriseQuickstart}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
A deployment runs some things differently -- maintenance,
|
|
long-running work, a few unsupported methods; see
|
|
[Moving to LanceDB Enterprise](/basics/sdk#moving-to-enterprise), and the
|
|
[Enterprise overview](/enterprise) for how a deployment is put together.
|
|
|
|
## 3\. Create a table and add data {#3-create-a-new-table}
|
|
|
|
Let's create a small table of characters from the kingdom of Camelot. Each row stores source text,
|
|
metadata, structured fields, and a vector embedding in the same LanceDB table.
|
|
|
|
<Info>
|
|
The embeddings we use in this example are synthetic and for demonstration purposes only. In a real AI
|
|
data workflow, you would generate them from text, images, audio, or video using an embedding model of choice.
|
|
</Info>
|
|
|
|
Each row has source text, metadata, structured fields, and a vector:
|
|
|
|
```json
|
|
{
|
|
"id": "2",
|
|
"name": "Merlin",
|
|
"role": "Wizard",
|
|
"description": "Advisor and prophet with deep magical knowledge.",
|
|
"stats": { "strength": 2, "magic": 5, "leadership": 4, "wisdom": 5 },
|
|
"vector": [0.2, 0.9, 0.4, 0.9]
|
|
}
|
|
```
|
|
|
|
The full raw records are included below:
|
|
|
|
<Accordion title="View raw character records">
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (sync)" language="Python" icon="python">
|
|
{PyQuickstartData}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartData}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartDefineStruct}
|
|
{RsQuickstartData}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
</Accordion>
|
|
|
|
You can now create a LanceDB table from those records. The code below creates a LanceDB table
|
|
with the appropriate schema and ingests the data.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (sync)" language="Python" icon="python">
|
|
{PyQuickstartCreateTable}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Python (async)" language="Python" icon="python">
|
|
{PyQuickstartCreateTableAsync}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartCreateTable}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartCreateTable}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
A table grows by adding rows to it. Add a fourth character:
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartAddData}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartAddData}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartAddData}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
## 4\. Semantic search {#4-semantic-search}
|
|
|
|
Search is a useful capability for all kinds of AI data pipelines. Below, we do a vector similarity
|
|
search for samples similar to a "_wise magical advisor_" (transforming the natural language query to
|
|
an embedding), and project only the columns needed by the next step.
|
|
|
|
Search (which requires random access) is a ubiquitous access pattern that appears in many workloads:
|
|
whether you're building a RAG or recommendation system, serving agent memory, or curating a training
|
|
dataset.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (sync)" language="Python" icon="python">
|
|
{PyQuickstartVectorSearch1}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Python (async)" language="Python" icon="python">
|
|
{PyQuickstartVectorSearch1Async}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartVectorSearch1}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartVectorSearch1}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
Merlin and Morgana, the two magic users, come back first. The example for Python above
|
|
converts the results to a Polars DataFrame. Depending on your language, you can collect
|
|
query results as a list/array of objects or DataFrames to be used downstream in your
|
|
application.
|
|
|
|
<Accordion title="Pandas users in Python can get results as a Pandas DataFrame">
|
|
Use the `to_pandas()` method to convert query results into a Pandas DataFrame.
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartOutputPandas}
|
|
</CodeBlock>
|
|
</Accordion>
|
|
|
|
## 5\. Curation {#5-curation}
|
|
|
|
Searching for relevant results can be more useful when combined with metadata filters.
|
|
In this tiny example, we filter to examples with high `magic` stats.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartCurateWithMetadata}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartCurateWithMetadata}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartCurateWithMetadata}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
When working with large datasets, it's common to use the same pattern to filter on quality labels,
|
|
train/eval splits, numeric fields, categorical values, timestamp windows, or generated tags and labels.
|
|
|
|
That is the core of working with LanceDB: connect, create, add, search, filter. The code
|
|
in steps 3 to 5 is the same whether you connected to a local directory, object storage or a
|
|
deployment.
|
|
|
|
## Optional extensions {#extensions}
|
|
|
|
Each extension below starts from the `characters` table of step 3 and stands on its own,
|
|
so take any of them, in any order, or none. In TypeScript and Rust, its code goes after
|
|
step 5's.
|
|
|
|
### Add a derived feature {#6-add-a-derived-feature}
|
|
|
|
Feature engineering is the process of cleaning up your data and creating new signals that
|
|
help your model learn, make better predictions, or your agent retrieve more useful information.
|
|
In the example below, we add a `power_score` column from the structured `stats` fields.
|
|
Lance supports data evolution, so you can add new columns without rewriting the entire table.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartAddFeature}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartAddFeature}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{ "use lancedb::table::NewColumnTransform;\n\n" }
|
|
{RsQuickstartAddFeature}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
Next, you can query a compact view of the new feature:
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartQueryFeature}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartQueryFeature}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartQueryFeature}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
| name | role | power_score |
|
|
| --- | --- | --- |
|
|
| King Arthur | King | 3.5 |
|
|
| Merlin | Wizard | 4.0 |
|
|
| Sir Lancelot | Knight | 3.0 |
|
|
| Morgana | Sorceress | 3.75 |
|
|
|
|
The same workflow is used for data preparation tasks when adding derived features, cached model signals, review scores, or dataset
|
|
quality indicators.
|
|
|
|
### Declare now, compute later {#computed-columns}
|
|
|
|
`add_columns` above computes every row as part of the call, which is fine for
|
|
four rows. On a table large enough that you would rather choose *when* the work
|
|
happens, declare a column instead and fill it separately, with a refresh. The
|
|
declaration is the same on every connection; the refresh is not:
|
|
|
|
- Embedded, `refresh_column` fills the column in your process.
|
|
- On LanceDB Enterprise, a refresh runs on the server as a [job](/basics/jobs):
|
|
`refresh_column` raises there, and `refresh_column_async` returns the job to
|
|
wait for. Computed columns are also off by default on a deployment -- its
|
|
administrator has to enable them before you can declare one.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python (embedded)" language="Python" icon="python">
|
|
{PyQuickstartComputedFeature}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Python (Enterprise)" language="Python" icon="python">
|
|
{PyQuickstartComputedFeatureEnterprise}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
A refresh gives values to rows that have none and recomputes only rows whose
|
|
inputs changed, so adding more characters and refreshing again fills just the
|
|
new ones -- it does not recompute the rows that are already current. That is
|
|
what makes a backfill safe to retry and cheap to repeat as data arrives.
|
|
|
|
See [Computed columns](/feature-engineering/add-computed-column) for
|
|
expressions, including what a deployment can compute.
|
|
See [Functions](/management/function) for filling a column with Python. {/* if functions launched */}
|
|
|
|
### Store multimodal data {#7-store-multimodal-data}
|
|
|
|
Multimodal data is a first-class citizen in LanceDB. Binary data (image, audio, video, etc.) is
|
|
stored as blobs or inline Arrow binary types in a LanceDB column, and they benefit from the same
|
|
table operations and data versioning semantics as other data types. All the data is governed
|
|
in the same table, so you can search, filter, and retrieve multimodal records together with structured
|
|
fields, metadata, and embeddings.
|
|
|
|
In this example, the
|
|
[`lancedb/magical_kingdom`](https://huggingface.co/datasets/lancedb/magical_kingdom) dataset stores
|
|
character images, descriptions, structured stats, image embeddings, and text embeddings together.
|
|
This step stores its portrait of Sir Lancelot:
|
|
|
|
<div style={{ display: "flex", justifyContent: "center" }}>
|
|
<img
|
|
src="/static/assets/images/quickstart/sir-lancelot.jpg"
|
|
alt="Sir Lancelot from the lancedb/magical_kingdom dataset"
|
|
width="320"
|
|
/>
|
|
</div>
|
|
|
|
From your application's directory, download the image into a `data` folder there:
|
|
|
|
```bash Terminal icon="terminal"
|
|
curl --create-dirs -L -o data/sir-lancelot.jpg https://docs.lancedb.com/static/assets/images/quickstart/sir-lancelot.jpg
|
|
```
|
|
|
|
Run the code below from that same directory: it reads `data/sir-lancelot.jpg`
|
|
relative to where it runs, and stores the bytes in an `image` column. The image
|
|
bytes can be used for downstream tasks like retrieval, evaluation, or training.
|
|
|
|
<CodeGroup >
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartMultimodalBytes}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
|
{TsQuickstartMultimodalBytes}
|
|
</CodeBlock>
|
|
|
|
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
|
{RsQuickstartMultimodalBytes}
|
|
</CodeBlock>
|
|
</CodeGroup >
|
|
|
|
For more examples, see [Data types](/management/table#multimodal).
|
|
|
|
### Train on the result {#8-train}
|
|
|
|
The table you searched and filtered is directly usable as a PyTorch
|
|
[Dataset](https://docs.pytorch.org/docs/stable/data.html#torch.utils.data.Dataset)
|
|
-- there is no export step, and no second copy of the data to keep in sync.
|
|
|
|
Install PyTorch with `pip install torch`. A model trains on numeric features,
|
|
so this step derives two from the structured `stats` field, then selects just
|
|
those columns for the loader: the full table also contains strings, structs,
|
|
and vectors.
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{PyQuickstartTraining}
|
|
</CodeBlock>
|
|
|
|
```python
|
|
for batch in dataloader:
|
|
... # train with batch["magic"] and batch["strength"]
|
|
```
|
|
|
|
Because the loader reads from the table, the feature columns you
|
|
materialized are read straight off storage rather than recomputed every epoch.
|
|
That is the whole argument for doing feature engineering in the lakehouse: an
|
|
expensive transform runs once, is versioned with the data, and every subsequent
|
|
epoch is a read. Selecting only the columns a step needs keeps the descriptions,
|
|
stats, and vectors from being read at all.
|
|
|
|
See [Training](/training/data-loading) for shuffling, multi-worker loading, and reading from
|
|
a remote table.
|
|
|
|
## Code {#code}
|
|
|
|
See the full code for these examples (including helper functions) in the
|
|
`quickstart` file for the appropriate client language in the
|
|
[tests they come from](https://github.com/lancedb/lancedb/tree/main/docs/web-tests).
|
|
|
|
|
|
## Next steps {#next-steps}
|
|
You've created a table, added to it, and searched and filtered it -- and, through the
|
|
extensions, derived features from it and trained on it -- in one table, with no export
|
|
step between any two stages. That arc is the argument for a multimodal lakehouse, and
|
|
every stage of it scales past this toy example without changing shape.
|
|
|
|
Continue to the guides to build on it with schema options, versioning, indexing,
|
|
full-text and hybrid search, reranking, and server-side feature engineering.
|
|
|
|
<Columns cols={2}>
|
|
<Card
|
|
title="Basic table operations"
|
|
icon="table"
|
|
href="/management/table"
|
|
>
|
|
Build on this quickstart with table creation, updates, and schema tips.
|
|
</Card>
|
|
<Card
|
|
title="Build a RAG App"
|
|
icon="search"
|
|
href="/tutorials/agents/contextual-rag"
|
|
>
|
|
Learn how to build Retrieval-Augmented Generation (RAG) applications using LanceDB.
|
|
</Card>
|
|
<Card
|
|
title="Indexing"
|
|
icon="layer-group"
|
|
href="/indexing/"
|
|
>
|
|
Create vector, full-text, and scalar indexes to speed up queries on larger datasets.
|
|
</Card>
|
|
<Card
|
|
title="Data loading and shuffles"
|
|
icon="boxes-stacked"
|
|
href="/training/data-loading"
|
|
>
|
|
Use LanceDB for projected, shuffled, random-access reads in training workflows.
|
|
</Card>
|
|
</Columns>
|