mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 07:12:27 +00:00
docs: fold Multimodal Data into the Table page, and repair stale links
Multimodal Data becomes a section of Table, between Data types -- which introduces blobs -- and Schema evolution. The rest is link repair. Earlier merges left links pointing at pages that had moved, which the redirect table covered but the link checker does not follow, so they were broken in the rendered site: /catalog, /lance, /namespaces, /tables-and-namespaces, /search/multivector-search, /search/explain-analyze and /basics/data-types#blob. The redirect table itself had two conflicting entries for /tables/multimodal and four pointing at reranking pages that no longer exist; both are fixed.
This commit is contained in:
Jack Ye
@@ -46,7 +46,7 @@ export LANCEDB_DATABASE="your-database-name"
|
||||
4. Check that you can reach the deployment and list tables:
|
||||
|
||||
```bash
|
||||
curl -X GET "$LANCEDB_URI/v1/tables" \
|
||||
curl -X GET "$LANCEDB_URI/v1/management/table" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "x-api-key: $LANCEDB_API_KEY" \
|
||||
-H "x-lancedb-database: $LANCEDB_DATABASE"
|
||||
@@ -66,7 +66,7 @@ curl -X POST "$LANCEDB_URI/v1/tables/words" \
|
||||
6. Check that the table has been created:
|
||||
|
||||
```bash
|
||||
curl -X GET "$LANCEDB_URI/v1/tables" \
|
||||
curl -X GET "$LANCEDB_URI/v1/management/table" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "x-api-key: $LANCEDB_API_KEY" \
|
||||
-H "x-lancedb-database: $LANCEDB_DATABASE"
|
||||
|
||||
+1
-2
@@ -73,7 +73,7 @@
|
||||
},
|
||||
{
|
||||
"source": "/tables/multimodal",
|
||||
"destination": "/basics/data-types#blob"
|
||||
"destination": "/management/table#multimodal"
|
||||
},
|
||||
{
|
||||
"source": "/search/select",
|
||||
@@ -220,7 +220,6 @@
|
||||
"management/database",
|
||||
"management/namespace",
|
||||
"management/table",
|
||||
"tables/multimodal",
|
||||
"tables/branching",
|
||||
"management/materialized-view",
|
||||
"management/function"
|
||||
|
||||
+1
-1
@@ -8,7 +8,7 @@ keywords: ["multimodal lakehouse", "training", "feature engineering", "search",
|
||||
|
||||
**LanceDB** is a [multimodal lakehouse](https://lancedb.com/blog/multimodal-lakehouse/) for AI teams that need
|
||||
one data layer for curation, feature engineering, search and retrieval, and model training.
|
||||
It is built on top of [Lance](/lance), an open-source lakehouse format designed for multimodal AI data.
|
||||
It is built on top of [Lance](/basics/lance), an open-source lakehouse format designed for multimodal AI data.
|
||||
|
||||
Move from data exploration to model training on one, unified platform without needing to manage a
|
||||
fragmented stack of storage, feature, retrieval, and training systems.
|
||||
|
||||
@@ -159,7 +159,7 @@ println!("Created table: {}", table.name());
|
||||
|
||||
The example above showed local directory-based namespaces. LanceDB also supports namespaces backed by remote object stores and external catalogs, via the REST namespace implementation.
|
||||
|
||||
For remote object stores with central metadata/catalog services (either commercial or open source),
|
||||
For remote object stores with central metadata/basics/catalog services (either commercial or open source),
|
||||
use the REST namespace implementation. It is backed by REST routes
|
||||
(for example `POST /v1/namespace/{id}/create` and `GET /v1/namespace/{id}/list`) and server-provided table locations.
|
||||
|
||||
|
||||
@@ -84,6 +84,34 @@ import {
|
||||
RsVersioningTags as RsVersioningTags,
|
||||
RsVersioningMakeQuotesReader as RsVersioningMakeQuotesReader,
|
||||
} from '/snippets/tables.mdx';
|
||||
import {
|
||||
PyMultimodalImports as MultimodalImports,
|
||||
TsMultimodalImports as TsMultimodalImports,
|
||||
RsMultimodalImports as RsMultimodalImports,
|
||||
PyCreateDummyData as CreateDummyData,
|
||||
TsCreateDummyData as TsCreateDummyData,
|
||||
RsCreateDummyData as RsCreateDummyData,
|
||||
PyDefineSchema as DefineSchema,
|
||||
TsDefineSchema as TsDefineSchema,
|
||||
RsDefineSchema as RsDefineSchema,
|
||||
PyIngestData as IngestData,
|
||||
TsIngestData as TsIngestData,
|
||||
RsIngestData as RsIngestData,
|
||||
PySearchData as SearchData,
|
||||
TsSearchData as TsSearchData,
|
||||
RsSearchData as RsSearchData,
|
||||
PyProcessResults as ProcessResults,
|
||||
TsProcessResults as TsProcessResults,
|
||||
RsProcessResults as RsProcessResults,
|
||||
PyBlobApiSchema as BlobApiSchema,
|
||||
TsBlobApiSchema as TsBlobApiSchema,
|
||||
RsBlobApiSchema as RsBlobApiSchema,
|
||||
PyBlobApiIngest as BlobApiIngest,
|
||||
TsBlobApiIngest as TsBlobApiIngest,
|
||||
RsBlobApiIngest as RsBlobApiIngest,
|
||||
PyBlobApiToPandas as BlobApiToPandas,
|
||||
PyQueryToPandasKwargs as QueryToPandasKwargs,
|
||||
} from '/snippets/multimodal.mdx';
|
||||
|
||||
A table is rows and columns, versioned. It is what searches read, what writes
|
||||
commit to, and what indexes and branches belong to.
|
||||
@@ -233,7 +261,7 @@ as it is to the format.
|
||||
| String and binary | `utf8`, `large_utf8`, `binary`, `large_binary`, `fixed_size_binary(n)` |
|
||||
| Date and time | `date32`, `date64`, `time32`, `time64`, `timestamp`, `duration` |
|
||||
| Nested | `struct`, `list`, `large_list`, `fixed_size_list`, `map` |
|
||||
| Lance extensions | vectors, [JSON](/sql/data-types#json), [blobs](/tables/multimodal), bfloat16 |
|
||||
| Lance extensions | vectors, [JSON](/sql/data-types#json), [blobs](#multimodal), bfloat16 |
|
||||
|
||||
A vector column is a `fixed_size_list<float32, dim>` -- an ordinary Arrow type,
|
||||
which is why it is selectable and joinable but why arithmetic over it is list
|
||||
@@ -242,6 +270,219 @@ arithmetic rather than scalar.
|
||||
See [SQL data types](/sql/data-types) for the full reference, including the
|
||||
extension types and pseudo-columns.
|
||||
|
||||
## Multimodal data {#multimodal}
|
||||
|
||||
Binary data -- images, audio, video, model artifacts -- lives in the table
|
||||
beside the structured columns and embeddings that describe it, and is governed
|
||||
by the same versioning and search.
|
||||
|
||||
Small values go in an ordinary `binary` column. Large ones go in a **blob**
|
||||
column, stored out of line so a scan over the other columns stays fast. See
|
||||
[Data types](/sql/data-types#blobs) for how the two differ to a reader.
|
||||
|
||||
### Store binary data {#multimodal-store-binary-data}
|
||||
|
||||
To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust).
|
||||
|
||||
#### 1\. Setup and imports {#multimodal-1-setup-and-imports}
|
||||
|
||||
First, import the necessary libraries for LanceDB and Arrow in your SDK.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{MultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsMultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsMultimodalImports}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 2\. Prepare data {#multimodal-2-prepare-data}
|
||||
|
||||
For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{CreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsCreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsCreateDummyData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 3\. Define the schema {#multimodal-3-define-the-schema}
|
||||
|
||||
When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{DefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsDefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsDefineSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 4\. Ingest data {#multimodal-4-ingest-data}
|
||||
|
||||
Now, create the table using the data and the defined schema.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{IngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsIngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsIngestData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Retrieve and use blobs {#multimodal-retrieve-and-use-blobs}
|
||||
|
||||
When you search your LanceDB table, you can retrieve the binary column just like any other metadata.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{SearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsSearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsSearchData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### Convert bytes back to objects {#multimodal-convert-bytes-back-to-objects}
|
||||
|
||||
Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer).
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{ProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsProcessResults}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Large Blobs (Blob API) {#multimodal-large-blobs-blob-api}
|
||||
|
||||
For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront.
|
||||
|
||||
#### 1\. Define a blob schema {#multimodal-1-define-a-blob-schema}
|
||||
|
||||
To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 2\. Ingest large blobs {#multimodal-2-ingest-large-blobs}
|
||||
|
||||
You can then ingest data normally, and Lance will handle the optimized storage.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
<Card>
|
||||
For more advanced usage, including random access and file-like reading of blobs, see the
|
||||
Lance format's [blob API documentation](https://lance.org/guide/blob/).
|
||||
</Card>
|
||||
|
||||
#### 3\. Convert blob tables to pandas {#multimodal-3-convert-blob-tables-to-pandas}
|
||||
|
||||
When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`.
|
||||
|
||||
`blob_mode` accepts:
|
||||
|
||||
- `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path.
|
||||
- `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process.
|
||||
- `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob.
|
||||
|
||||
`"bytes"` and `"descriptions"` require a filesystem-backed Lance dataset and are not supported on in-memory tables.
|
||||
|
||||
Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiToPandas}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
Query builders also accept `blob_mode` on their `to_pandas()` method:
|
||||
|
||||
- Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`.
|
||||
- Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes.
|
||||
- This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{QueryToPandasKwargs}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Other modalities {#multimodal-other-modalities}
|
||||
|
||||
The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data:
|
||||
|
||||
- **Audio:** Read `.wav` or `.mp3` files as bytes.
|
||||
- **Video:** Store video transitions or full clips using the Blob API.
|
||||
- **PDFs/Documents:** Store the raw file content for document search.
|
||||
|
||||
### bfloat16 {#multimodal-bfloat16}
|
||||
|
||||
Sixteen-bit floats in the bfloat16 layout, stored as `fixed_size_binary(2)`.
|
||||
Useful as a vector element type when halving the memory matters more than the
|
||||
precision does.
|
||||
|
||||
## Schema evolution {#schema}
|
||||
|
||||
Schema evolution enables non-breaking modifications to a database table's structure — such as adding columns, altering data types, or dropping fields — to adapt to evolving data requirements without service interruptions.
|
||||
|
||||
@@ -454,7 +454,7 @@ These snippets load the local image file and store the bytes in an `image` colum
|
||||
</CodeBlock>
|
||||
</CodeGroup >
|
||||
|
||||
For more examples, see [Data types](/basics/data-types#blob).
|
||||
For more examples, see [Data types](/management/table#multimodal).
|
||||
|
||||
## 8\. Train on the result {#8-train}
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ icon: "list"
|
||||
| Feature | Description |
|
||||
|:---------------|:------------|
|
||||
| [Vector Search](/search/vector-search/) | Semantic similarity search with multiple distance metrics |
|
||||
| [Multivector Search](/search/multivector-search/) | Search using multiple vector embeddings per document |
|
||||
| [Multivector Search](/search/vector-search) | Search using multiple vector embeddings per document |
|
||||
| [Full-Text Search](/search/full-text-search/) | Keyword-based search with BM25 and pre-filtering |
|
||||
| [Hybrid Search](/search/hybrid-search/) | Combines vector and full-text search with reranking |
|
||||
| [Filtering](/search/filtering/) | Filter results based on metadata fields |
|
||||
|
||||
@@ -281,7 +281,7 @@ Every query vector must match the inner dimension of the multivector column; Lan
|
||||
|
||||
Here you can see how to take 2 query vectors and find the best matching pairs between them and document vectors using late interaction. The `np.random.random(size=(2, 256))` creates a 2×256 array with two random query vectors, `.limit(5)` returns the top 5 best document-query combinations, and `.to_pandas()` provides results in a DataFrame format.
|
||||
|
||||
**Read more:** [Multivector search](/search/multivector-search/)
|
||||
**Read more:** [Multivector search](/search/vector-search)
|
||||
|
||||
## Advanced Search Scenarios {#advanced-search-scenarios}
|
||||
|
||||
|
||||
@@ -30,7 +30,7 @@ For example, a real-time dashboard might require strong consistency, while a bat
|
||||
fine with eventual consistency. Stronger consistency is not free — the smaller the interval, the more
|
||||
often each read pays the cost of refreshing against object storage, which raises per-read latency and cost.
|
||||
|
||||
This setting works for both local ([LanceTable](/tables-and-namespaces#understanding-tables)) and remote
|
||||
This setting works for both local ([LanceTable](/management/database#uris)) and remote
|
||||
tables. It only affects read operations —
|
||||
write operations are always consistent.
|
||||
|
||||
|
||||
@@ -1,247 +0,0 @@
|
||||
---
|
||||
title: "Multimodal data"
|
||||
sidebarTitle: "Multimodal Data"
|
||||
description: "Storing images, audio and video in a table, as binary columns or blobs."
|
||||
icon: photo-film
|
||||
keywords: ["multimodal", "blob", "binary", "images", "video", "blob api"]
|
||||
---
|
||||
|
||||
import {
|
||||
PyMultimodalImports as MultimodalImports,
|
||||
TsMultimodalImports as TsMultimodalImports,
|
||||
RsMultimodalImports as RsMultimodalImports,
|
||||
PyCreateDummyData as CreateDummyData,
|
||||
TsCreateDummyData as TsCreateDummyData,
|
||||
RsCreateDummyData as RsCreateDummyData,
|
||||
PyDefineSchema as DefineSchema,
|
||||
TsDefineSchema as TsDefineSchema,
|
||||
RsDefineSchema as RsDefineSchema,
|
||||
PyIngestData as IngestData,
|
||||
TsIngestData as TsIngestData,
|
||||
RsIngestData as RsIngestData,
|
||||
PySearchData as SearchData,
|
||||
TsSearchData as TsSearchData,
|
||||
RsSearchData as RsSearchData,
|
||||
PyProcessResults as ProcessResults,
|
||||
TsProcessResults as TsProcessResults,
|
||||
RsProcessResults as RsProcessResults,
|
||||
PyBlobApiSchema as BlobApiSchema,
|
||||
TsBlobApiSchema as TsBlobApiSchema,
|
||||
RsBlobApiSchema as RsBlobApiSchema,
|
||||
PyBlobApiIngest as BlobApiIngest,
|
||||
TsBlobApiIngest as TsBlobApiIngest,
|
||||
RsBlobApiIngest as RsBlobApiIngest,
|
||||
PyBlobApiToPandas as BlobApiToPandas,
|
||||
PyQueryToPandasKwargs as QueryToPandasKwargs,
|
||||
} from '/snippets/multimodal.mdx';
|
||||
|
||||
Binary data -- images, audio, video, model artifacts -- lives in the table
|
||||
beside the structured columns and embeddings that describe it, and is governed
|
||||
by the same versioning and search.
|
||||
|
||||
Small values go in an ordinary `binary` column. Large ones go in a **blob**
|
||||
column, stored out of line so a scan over the other columns stays fast. See
|
||||
[Data types](/sql/data-types#blobs) for how the two differ to a reader.
|
||||
|
||||
## Store binary data {#store-binary-data}
|
||||
|
||||
To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust).
|
||||
|
||||
### 1\. Setup and imports {#1-setup-and-imports}
|
||||
|
||||
First, import the necessary libraries for LanceDB and Arrow in your SDK.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{MultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsMultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsMultimodalImports}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### 2\. Prepare data {#2-prepare-data}
|
||||
|
||||
For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{CreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsCreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsCreateDummyData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### 3\. Define the schema {#3-define-the-schema}
|
||||
|
||||
When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{DefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsDefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsDefineSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### 4\. Ingest data {#4-ingest-data}
|
||||
|
||||
Now, create the table using the data and the defined schema.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{IngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsIngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsIngestData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
## Retrieve and use blobs {#retrieve-and-use-blobs}
|
||||
|
||||
When you search your LanceDB table, you can retrieve the binary column just like any other metadata.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{SearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsSearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsSearchData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Convert bytes back to objects {#convert-bytes-back-to-objects}
|
||||
|
||||
Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer).
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{ProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsProcessResults}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
## Large Blobs (Blob API) {#large-blobs-blob-api}
|
||||
|
||||
For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront.
|
||||
|
||||
### 1\. Define a blob schema {#1-define-a-blob-schema}
|
||||
|
||||
To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### 2\. Ingest large blobs {#2-ingest-large-blobs}
|
||||
|
||||
You can then ingest data normally, and Lance will handle the optimized storage.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
<Card>
|
||||
For more advanced usage, including random access and file-like reading of blobs, see the
|
||||
Lance format's [blob API documentation](https://lance.org/guide/blob/).
|
||||
</Card>
|
||||
|
||||
### 3\. Convert blob tables to pandas {#3-convert-blob-tables-to-pandas}
|
||||
|
||||
When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`.
|
||||
|
||||
`blob_mode` accepts:
|
||||
|
||||
- `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path.
|
||||
- `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process.
|
||||
- `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob.
|
||||
|
||||
`"bytes"` and `"descriptions"` require a filesystem-backed Lance dataset and are not supported on in-memory tables.
|
||||
|
||||
Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiToPandas}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
Query builders also accept `blob_mode` on their `to_pandas()` method:
|
||||
|
||||
- Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`.
|
||||
- Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes.
|
||||
- This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{QueryToPandasKwargs}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
## Other modalities {#other-modalities}
|
||||
|
||||
The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data:
|
||||
|
||||
- **Audio:** Read `.wav` or `.mp3` files as bytes.
|
||||
- **Video:** Store video transitions or full clips using the Blob API.
|
||||
- **PDFs/Documents:** Store the raw file content for document search.
|
||||
|
||||
## bfloat16 {#bfloat16}
|
||||
|
||||
Sixteen-bit floats in the bfloat16 layout, stored as `fixed_size_binary(2)`.
|
||||
Useful as a vector element type when halving the memory matters more than the
|
||||
precision does.
|
||||
Reference in New Issue
Block a user