lancedb/docs/web/tables/multimodal.mdx

---
title: Multimodal Data (Blobs)
sidebarTitle: "Working with multimodal data"
description: Learn how to store and query multimodal data (images, audio, video) directly in LanceDB using binary columns.
icon: "images"
keywords: ["blob", "large binary", "blobs", "multimodal"]
---

import {
    PyMultimodalImports as MultimodalImports,
    TsMultimodalImports as TsMultimodalImports,
    RsMultimodalImports as RsMultimodalImports,
    PyCreateDummyData as CreateDummyData,
    TsCreateDummyData as TsCreateDummyData,
    RsCreateDummyData as RsCreateDummyData,
    PyDefineSchema as DefineSchema,
    TsDefineSchema as TsDefineSchema,
    RsDefineSchema as RsDefineSchema,
    PyIngestData as IngestData,
    TsIngestData as TsIngestData,
    RsIngestData as RsIngestData,
    PySearchData as SearchData,
    TsSearchData as TsSearchData,
    RsSearchData as RsSearchData,
    PyProcessResults as ProcessResults,
    TsProcessResults as TsProcessResults,
    RsProcessResults as RsProcessResults,
    PyBlobApiSchema as BlobApiSchema,
    TsBlobApiSchema as TsBlobApiSchema,
    RsBlobApiSchema as RsBlobApiSchema,
    PyBlobApiIngest as BlobApiIngest,
    TsBlobApiIngest as TsBlobApiIngest,
    RsBlobApiIngest as RsBlobApiIngest,
    PyBlobApiToPandas as BlobApiToPandas,
    PyQueryToPandasKwargs as QueryToPandasKwargs,
} from '/snippets/multimodal.mdx';

LanceDB handles multimodal data—images, audio, video, and PDF files—natively by storing the raw bytes in a binary column alongside your vectors and metadata. This approach simplifies your data infrastructure by keeping the raw assets and their embeddings in the same database, eliminating the need for separate object storage for many use cases.

This guide demonstrates how to ingest, store, and retrieve image data using standard binary columns, and also introduces the **Lance Blob API** for optimized handling of larger multimodal files.

## Store binary data {#store-binary-data}

To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust).

### 1\. Setup and imports {#1-setup-and-imports}

First, import the necessary libraries for LanceDB and Arrow in your SDK.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {MultimodalImports}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsMultimodalImports}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsMultimodalImports}
    </CodeBlock>
</CodeGroup>

### 2\. Prepare data {#2-prepare-data}

For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {CreateDummyData}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsCreateDummyData}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsCreateDummyData}
    </CodeBlock>
</CodeGroup>

### 3\. Define the schema {#3-define-the-schema}

When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {DefineSchema}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsDefineSchema}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsDefineSchema}
    </CodeBlock>
</CodeGroup>

### 4\. Ingest data {#4-ingest-data}

Now, create the table using the data and the defined schema.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {IngestData}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsIngestData}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsIngestData}
    </CodeBlock>
</CodeGroup>

## Retrieve and use blobs {#retrieve-and-use-blobs}

When you search your LanceDB table, you can retrieve the binary column just like any other metadata.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {SearchData}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsSearchData}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsSearchData}
    </CodeBlock>
</CodeGroup>

### Convert bytes back to objects {#convert-bytes-back-to-objects}

Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer).

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {ProcessResults}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsProcessResults}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsProcessResults}
    </CodeBlock>
</CodeGroup>

## Large Blobs (Blob API) {#large-blobs-blob-api}

For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront.

### 1\. Define a blob schema {#1-define-a-blob-schema}

To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {BlobApiSchema}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsBlobApiSchema}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsBlobApiSchema}
    </CodeBlock>
</CodeGroup>

### 2\. Ingest large blobs {#2-ingest-large-blobs}

You can then ingest data normally, and Lance will handle the optimized storage.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {BlobApiIngest}
    </CodeBlock>

    <CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
    {TsBlobApiIngest}
    </CodeBlock>

    <CodeBlock filename="Rust" language="Rust" icon="rust">
    {RsBlobApiIngest}
    </CodeBlock>
</CodeGroup>

<Card>
For more advanced usage, including random access and file-like reading of blobs, see the
Lance format's [blob API documentation](https://lance.org/guide/blob/).
</Card>

### 3\. Convert blob tables to pandas {#3-convert-blob-tables-to-pandas}

When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`.

`blob_mode` accepts:

- `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path.
- `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process.
- `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob.

`"bytes"` and `"descriptions"` require a filesystem-backed Lance dataset and are not supported on in-memory tables.

Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`:

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {BlobApiToPandas}
    </CodeBlock>
</CodeGroup>

Query builders also accept `blob_mode` on their `to_pandas()` method:

- Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`.
- Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes.
- This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded.

<CodeGroup>
    <CodeBlock filename="Python" language="Python" icon="python">
    {QueryToPandasKwargs}
    </CodeBlock>
</CodeGroup>

## Other modalities {#other-modalities}

The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data:

- **Audio:** Read `.wav` or `.mp3` files as bytes.
- **Video:** Store video transitions or full clips using the Blob API.
- **PDFs/Documents:** Store the raw file content for document search.