docs: document Blob V2 in the multimodal guide (#4623)

This commit is contained in:
Drew
2026-10-26 09:37:28 -07:00
committed by GitHub
parent 7952ee2a71
commit 436c209580
22 changed files with 3157 additions and 253 deletions
+370 -111
View File
@@ -110,15 +110,40 @@ import {
PyProcessResults as ProcessResults,
TsProcessResults as TsProcessResults,
RsProcessResults as RsProcessResults,
PyBlobApiSetup as BlobApiSetup,
TsBlobApiSetup as TsBlobApiSetup,
RsBlobApiImports as RsBlobApiImports,
RsBlobApiSetup as RsBlobApiSetup,
PyBlobApiSchema as BlobApiSchema,
TsBlobApiSchema as TsBlobApiSchema,
RsBlobApiSchema as RsBlobApiSchema,
PyBlobApiIngest as BlobApiIngest,
PyBlobApiBytes as BlobApiBytes,
TsBlobApiIngest as TsBlobApiIngest,
RsBlobApiIngest as RsBlobApiIngest,
PyBlobApiFiles as BlobApiFiles,
TsBlobApiFiles as TsBlobApiFiles,
RsBlobApiFiles as RsBlobApiFiles,
PyBlobApiFetch as BlobApiFetch,
TsBlobApiFetch as TsBlobApiFetch,
RsBlobApiFetch as RsBlobApiFetch,
PyBlobApiRanges as BlobApiRanges,
TsBlobApiRanges as TsBlobApiRanges,
RsBlobApiRanges as RsBlobApiRanges,
TsBlobApiBytesMode as TsBlobApiBytesMode,
PyLegacyBlobApiSchema as LegacyBlobApiSchema,
TsLegacyBlobApiSchema as TsLegacyBlobApiSchema,
RsLegacyBlobApiSchema as RsLegacyBlobApiSchema,
PyLegacyBlobApiIngest as LegacyBlobApiIngest,
TsLegacyBlobApiIngest as TsLegacyBlobApiIngest,
RsLegacyBlobApiIngest as RsLegacyBlobApiIngest,
PyLegacyBlobApiToPandas as LegacyBlobApiToPandas,
PyLegacyQueryToPandasKwargs as LegacyQueryToPandasKwargs,
} from '/snippets/multimodal.mdx';
import {
PyBlobApiToPandas as BlobApiToPandas,
PyQueryToPandasKwargs as QueryToPandasKwargs,
} from '/snippets/multimodal.mdx';
} from '/snippets/blob_pylance.mdx';
A table stores data in rows and columns and keeps a history of committed
versions. Queries read table data, writes update it, and indexes accelerate
@@ -307,138 +332,280 @@ extension types and pseudo-columns.
## Multimodal data {#multimodal}
Binary data (images, audio, video, model artifacts) lives in the table
beside the structured columns and embeddings that describe it, and is governed
by the same versioning and search.
Store images, audio, video, and documents alongside structured columns and
embeddings in the same versioned table.
Small values go in an ordinary `binary` column. Large ones go in a **blob**
column, stored out of line so a scan over the other columns stays fast. See
[Data types](/sql/data-types#blobs) for how the two differ to a reader.
Use `blob` columns for these payloads. Small values stay inline in the data
file, up to 64 KiB by default, and larger values are stored outside it.
### Store binary data {#multimodal-store-binary-data}
### Blob columns {#multimodal-blob-v2}
To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust).
<Warning>
Blob V2 is in early access.
</Warning>
#### 1\. Setup and imports {#multimodal-1-setup-and-imports}
A `blob` column keeps metadata separate from the payload bytes, so queries can
filter rows without loading the blobs.
First, import the necessary libraries for LanceDB and Arrow in your SDK.
Most queries return descriptors, which contain metadata about the blob instead
of its bytes. You can
[read the bytes in the query results](#multimodal-blob-query-results) or
[fetch them by row ID](#multimodal-read-blobs).
In Python, plain `to_pandas()` scans can also return `BlobFile` handles when
`pylance` is installed.
When you write raw bytes, Lance chooses a storage layout based on the payload
size. A descriptor's `kind` tells you which one was used:
| Storage mode | `kind` | Default placement | Storage behavior |
| --- | --- | --- | --- |
| `Inline` | `0` | Up to 64 KiB | Stored in the main data file. |
| `Packed` | `1` | Above 64 KiB, up to 4 MiB | Packed with other blobs into a shared sidecar file, up to 1 GiB per file by default. |
| `Dedicated` | `2` | Above 4 MiB | Stored in its own raw sidecar file. |
| `Managed` | `4` or `5` | Out-of-line values when managed storage is enabled | Stored in a Lance-owned object independently of the data file. |
Managed storage must be enabled separately. See
[Managed blobs](#managed-blobs) for details. `kind=5` identifies a Managed
object using an explicitly registered base.
Lance uses the default thresholds unless you customize them. Explicitly
configured thresholds are stored in the table schema and preserved on append.
#### Setup {#multimodal-blob-v2-setup}
Import the SDK and open a local database. The examples in this section use `db`.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{MultimodalImports}
{BlobApiSetup}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsMultimodalImports}
{TsMultimodalImports}{"\n"}{TsBlobApiSetup}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsMultimodalImports}
{RsBlobApiImports}{"\n"}{RsBlobApiSetup}
</CodeBlock>
</CodeGroup>
#### 2\. Prepare data {#multimodal-2-prepare-data}
#### Define a blob schema {#multimodal-blob-v2-schema}
For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object.
Declare a blob column with `lancedb.blob()` in Python and TypeScript, or
`blob()` in Rust:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{CreateDummyData}
{BlobApiSchema}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsCreateDummyData}
{TsBlobApiSchema}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsCreateDummyData}
{RsBlobApiSchema}
</CodeBlock>
</CodeGroup>
#### 3\. Define the schema {#multimodal-3-define-the-schema}
#### Ingest blobs {#multimodal-blob-v2-ingest}
When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list.
Pass raw bytes for each blob value. In Python, a blob column accepts `bytes`.
In TypeScript, it accepts a `Buffer`. The Python example creates two JPEGs
with EXIF metadata, which we'll read using a blob handle below.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{DefineSchema}
{BlobApiIngest}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsDefineSchema}
{TsBlobApiIngest}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsDefineSchema}
{RsBlobApiIngest}
</CodeBlock>
</CodeGroup>
#### 4\. Ingest data {#multimodal-4-ingest-data}
#### Read bytes in query results {#multimodal-blob-query-results}
Now, create the table using the data and the defined schema.
Python and TypeScript can return blob bytes in query results without
collecting `_rowid`. Rust uses [`fetch_blobs`](#multimodal-read-blobs) with
row IDs instead.
##### Python {#multimodal-blob-python}
`to_list(blob_mode="bytes")` returns the `image` bytes from the `photos` table
above:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{IngestData}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsIngestData}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsIngestData}
{BlobApiBytes}
</CodeBlock>
</CodeGroup>
### Retrieve and use blobs {#multimodal-retrieve-and-use-blobs}
In the Python SDK, `Table.to_pandas()`, `Table.to_arrow()`, and the query builder
methods `to_pandas()`, `to_arrow()`, and `to_list()` accept a `blob_mode`
argument:
When you search your LanceDB table, you can retrieve the binary column just like any other metadata.
| `blob_mode` | `to_pandas()` | `to_arrow()` and `to_list()` |
| --- | --- | --- |
| `"lazy"` | Default. Plain scans return seekable `BlobFile` handles. Vector, full-text, hybrid, and ordered queries return descriptor dicts. | Default. Returns descriptors carrying `_lance_row_id` so the result table can be passed to `fetch_blobs` or `fetch_blob_files`. |
| `"descriptions"` | Returns descriptor dicts with `kind`, `position`, `size`, `blob_id`, and `blob_uri`. | Returns the 5-field descriptor without `_lance_row_id`. |
| `"bytes"` | Materializes each blob as `bytes` in the DataFrame. | Materializes each blob as `large_binary` in Arrow or `bytes` in Python dicts. |
`to_pandas(blob_mode="lazy")` and `to_pandas(blob_mode="bytes")` require
pylance. Without it they raise `ImportError`. `to_arrow()`,
`to_list(blob_mode="bytes")`, `to_pandas(blob_mode="descriptions")`, and the
fetch methods do not.
`blob_mode="bytes"` requires a local table snapshot, so remote and in-memory
tables raise `NotImplementedError`. On a remote table, `Table.to_pandas()` and
`Table.to_arrow()` aren't supported, and a query's `to_pandas()` also raises in
the default `"lazy"` mode because it would open `BlobFile` handles. Query with
`blob_mode="descriptions"`, then read bytes with `fetch_blobs` or
`fetch_blob_files`.
`blob_mode="bytes"` materializes top-level and struct-nested blob columns. It
raises `ValueError` when the selected columns include blobs
inside a list. Select those columns with `blob_mode="descriptions"`, or leave
them out of the projection. Extra keyword arguments on
`to_pandas()`, such as `split_blocks` and `self_destruct`, are forwarded to
PyArrow.
###### Optional pandas examples {#multimodal-blob-pandas}
These examples require pylance. The query example expects `tbl_async`,
`query_vector`, and a `video` column, which the `photos` table does not define.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{SearchData}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsSearchData}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsSearchData}
{BlobApiToPandas}
</CodeBlock>
</CodeGroup>
#### Convert bytes back to objects {#multimodal-convert-bytes-back-to-objects}
Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer).
Sync and async query builders accept the same `blob_mode` values:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{ProcessResults}
{QueryToPandasKwargs}
</CodeBlock>
</CodeGroup>
##### TypeScript {#multimodal-blob-typescript}
In TypeScript, `toArray()` and `toArrow()` accept a `blobMode` option:
| `blobMode` | Result |
| --- | --- |
| `"descriptions"` | Default. Returns descriptors. |
| `"bytes"` | Returns each top-level blob as bytes, or `null` for a null blob. Blobs nested in a struct or list keep their descriptors. |
With `"bytes"`, the rows and the bytes come from the same table version. If the
table changes while the query runs, the query runs again, up to three times:
<CodeGroup>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBlobApiBytesMode}
</CodeBlock>
</CodeGroup>
#### Read blobs on demand {#multimodal-read-blobs}
Query the rows you want and pass their `_rowid` values to a fetch method.
`fetch_blob_files` opens a seekable handle and loads bytes only when you read.
The Python example opens the JPEG from the ingest step and reads its EXIF
Make tag with Pillow. TypeScript and Rust read the first bytes from the handle.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiFiles}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsProcessResults}
{TsBlobApiFiles}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsProcessResults}
{RsBlobApiFiles}
</CodeBlock>
</CodeGroup>
The other fetch methods load bytes directly:
| Need | Python | TypeScript | Rust |
| --- | --- | --- | --- |
| Whole payloads for a set of rows | `fetch_blobs` | `fetchBlobs` | `fetch_blobs` |
| Byte slices across many rows in one call | `fetch_blob_ranges` | Not available. Use `fetchBlobFiles` and `readRanges`. | `fetch_blob_ranges` |
- `fetch_blobs` returns the full bytes for each row ID in order. Null blobs
produce null entries, and empty blobs produce empty byte strings.
- `fetch_blob_files` loads bytes when you call `read`, `read_range`, or
`read_ranges`. Range reads don't move the file cursor. Null blobs produce
null handles.
- `fetch_blob_ranges` takes `(row_id, offset, length)` requests and returns one
slice per request, in order. A request list can repeat a row to read more
than one slice of that blob.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiFetch}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBlobApiFetch}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBlobApiFetch}
</CodeBlock>
</CodeGroup>
Python and Rust read slices across rows with `fetch_blob_ranges`. On a handle,
`read_ranges` reads more than one slice of one blob. Python takes
`(offset, length)` pairs, TypeScript takes `{ start, end }` ranges with an
exclusive `end`, and Rust takes `start..end` ranges:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiRanges}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBlobApiRanges}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBlobApiRanges}
</CodeBlock>
</CodeGroup>
#### Notes and limits {#multimodal-blob-notes}
Blob fetches fail if any requested row ID is missing or deleted. With stable
row IDs enabled, IDs remain valid across compaction.
A blob field can sit at the top level of the schema or inside a struct or list.
Struct-nested blob columns use dotted paths such as `clip.video` in the fetch
APIs. Blobs inside lists can be written and queried as descriptors. A table
row ID does not address one list element, so `blob_columns()`, `fetch_blobs`,
`fetch_blob_files`, and `blob_mode="bytes"` can't return those bytes.
See the Lance [blob guide](https://lance.org/guide/blob/) for the storage format.
### Managed blobs (Blob v2) {#managed-blobs}
Use `lancedb.blob()` for images, audio, videos, and other binary values whose
payloads should load separately from the rest of a row. Write Python `bytes`
normally. Set the table schema metadata `lance.blob.managed` to `true` to
store small values inline and larger values in managed objects under the table's `_blobs/` directory. Medium values can share a packed
object; large values use a dedicated object.
Set the table schema metadata `lance.blob.managed` to `true` on a table with
[`lancedb.blob()`](#multimodal-blob-v2) columns to store small values inline and
larger values in managed objects under the table's `_blobs/` directory. Medium
values can share a packed object, and large values use a dedicated object.
The table owns these objects. You do not upload them separately, keep an object
catalog, or delete their files yourself. A descriptor records where to read the
bytes. Compaction preserves the descriptor and its payload object, so merging
table fragments does not rewrite the blob payloads.
catalog, or delete their files yourself. Compaction preserves each descriptor
and its payload object, so merging table fragments does not rewrite the blob
payloads.
<Warning>
Use an SDK and Enterprise deployment that support Blob v2 Managed storage.
@@ -484,16 +651,10 @@ assert by_id[1] == b""
assert len(by_id[2]) == 102400
```
`None` is a null value; `b""` is a present, empty blob. `fetch_blobs` returns
bytes in the order of the supplied row IDs. It materializes the selected
payloads, so fetch bounded batches when blobs are large. Row IDs belong to
their table; fetch them from the same table and snapshot that produced the
query result.
Keep descriptors opaque in application code. Their representation can differ
between inline values, Managed objects, and externally referenced blobs. For
streaming and range reads, see the
[Lance blob guide](https://lance.org/guide/blob/).
`fetch_blobs` materializes the selected payloads in memory, so fetch bounded
batches when blobs are large. Row IDs belong to their table and snapshot. Keep
descriptors opaque in application code, because their representation can differ
between inline values and Managed objects.
#### Derived blobs and maintenance {#managed-blobs-lifecycle}
@@ -509,92 +670,190 @@ any of its contained blobs is still referenced.
### Legacy large-binary blobs {#multimodal-large-blobs-blob-api}
The schema below describes the earlier large-binary Blob API. It is distinct
from the `lancedb.blob()` schema above; changing the field's metadata alone does
not migrate an existing column to Blob v2.
For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront.
Use [`lancedb.blob()`](#multimodal-blob-v2) for new tables. The sections below
cover existing tables that use the V1 large-binary Blob API
(`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and
`DataType::LargeBinary` in Rust with `{"lance-encoding:blob": "true"}` field
metadata). Changing a field's metadata alone does not migrate an existing V1
column to Blob v2.
#### 1\. Define a blob schema {#multimodal-1-define-a-blob-schema}
To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata.
V1 blob columns use a `large_binary` field marked with `{"lance-encoding:blob": "true"}` metadata:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiSchema}
{LegacyBlobApiSchema}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBlobApiSchema}
{TsLegacyBlobApiSchema}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBlobApiSchema}
{RsLegacyBlobApiSchema}
</CodeBlock>
</CodeGroup>
#### 2\. Ingest large blobs {#multimodal-2-ingest-large-blobs}
You can then ingest data normally, and Lance will handle the optimized storage.
Write raw bytes to a V1 blob column:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiIngest}
{LegacyBlobApiIngest}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsBlobApiIngest}
{TsLegacyBlobApiIngest}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsBlobApiIngest}
{RsLegacyBlobApiIngest}
</CodeBlock>
</CodeGroup>
<Card>
For more advanced usage, including random access and file-like reading of blobs, see the
Lance format's [blob API documentation](https://lance.org/guide/blob/).
</Card>
#### 3\. Convert blob tables to pandas {#multimodal-3-convert-blob-tables-to-pandas}
When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`.
`blob_mode` accepts:
- `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path.
- `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process.
- `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob.
`"bytes"` is not supported on in-memory tables.
Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`:
On local tables with V1 blob columns, `to_pandas()` in Python accepts `blob_mode`
(`"lazy"`, `"bytes"`, or `"descriptions"`). Remote tables raise
`NotImplementedError`, and `"bytes"` is not supported on in-memory tables.
Extra keyword arguments such as `split_blocks` or `self_destruct` are forwarded
to the PyArrow / Lance pandas conversion:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{BlobApiToPandas}
{LegacyBlobApiToPandas}
</CodeBlock>
</CodeGroup>
Query builders also accept `blob_mode` on their `to_pandas()` method:
- Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`.
- Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes.
- This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded.
On query builders with V1 blob columns, plain scans support `"lazy"`, `"bytes"`,
and `"descriptions"`. Vector, FTS, hybrid, and ordered queries cannot
materialize V1 blob columns through `to_pandas()`, so omit those columns from
the projection:
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{QueryToPandasKwargs}
{LegacyQueryToPandasKwargs}
</CodeBlock>
</CodeGroup>
### Store binary data {#multimodal-store-binary-data}
Use a `binary` column when the query should return the raw bytes. Declare it as `pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust.
#### 1\. Setup and imports {#multimodal-1-setup-and-imports}
Import LanceDB and Arrow in your SDK.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{MultimodalImports}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsMultimodalImports}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsMultimodalImports}
</CodeBlock>
</CodeGroup>
#### 2\. Prepare data {#multimodal-2-prepare-data}
Encode each image or file into raw bytes before writing.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{CreateDummyData}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsCreateDummyData}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsCreateDummyData}
</CodeBlock>
</CodeGroup>
#### 3\. Define the schema {#multimodal-3-define-the-schema}
Declare the schema explicitly so Arrow and LanceDB store the field as `binary` instead of inferring a string or integer list.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{DefineSchema}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsDefineSchema}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsDefineSchema}
</CodeBlock>
</CodeGroup>
#### 4\. Ingest data {#multimodal-4-ingest-data}
Create the table from the rows and schema.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{IngestData}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsIngestData}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsIngestData}
</CodeBlock>
</CodeGroup>
### Retrieve binary data {#multimodal-retrieve-and-use-blobs}
When you search the table, select the `binary` column alongside your metadata columns.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{SearchData}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsSearchData}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsSearchData}
</CodeBlock>
</CodeGroup>
#### Convert bytes back to objects {#multimodal-convert-bytes-back-to-objects}
Decode the returned bytes into an image or media buffer in your application.
<CodeGroup>
<CodeBlock filename="Python" language="Python" icon="python">
{ProcessResults}
</CodeBlock>
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
{TsProcessResults}
</CodeBlock>
<CodeBlock filename="Rust" language="Rust" icon="rust">
{RsProcessResults}
</CodeBlock>
</CodeGroup>
### Other modalities {#multimodal-other-modalities}
The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data:
- **Audio:** Read `.wav` or `.mp3` files as bytes.
- **Video:** Store video transitions or full clips using the Blob API.
- **PDFs/Documents:** Store the raw file content for document search.
For PDFs, audio recordings, and other files, keep searchable fields in separate
columns so you can filter rows before fetching blob bytes.
### bfloat16 {#multimodal-bfloat16}