mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-27 14:06:14 +00:00
docs: document Blob V2 in the multimodal guide (#4623)
This commit is contained in:
Drew
@@ -146,7 +146,8 @@ You can define a custom Arrow schema for the table. This is useful when you want
|
||||
An explicit schema also determines which columns accept missing values (`null`). If later writes omit a
|
||||
non-nullable column, or provide actual nulls for it, ingestion fails; nullable columns can be
|
||||
omitted or written with null values. Without an explicit schema, Python infers list-like vector
|
||||
values as fixed-size `float32` vector fields from the observed dimension.
|
||||
values as fixed-size `float32` vector fields from the observed dimension. For images, audio, video, or
|
||||
other binary columns whose payloads should load on demand, declare a [blob column](/management/table#multimodal-blob-v2) in the schema.
|
||||
|
||||
For Python ingestion, malformed vectors fail by default (`on_bad_vectors="error"`). This includes
|
||||
vectors with incorrect sizes, NaNs, or null components, including pandas NaNs converted to Arrow
|
||||
|
||||
+370
-111
@@ -110,15 +110,40 @@ import {
|
||||
PyProcessResults as ProcessResults,
|
||||
TsProcessResults as TsProcessResults,
|
||||
RsProcessResults as RsProcessResults,
|
||||
PyBlobApiSetup as BlobApiSetup,
|
||||
TsBlobApiSetup as TsBlobApiSetup,
|
||||
RsBlobApiImports as RsBlobApiImports,
|
||||
RsBlobApiSetup as RsBlobApiSetup,
|
||||
PyBlobApiSchema as BlobApiSchema,
|
||||
TsBlobApiSchema as TsBlobApiSchema,
|
||||
RsBlobApiSchema as RsBlobApiSchema,
|
||||
PyBlobApiIngest as BlobApiIngest,
|
||||
PyBlobApiBytes as BlobApiBytes,
|
||||
TsBlobApiIngest as TsBlobApiIngest,
|
||||
RsBlobApiIngest as RsBlobApiIngest,
|
||||
PyBlobApiFiles as BlobApiFiles,
|
||||
TsBlobApiFiles as TsBlobApiFiles,
|
||||
RsBlobApiFiles as RsBlobApiFiles,
|
||||
PyBlobApiFetch as BlobApiFetch,
|
||||
TsBlobApiFetch as TsBlobApiFetch,
|
||||
RsBlobApiFetch as RsBlobApiFetch,
|
||||
PyBlobApiRanges as BlobApiRanges,
|
||||
TsBlobApiRanges as TsBlobApiRanges,
|
||||
RsBlobApiRanges as RsBlobApiRanges,
|
||||
TsBlobApiBytesMode as TsBlobApiBytesMode,
|
||||
PyLegacyBlobApiSchema as LegacyBlobApiSchema,
|
||||
TsLegacyBlobApiSchema as TsLegacyBlobApiSchema,
|
||||
RsLegacyBlobApiSchema as RsLegacyBlobApiSchema,
|
||||
PyLegacyBlobApiIngest as LegacyBlobApiIngest,
|
||||
TsLegacyBlobApiIngest as TsLegacyBlobApiIngest,
|
||||
RsLegacyBlobApiIngest as RsLegacyBlobApiIngest,
|
||||
PyLegacyBlobApiToPandas as LegacyBlobApiToPandas,
|
||||
PyLegacyQueryToPandasKwargs as LegacyQueryToPandasKwargs,
|
||||
} from '/snippets/multimodal.mdx';
|
||||
import {
|
||||
PyBlobApiToPandas as BlobApiToPandas,
|
||||
PyQueryToPandasKwargs as QueryToPandasKwargs,
|
||||
} from '/snippets/multimodal.mdx';
|
||||
} from '/snippets/blob_pylance.mdx';
|
||||
|
||||
A table stores data in rows and columns and keeps a history of committed
|
||||
versions. Queries read table data, writes update it, and indexes accelerate
|
||||
@@ -307,138 +332,280 @@ extension types and pseudo-columns.
|
||||
|
||||
## Multimodal data {#multimodal}
|
||||
|
||||
Binary data (images, audio, video, model artifacts) lives in the table
|
||||
beside the structured columns and embeddings that describe it, and is governed
|
||||
by the same versioning and search.
|
||||
Store images, audio, video, and documents alongside structured columns and
|
||||
embeddings in the same versioned table.
|
||||
|
||||
Small values go in an ordinary `binary` column. Large ones go in a **blob**
|
||||
column, stored out of line so a scan over the other columns stays fast. See
|
||||
[Data types](/sql/data-types#blobs) for how the two differ to a reader.
|
||||
Use `blob` columns for these payloads. Small values stay inline in the data
|
||||
file, up to 64 KiB by default, and larger values are stored outside it.
|
||||
|
||||
### Store binary data {#multimodal-store-binary-data}
|
||||
### Blob columns {#multimodal-blob-v2}
|
||||
|
||||
To store binary data, define a binary Arrow field in your schema (`pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust).
|
||||
<Warning>
|
||||
Blob V2 is in early access.
|
||||
</Warning>
|
||||
|
||||
#### 1\. Setup and imports {#multimodal-1-setup-and-imports}
|
||||
A `blob` column keeps metadata separate from the payload bytes, so queries can
|
||||
filter rows without loading the blobs.
|
||||
|
||||
First, import the necessary libraries for LanceDB and Arrow in your SDK.
|
||||
Most queries return descriptors, which contain metadata about the blob instead
|
||||
of its bytes. You can
|
||||
[read the bytes in the query results](#multimodal-blob-query-results) or
|
||||
[fetch them by row ID](#multimodal-read-blobs).
|
||||
|
||||
In Python, plain `to_pandas()` scans can also return `BlobFile` handles when
|
||||
`pylance` is installed.
|
||||
|
||||
When you write raw bytes, Lance chooses a storage layout based on the payload
|
||||
size. A descriptor's `kind` tells you which one was used:
|
||||
|
||||
| Storage mode | `kind` | Default placement | Storage behavior |
|
||||
| --- | --- | --- | --- |
|
||||
| `Inline` | `0` | Up to 64 KiB | Stored in the main data file. |
|
||||
| `Packed` | `1` | Above 64 KiB, up to 4 MiB | Packed with other blobs into a shared sidecar file, up to 1 GiB per file by default. |
|
||||
| `Dedicated` | `2` | Above 4 MiB | Stored in its own raw sidecar file. |
|
||||
| `Managed` | `4` or `5` | Out-of-line values when managed storage is enabled | Stored in a Lance-owned object independently of the data file. |
|
||||
|
||||
Managed storage must be enabled separately. See
|
||||
[Managed blobs](#managed-blobs) for details. `kind=5` identifies a Managed
|
||||
object using an explicitly registered base.
|
||||
|
||||
Lance uses the default thresholds unless you customize them. Explicitly
|
||||
configured thresholds are stored in the table schema and preserved on append.
|
||||
|
||||
#### Setup {#multimodal-blob-v2-setup}
|
||||
|
||||
Import the SDK and open a local database. The examples in this section use `db`.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{MultimodalImports}
|
||||
{BlobApiSetup}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsMultimodalImports}
|
||||
{TsMultimodalImports}{"\n"}{TsBlobApiSetup}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsMultimodalImports}
|
||||
{RsBlobApiImports}{"\n"}{RsBlobApiSetup}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 2\. Prepare data {#multimodal-2-prepare-data}
|
||||
#### Define a blob schema {#multimodal-blob-v2-schema}
|
||||
|
||||
For this example, we'll create some dummy in-memory images. In a real application, you would read these from files or an API. The key is to convert your data (image, audio, etc.) into a raw `bytes` object.
|
||||
Declare a blob column with `lancedb.blob()` in Python and TypeScript, or
|
||||
`blob()` in Rust:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{CreateDummyData}
|
||||
{BlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsCreateDummyData}
|
||||
{TsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsCreateDummyData}
|
||||
{RsBlobApiSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 3\. Define the schema {#multimodal-3-define-the-schema}
|
||||
#### Ingest blobs {#multimodal-blob-v2-ingest}
|
||||
|
||||
When creating the table, it is **highly recommended** to define the schema explicitly. This ensures that your binary data is correctly interpreted as a `binary` type by Arrow/LanceDB and not as a generic string or list.
|
||||
Pass raw bytes for each blob value. In Python, a blob column accepts `bytes`.
|
||||
In TypeScript, it accepts a `Buffer`. The Python example creates two JPEGs
|
||||
with EXIF metadata, which we'll read using a blob handle below.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{DefineSchema}
|
||||
{BlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsDefineSchema}
|
||||
{TsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsDefineSchema}
|
||||
{RsBlobApiIngest}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 4\. Ingest data {#multimodal-4-ingest-data}
|
||||
#### Read bytes in query results {#multimodal-blob-query-results}
|
||||
|
||||
Now, create the table using the data and the defined schema.
|
||||
Python and TypeScript can return blob bytes in query results without
|
||||
collecting `_rowid`. Rust uses [`fetch_blobs`](#multimodal-read-blobs) with
|
||||
row IDs instead.
|
||||
|
||||
##### Python {#multimodal-blob-python}
|
||||
|
||||
`to_list(blob_mode="bytes")` returns the `image` bytes from the `photos` table
|
||||
above:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{IngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsIngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsIngestData}
|
||||
{BlobApiBytes}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Retrieve and use blobs {#multimodal-retrieve-and-use-blobs}
|
||||
In the Python SDK, `Table.to_pandas()`, `Table.to_arrow()`, and the query builder
|
||||
methods `to_pandas()`, `to_arrow()`, and `to_list()` accept a `blob_mode`
|
||||
argument:
|
||||
|
||||
When you search your LanceDB table, you can retrieve the binary column just like any other metadata.
|
||||
| `blob_mode` | `to_pandas()` | `to_arrow()` and `to_list()` |
|
||||
| --- | --- | --- |
|
||||
| `"lazy"` | Default. Plain scans return seekable `BlobFile` handles. Vector, full-text, hybrid, and ordered queries return descriptor dicts. | Default. Returns descriptors carrying `_lance_row_id` so the result table can be passed to `fetch_blobs` or `fetch_blob_files`. |
|
||||
| `"descriptions"` | Returns descriptor dicts with `kind`, `position`, `size`, `blob_id`, and `blob_uri`. | Returns the 5-field descriptor without `_lance_row_id`. |
|
||||
| `"bytes"` | Materializes each blob as `bytes` in the DataFrame. | Materializes each blob as `large_binary` in Arrow or `bytes` in Python dicts. |
|
||||
|
||||
`to_pandas(blob_mode="lazy")` and `to_pandas(blob_mode="bytes")` require
|
||||
pylance. Without it they raise `ImportError`. `to_arrow()`,
|
||||
`to_list(blob_mode="bytes")`, `to_pandas(blob_mode="descriptions")`, and the
|
||||
fetch methods do not.
|
||||
|
||||
`blob_mode="bytes"` requires a local table snapshot, so remote and in-memory
|
||||
tables raise `NotImplementedError`. On a remote table, `Table.to_pandas()` and
|
||||
`Table.to_arrow()` aren't supported, and a query's `to_pandas()` also raises in
|
||||
the default `"lazy"` mode because it would open `BlobFile` handles. Query with
|
||||
`blob_mode="descriptions"`, then read bytes with `fetch_blobs` or
|
||||
`fetch_blob_files`.
|
||||
|
||||
`blob_mode="bytes"` materializes top-level and struct-nested blob columns. It
|
||||
raises `ValueError` when the selected columns include blobs
|
||||
inside a list. Select those columns with `blob_mode="descriptions"`, or leave
|
||||
them out of the projection. Extra keyword arguments on
|
||||
`to_pandas()`, such as `split_blocks` and `self_destruct`, are forwarded to
|
||||
PyArrow.
|
||||
|
||||
###### Optional pandas examples {#multimodal-blob-pandas}
|
||||
|
||||
These examples require pylance. The query example expects `tbl_async`,
|
||||
`query_vector`, and a `video` column, which the `photos` table does not define.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{SearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsSearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsSearchData}
|
||||
{BlobApiToPandas}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### Convert bytes back to objects {#multimodal-convert-bytes-back-to-objects}
|
||||
|
||||
Once you have the bytes back from the search result, you can decode them into the original format (for example, an image object or audio buffer).
|
||||
Sync and async query builders accept the same `blob_mode` values:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{ProcessResults}
|
||||
{QueryToPandasKwargs}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
##### TypeScript {#multimodal-blob-typescript}
|
||||
|
||||
In TypeScript, `toArray()` and `toArrow()` accept a `blobMode` option:
|
||||
|
||||
| `blobMode` | Result |
|
||||
| --- | --- |
|
||||
| `"descriptions"` | Default. Returns descriptors. |
|
||||
| `"bytes"` | Returns each top-level blob as bytes, or `null` for a null blob. Blobs nested in a struct or list keep their descriptors. |
|
||||
|
||||
With `"bytes"`, the rows and the bytes come from the same table version. If the
|
||||
table changes while the query runs, the query runs again, up to three times:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiBytesMode}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### Read blobs on demand {#multimodal-read-blobs}
|
||||
|
||||
Query the rows you want and pass their `_rowid` values to a fetch method.
|
||||
`fetch_blob_files` opens a seekable handle and loads bytes only when you read.
|
||||
The Python example opens the JPEG from the ingest step and reads its EXIF
|
||||
Make tag with Pillow. TypeScript and Rust read the first bytes from the handle.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiFiles}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsProcessResults}
|
||||
{TsBlobApiFiles}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsProcessResults}
|
||||
{RsBlobApiFiles}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
The other fetch methods load bytes directly:
|
||||
|
||||
| Need | Python | TypeScript | Rust |
|
||||
| --- | --- | --- | --- |
|
||||
| Whole payloads for a set of rows | `fetch_blobs` | `fetchBlobs` | `fetch_blobs` |
|
||||
| Byte slices across many rows in one call | `fetch_blob_ranges` | Not available. Use `fetchBlobFiles` and `readRanges`. | `fetch_blob_ranges` |
|
||||
|
||||
- `fetch_blobs` returns the full bytes for each row ID in order. Null blobs
|
||||
produce null entries, and empty blobs produce empty byte strings.
|
||||
- `fetch_blob_files` loads bytes when you call `read`, `read_range`, or
|
||||
`read_ranges`. Range reads don't move the file cursor. Null blobs produce
|
||||
null handles.
|
||||
- `fetch_blob_ranges` takes `(row_id, offset, length)` requests and returns one
|
||||
slice per request, in order. A request list can repeat a row to read more
|
||||
than one slice of that blob.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiFetch}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiFetch}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiFetch}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
Python and Rust read slices across rows with `fetch_blob_ranges`. On a handle,
|
||||
`read_ranges` reads more than one slice of one blob. Python takes
|
||||
`(offset, length)` pairs, TypeScript takes `{ start, end }` ranges with an
|
||||
exclusive `end`, and Rust takes `start..end` ranges:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiRanges}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiRanges}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiRanges}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### Notes and limits {#multimodal-blob-notes}
|
||||
|
||||
Blob fetches fail if any requested row ID is missing or deleted. With stable
|
||||
row IDs enabled, IDs remain valid across compaction.
|
||||
|
||||
A blob field can sit at the top level of the schema or inside a struct or list.
|
||||
Struct-nested blob columns use dotted paths such as `clip.video` in the fetch
|
||||
APIs. Blobs inside lists can be written and queried as descriptors. A table
|
||||
row ID does not address one list element, so `blob_columns()`, `fetch_blobs`,
|
||||
`fetch_blob_files`, and `blob_mode="bytes"` can't return those bytes.
|
||||
|
||||
See the Lance [blob guide](https://lance.org/guide/blob/) for the storage format.
|
||||
|
||||
### Managed blobs (Blob v2) {#managed-blobs}
|
||||
|
||||
Use `lancedb.blob()` for images, audio, videos, and other binary values whose
|
||||
payloads should load separately from the rest of a row. Write Python `bytes`
|
||||
normally. Set the table schema metadata `lance.blob.managed` to `true` to
|
||||
store small values inline and larger values in managed objects under the table's `_blobs/` directory. Medium values can share a packed
|
||||
object; large values use a dedicated object.
|
||||
Set the table schema metadata `lance.blob.managed` to `true` on a table with
|
||||
[`lancedb.blob()`](#multimodal-blob-v2) columns to store small values inline and
|
||||
larger values in managed objects under the table's `_blobs/` directory. Medium
|
||||
values can share a packed object, and large values use a dedicated object.
|
||||
|
||||
The table owns these objects. You do not upload them separately, keep an object
|
||||
catalog, or delete their files yourself. A descriptor records where to read the
|
||||
bytes. Compaction preserves the descriptor and its payload object, so merging
|
||||
table fragments does not rewrite the blob payloads.
|
||||
catalog, or delete their files yourself. Compaction preserves each descriptor
|
||||
and its payload object, so merging table fragments does not rewrite the blob
|
||||
payloads.
|
||||
|
||||
<Warning>
|
||||
Use an SDK and Enterprise deployment that support Blob v2 Managed storage.
|
||||
@@ -484,16 +651,10 @@ assert by_id[1] == b""
|
||||
assert len(by_id[2]) == 102400
|
||||
```
|
||||
|
||||
`None` is a null value; `b""` is a present, empty blob. `fetch_blobs` returns
|
||||
bytes in the order of the supplied row IDs. It materializes the selected
|
||||
payloads, so fetch bounded batches when blobs are large. Row IDs belong to
|
||||
their table; fetch them from the same table and snapshot that produced the
|
||||
query result.
|
||||
|
||||
Keep descriptors opaque in application code. Their representation can differ
|
||||
between inline values, Managed objects, and externally referenced blobs. For
|
||||
streaming and range reads, see the
|
||||
[Lance blob guide](https://lance.org/guide/blob/).
|
||||
`fetch_blobs` materializes the selected payloads in memory, so fetch bounded
|
||||
batches when blobs are large. Row IDs belong to their table and snapshot. Keep
|
||||
descriptors opaque in application code, because their representation can differ
|
||||
between inline values and Managed objects.
|
||||
|
||||
#### Derived blobs and maintenance {#managed-blobs-lifecycle}
|
||||
|
||||
@@ -509,92 +670,190 @@ any of its contained blobs is still referenced.
|
||||
|
||||
### Legacy large-binary blobs {#multimodal-large-blobs-blob-api}
|
||||
|
||||
The schema below describes the earlier large-binary Blob API. It is distinct
|
||||
from the `lancedb.blob()` schema above; changing the field's metadata alone does
|
||||
not migrate an existing column to Blob v2.
|
||||
|
||||
For larger files like high-resolution images or videos, Lance provides a specialized **Blob API**. By using a large-binary Arrow type (`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and `DataType::LargeBinary` in Rust) and specific metadata, you enable **lazy loading** and optimized encoding. This allows you to work with massive datasets without loading all binary data into memory upfront.
|
||||
Use [`lancedb.blob()`](#multimodal-blob-v2) for new tables. The sections below
|
||||
cover existing tables that use the V1 large-binary Blob API
|
||||
(`pa.large_binary()` in Python, `LargeBinary` in TypeScript, and
|
||||
`DataType::LargeBinary` in Rust with `{"lance-encoding:blob": "true"}` field
|
||||
metadata). Changing a field's metadata alone does not migrate an existing V1
|
||||
column to Blob v2.
|
||||
|
||||
#### 1\. Define a blob schema {#multimodal-1-define-a-blob-schema}
|
||||
|
||||
To use the Blob API, you must mark the column with `{"lance-encoding:blob": "true"}` metadata.
|
||||
V1 blob columns use a `large_binary` field marked with `{"lance-encoding:blob": "true"}` metadata:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiSchema}
|
||||
{LegacyBlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiSchema}
|
||||
{TsLegacyBlobApiSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiSchema}
|
||||
{RsLegacyBlobApiSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 2\. Ingest large blobs {#multimodal-2-ingest-large-blobs}
|
||||
|
||||
You can then ingest data normally, and Lance will handle the optimized storage.
|
||||
Write raw bytes to a V1 blob column:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiIngest}
|
||||
{LegacyBlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsBlobApiIngest}
|
||||
{TsLegacyBlobApiIngest}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsBlobApiIngest}
|
||||
{RsLegacyBlobApiIngest}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
<Card>
|
||||
For more advanced usage, including random access and file-like reading of blobs, see the
|
||||
Lance format's [blob API documentation](https://lance.org/guide/blob/).
|
||||
</Card>
|
||||
|
||||
#### 3\. Convert blob tables to pandas {#multimodal-3-convert-blob-tables-to-pandas}
|
||||
|
||||
When you call `to_pandas()` on a local LanceDB table that contains Blob API columns, the `blob_mode` argument controls how those columns materialize. This is available in the Python SDK on local tables; remote tables raise `NotImplementedError`.
|
||||
|
||||
`blob_mode` accepts:
|
||||
|
||||
- `"lazy"` (default): returns blob columns as lazy `BlobFile` objects without eagerly materializing their payloads. Use this when you want to stream blob bytes on demand or only inspect a subset of rows. Namespace-backed local tables also use the Lance native blob-aware pandas conversion for lazy blobs; in-memory datasets fall back to the standard PyArrow `to_pandas()` path.
|
||||
- `"bytes"`: eagerly materializes each blob as `bytes`. Use this when you need the raw payload in the DataFrame, for example to decode an image or audio clip in-process.
|
||||
- `"descriptions"`: returns blob descriptors (offsets, sizes, and positions) instead of the data itself. Use this when you want to plan I/O without paying the cost of loading every blob.
|
||||
|
||||
`"bytes"` is not supported on in-memory tables.
|
||||
|
||||
Extra keyword arguments are forwarded to the underlying PyArrow / Lance pandas conversion, so you can also pass options like `split_blocks` or `self_destruct`:
|
||||
On local tables with V1 blob columns, `to_pandas()` in Python accepts `blob_mode`
|
||||
(`"lazy"`, `"bytes"`, or `"descriptions"`). Remote tables raise
|
||||
`NotImplementedError`, and `"bytes"` is not supported on in-memory tables.
|
||||
Extra keyword arguments such as `split_blocks` or `self_destruct` are forwarded
|
||||
to the PyArrow / Lance pandas conversion:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{BlobApiToPandas}
|
||||
{LegacyBlobApiToPandas}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
Query builders also accept `blob_mode` on their `to_pandas()` method:
|
||||
|
||||
- Plain scans support `"lazy"`, `"bytes"`, and `"descriptions"` with filters, projections, aliases, `limit`, and `offset`.
|
||||
- Vector, FTS, hybrid, and ordered queries can't materialize blob columns through `to_pandas()`; omit blob columns from the projection for those query shapes.
|
||||
- This works on both sync and async query builders. Extra PyArrow kwargs like `split_blocks` and `self_destruct` are still forwarded.
|
||||
On query builders with V1 blob columns, plain scans support `"lazy"`, `"bytes"`,
|
||||
and `"descriptions"`. Vector, FTS, hybrid, and ordered queries cannot
|
||||
materialize V1 blob columns through `to_pandas()`, so omit those columns from
|
||||
the projection:
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{QueryToPandasKwargs}
|
||||
{LegacyQueryToPandasKwargs}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Store binary data {#multimodal-store-binary-data}
|
||||
|
||||
Use a `binary` column when the query should return the raw bytes. Declare it as `pa.binary()` in Python, `Binary` in TypeScript, and `DataType::Binary` in Rust.
|
||||
|
||||
#### 1\. Setup and imports {#multimodal-1-setup-and-imports}
|
||||
|
||||
Import LanceDB and Arrow in your SDK.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{MultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsMultimodalImports}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsMultimodalImports}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 2\. Prepare data {#multimodal-2-prepare-data}
|
||||
|
||||
Encode each image or file into raw bytes before writing.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{CreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsCreateDummyData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsCreateDummyData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 3\. Define the schema {#multimodal-3-define-the-schema}
|
||||
|
||||
Declare the schema explicitly so Arrow and LanceDB store the field as `binary` instead of inferring a string or integer list.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{DefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsDefineSchema}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsDefineSchema}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### 4\. Ingest data {#multimodal-4-ingest-data}
|
||||
|
||||
Create the table from the rows and schema.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{IngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsIngestData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsIngestData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Retrieve binary data {#multimodal-retrieve-and-use-blobs}
|
||||
|
||||
When you search the table, select the `binary` column alongside your metadata columns.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{SearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsSearchData}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsSearchData}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
#### Convert bytes back to objects {#multimodal-convert-bytes-back-to-objects}
|
||||
|
||||
Decode the returned bytes into an image or media buffer in your application.
|
||||
|
||||
<CodeGroup>
|
||||
<CodeBlock filename="Python" language="Python" icon="python">
|
||||
{ProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
|
||||
{TsProcessResults}
|
||||
</CodeBlock>
|
||||
|
||||
<CodeBlock filename="Rust" language="Rust" icon="rust">
|
||||
{RsProcessResults}
|
||||
</CodeBlock>
|
||||
</CodeGroup>
|
||||
|
||||
### Other modalities {#multimodal-other-modalities}
|
||||
|
||||
The `pa.binary()` and `pa.large_binary()` types are universal. You can use this same pattern for other types of multimodal data:
|
||||
|
||||
- **Audio:** Read `.wav` or `.mp3` files as bytes.
|
||||
- **Video:** Store video transitions or full clips using the Blob API.
|
||||
- **PDFs/Documents:** Store the raw file content for document search.
|
||||
For PDFs, audio recordings, and other files, keep searchable fields in separate
|
||||
columns so you can filter rows before fetching blob bytes.
|
||||
|
||||
### bfloat16 {#multimodal-bfloat16}
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
{/* Auto-generated by scripts/mdx_snippets_gen.py. Do not edit manually. */}
|
||||
|
||||
export const PyBlobApiToPandas = "df_lazy = tbl.to_pandas()\n\ndf_bytes = tbl.to_pandas(blob_mode=\"bytes\")\n\ndf_desc = tbl.to_pandas(blob_mode=\"descriptions\")\n\narrow_bytes = tbl.to_arrow(blob_mode=\"bytes\")\n\ndf_typed = tbl.to_pandas(split_blocks=True, self_destruct=True)\n";
|
||||
|
||||
export const PyQueryToPandasKwargs = "df_lazy = (\n tbl.search().where(\"id = 1\").select([\"id\", \"video\"]).to_pandas(blob_mode=\"lazy\")\n)\n\ndf_bytes = await (\n tbl_async.query()\n .where(\"id = 1\")\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"bytes\")\n)\n\ndf_vec = (\n tbl.search(query_vector)\n .limit(2)\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"bytes\", split_blocks=True, self_destruct=True)\n)\n";
|
||||
|
||||
@@ -1,28 +1,52 @@
|
||||
{/* Auto-generated by scripts/mdx_snippets_gen.py. Do not edit manually. */}
|
||||
|
||||
export const PyBlobApiIngest = "import lancedb\n\ndb = lancedb.connect(db_path_factory(\"blob_db\"))\n \n# Create sample data\ndata = [\n {\"id\": 1, \"video\": b\"fake_video_bytes_1\"},\n {\"id\": 2, \"video\": b\"fake_video_bytes_2\"}\n]\n \n# Create the table\ntbl = db.create_table(\"videos\", data=data, schema=schema)\n";
|
||||
export const PyBlobApiBytes = "rows = (\n tbl.search().where(\"id = 1\").select([\"id\", \"image\"]).to_list(blob_mode=\"bytes\")\n)\n";
|
||||
|
||||
export const PyBlobApiSchema = "import pyarrow as pa\n\n# Define schema with Blob API metadata for lazy loading\nschema = pa.schema([\n pa.field(\"id\", pa.int64()),\n pa.field(\n \"video\", \n pa.large_binary(), \n metadata={\"lance-encoding:blob\": \"true\"} # Enable Blob API\n ),\n])\n";
|
||||
export const PyBlobApiFetch = "rows = tbl.search().select([\"id\"]).with_row_id(True).to_list()\nrow_ids = [row[\"_rowid\"] for row in rows]\npayloads = tbl.fetch_blobs(\"image\", row_ids)\n";
|
||||
|
||||
export const PyBlobApiToPandas = "# Default: blob columns come back lazily\ndf_lazy = tbl.to_pandas()\n\n# Materialize blob bytes eagerly\ndf_bytes = tbl.to_pandas(blob_mode=\"bytes\")\n\n# Return descriptors instead of payloads\ndf_desc = tbl.to_pandas(blob_mode=\"descriptions\")\n\n# Forward extra kwargs to PyArrow's to_pandas\ndf_typed = tbl.to_pandas(split_blocks=True, self_destruct=True)\n";
|
||||
export const PyBlobApiFiles = "from PIL import ExifTags\n\nrows = tbl.search().where(\"id = 1\").select([\"label\"]).with_row_id(True).to_list()\nwith tbl.fetch_blob_files(\"image\", [rows[0][\"_rowid\"]])[0] as image_file:\n make = Image.open(image_file).getexif().get(ExifTags.Base.Make)\nprint(rows[0][\"label\"], make)\n";
|
||||
|
||||
export const PyCreateDummyData = "# Create some dummy images\ndef create_dummy_image(color):\n img = Image.new('RGB', (100, 100), color=color)\n buf = io.BytesIO()\n img.save(buf, format='PNG')\n return buf.getvalue()\n\n# Create dataset with metadata, vectors, and image blobs\ndata = [\n {\n \"id\": 1,\n \"filename\": \"red_square.png\",\n \"vector\": np.random.rand(128).astype(np.float32),\n \"image_blob\": create_dummy_image('red'),\n \"label\": \"red\"\n },\n {\n \"id\": 2,\n \"filename\": \"blue_square.png\",\n \"vector\": np.random.rand(128).astype(np.float32),\n \"image_blob\": create_dummy_image('blue'),\n \"label\": \"blue\"\n }\n]\n";
|
||||
export const PyBlobApiIngest = "import io\nfrom PIL import Image\n\ndef jpeg_with_make(color, make):\n image = Image.new(\"RGB\", (8, 8), color=color)\n exif = Image.Exif()\n exif[0x010F] = make\n buf = io.BytesIO()\n image.save(buf, format=\"JPEG\", exif=exif)\n return buf.getvalue()\n\ntbl = db.create_table(\n \"photos\",\n data=[\n {\"id\": 1, \"label\": \"red\", \"image\": jpeg_with_make(\"red\", \"TestCam\")},\n {\"id\": 2, \"label\": \"blue\", \"image\": jpeg_with_make(\"blue\", \"OtherCam\")},\n ],\n schema=schema,\n mode=\"overwrite\",\n)\n";
|
||||
|
||||
export const PyDefineSchema = "# Define schema explicitly to ensure image_blob is treated as binary\nschema = pa.schema([\n pa.field(\"id\", pa.int32()),\n pa.field(\"filename\", pa.string()),\n pa.field(\"vector\", pa.list_(pa.float32(), 128)),\n pa.field(\"image_blob\", pa.binary()), # Important: Use pa.binary() for blobs\n pa.field(\"label\", pa.string())\n])\n";
|
||||
export const PyBlobApiRanges = "slices = tbl.fetch_blob_ranges(\"image\", [(row_ids[0], 0, 2)])\n\nwith tbl.fetch_blob_files(\"image\", [row_ids[0]])[0] as image_file:\n parts = image_file.read_ranges([(0, 2)])\n";
|
||||
|
||||
export const PyBlobApiSchema = "import lancedb\nimport pyarrow as pa\n\nschema = pa.schema(\n [\n pa.field(\"id\", pa.int64()),\n pa.field(\"label\", pa.string()),\n lancedb.blob(\"image\"),\n ]\n)\n";
|
||||
|
||||
export const PyBlobApiSetup = "import lancedb\n\ndb = lancedb.connect(\"./blob_db\")\n";
|
||||
|
||||
export const PyCreateDummyData = "def create_dummy_image(color):\n img = Image.new(\"RGB\", (100, 100), color=color)\n buf = io.BytesIO()\n img.save(buf, format=\"PNG\")\n return buf.getvalue()\n\ndata = [\n {\n \"id\": 1,\n \"filename\": \"red_square.png\",\n \"vector\": np.random.rand(128).astype(np.float32),\n \"image_blob\": create_dummy_image(\"red\"),\n \"label\": \"red\",\n },\n {\n \"id\": 2,\n \"filename\": \"blue_square.png\",\n \"vector\": np.random.rand(128).astype(np.float32),\n \"image_blob\": create_dummy_image(\"blue\"),\n \"label\": \"blue\",\n },\n]\n";
|
||||
|
||||
export const PyDefineSchema = "schema = pa.schema(\n [\n pa.field(\"id\", pa.int32()),\n pa.field(\"filename\", pa.string()),\n pa.field(\"vector\", pa.list_(pa.float32(), 128)),\n pa.field(\"image_blob\", pa.binary()),\n pa.field(\"label\", pa.string()),\n ]\n)\n";
|
||||
|
||||
export const PyIngestData = "tbl = db.create_table(\"images\", data=data, schema=schema, mode=\"overwrite\")\n";
|
||||
|
||||
export const PyMultimodalImports = "import lancedb\nimport pyarrow as pa\nimport pandas as pd\nimport numpy as np\nimport io\nfrom PIL import Image\n";
|
||||
export const PyLegacyBlobApiIngest = "import lancedb\n\ndb = lancedb.connect(\"./blob_db\")\n\n# Create sample data\ndata = [\n {\"id\": 1, \"video\": b\"fake_video_bytes_1\"},\n {\"id\": 2, \"video\": b\"fake_video_bytes_2\"},\n]\n\n# Create the table\ntbl = db.create_table(\"videos\", data=data, schema=schema)\n";
|
||||
|
||||
export const PyProcessResults = "# Convert back to PIL Image\nfor _, row in results.iterrows():\n image_bytes = row['image_blob']\n image = Image.open(io.BytesIO(image_bytes))\n print(f\"Retrieved image: {row['filename']}, Size: {image.size}\")\n # You can now use 'image' with other libraries or display it\n";
|
||||
export const PyLegacyBlobApiSchema = "import pyarrow as pa\n\n# Define schema with Blob API metadata for lazy loading\nschema = pa.schema(\n [\n pa.field(\"id\", pa.int64()),\n pa.field(\n \"video\",\n pa.large_binary(),\n metadata={\"lance-encoding:blob\": \"true\"}, # Enable Blob API\n ),\n ]\n)\n";
|
||||
|
||||
export const PyQueryToPandasKwargs = "# Plain scan query: blob_mode is supported end to end\ndf_lazy = (\n tbl.search()\n .where(\"id = 1\")\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"lazy\")\n)\n\n# Same call shape works on async query builders\ndf_bytes = await (\n tbl_async.query()\n .where(\"id = 1\")\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"bytes\")\n)\n\n# Vector / FTS / hybrid queries can't materialize blob columns,\n# so omit them from the projection\ndf_vec = (\n tbl.search(query_vector)\n .limit(10)\n .select([\"id\", \"vector\"])\n .to_pandas(split_blocks=True, self_destruct=True)\n)\n";
|
||||
export const PyLegacyBlobApiToPandas = "# Default: blob columns come back lazily\ndf_lazy = tbl.to_pandas()\n\n# Materialize blob bytes eagerly\ndf_bytes = tbl.to_pandas(blob_mode=\"bytes\")\n\n# Return descriptors instead of payloads\ndf_desc = tbl.to_pandas(blob_mode=\"descriptions\")\n\n# Forward extra kwargs to PyArrow's to_pandas\ndf_typed = tbl.to_pandas(split_blocks=True, self_destruct=True)\n";
|
||||
|
||||
export const PySearchData = "# Search for similar images\nquery_vector = np.random.rand(128).astype(np.float32)\nresults = tbl.search(query_vector).limit(1).to_pandas()\n";
|
||||
export const PyLegacyQueryToPandasKwargs = "# Plain scan query: blob_mode is supported end to end\ndf_lazy = (\n tbl.search()\n .where(\"id = 1\")\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"lazy\")\n)\n\n# Same call shape works on async query builders\ndf_bytes = await (\n tbl_async.query()\n .where(\"id = 1\")\n .select([\"id\", \"video\"])\n .to_pandas(blob_mode=\"bytes\")\n)\n\n# Vector / FTS / hybrid queries can't materialize blob columns,\n# so omit them from the projection\ndf_vec = (\n tbl.search(query_vector)\n .limit(10)\n .select([\"id\", \"vector\"])\n .to_pandas(split_blocks=True, self_destruct=True)\n)\n";
|
||||
|
||||
export const TsBlobApiIngest = "const blobData = lancedb.makeArrowTable(\n [\n { id: 1, video: Buffer.from(\"fake_video_bytes_1\") },\n { id: 2, video: Buffer.from(\"fake_video_bytes_2\") },\n ],\n { schema: blobSchema },\n);\nconst blobTable = await db.createTable(\"videos\", blobData, {\n mode: \"overwrite\",\n});\n";
|
||||
export const PyMultimodalImports = "import io\nimport lancedb\nimport numpy as np\nimport pyarrow as pa\nfrom PIL import Image\n";
|
||||
|
||||
export const TsBlobApiSchema = "const blobSchema = new arrow.Schema([\n new arrow.Field(\"id\", new arrow.Int64()),\n new arrow.Field(\n \"video\",\n new arrow.LargeBinary(),\n true,\n new Map([[\"lance-encoding:blob\", \"true\"]]),\n ),\n]);\n";
|
||||
export const PyProcessResults = "for _, row in results.iterrows():\n image_bytes = row[\"image_blob\"]\n image = Image.open(io.BytesIO(image_bytes))\n print(f\"Retrieved image: {row['filename']}, Size: {image.size}\")\n";
|
||||
|
||||
export const PySearchData = "query_vector = np.random.rand(128).astype(np.float32)\nresults = tbl.search(query_vector).limit(1).to_pandas()\n";
|
||||
|
||||
export const TsBlobApiBytesMode = "const withBytes = await blobTable\n .query()\n .select([\"id\", \"image\"])\n .toArray({ blobMode: \"bytes\" });\nconst firstImage: Uint8Array | null = withBytes[0].image;\n";
|
||||
|
||||
export const TsBlobApiFetch = "const payloads = await blobTable.fetchBlobs(\"image\", rowIds);\n";
|
||||
|
||||
export const TsBlobApiFiles = "const rows = await blobTable\n .query()\n .select([\"id\", \"label\"])\n .withRowId()\n .toArray();\nconst rowIds = rows.map((row) => row._rowid as bigint);\n\nconst [image] = await blobTable.fetchBlobFiles(\"image\", [rowIds[0]!]);\nconst header = await image!.readRange(0n, 6n);\nawait image!.close();\n";
|
||||
|
||||
export const TsBlobApiIngest = "const blobTable = await db.createEmptyTable(\"photos\", blobSchema, {\n mode: \"overwrite\",\n});\n\nawait blobTable.add([\n { id: 1n, label: \"door\", image: Buffer.from(\"inline_image_bytes\") },\n { id: 2n, label: \"window\", image: Buffer.from(\"second_image_bytes\") },\n]);\n";
|
||||
|
||||
export const TsBlobApiRanges = "const [secondImage] = await blobTable.fetchBlobFiles(\"image\", [rowIds[1]!]);\nconst parts = await secondImage!.readRanges([\n { start: 0n, end: 6n },\n { start: 7n, end: 12n },\n]);\nawait secondImage!.close();\n";
|
||||
|
||||
export const TsBlobApiSchema = "const blobSchema = new arrow.Schema([\n new arrow.Field(\"id\", new arrow.Int64()),\n new arrow.Field(\"label\", new arrow.Utf8()),\n lancedb.blob(\"image\"),\n]);\n";
|
||||
|
||||
export const TsBlobApiSetup = "const db = await lancedb.connect(\"./blob_db\");\n";
|
||||
|
||||
export const TsCreateDummyData = "const createDummyImage = (color: string): Uint8Array => {\n const pngHeader = Uint8Array.from([137, 80, 78, 71, 13, 10, 26, 10]);\n return Buffer.concat([Buffer.from(pngHeader), Buffer.from(color, \"utf8\")]);\n};\n\nconst data = [\n {\n id: 1,\n filename: \"red_square.png\",\n vector: Array.from({ length: 128 }, (_, i) => (i % 16) / 16),\n image_blob: createDummyImage(\"red\"),\n label: \"red\",\n },\n {\n id: 2,\n filename: \"blue_square.png\",\n vector: Array.from({ length: 128 }, (_, i) => ((i + 8) % 16) / 16),\n image_blob: createDummyImage(\"blue\"),\n label: \"blue\",\n },\n];\n";
|
||||
|
||||
@@ -30,23 +54,41 @@ export const TsDefineSchema = "const schema = new arrow.Schema([\n new arrow.Fi
|
||||
|
||||
export const TsIngestData = "const multimodalData = lancedb.makeArrowTable(data, { schema });\nconst tbl = await db.createTable(\"images\", multimodalData, {\n mode: \"overwrite\",\n});\n";
|
||||
|
||||
export const TsLegacyBlobApiIngest = "const legacyBlobData = lancedb.makeArrowTable(\n [\n { id: 1, video: Buffer.from(\"fake_video_bytes_1\") },\n { id: 2, video: Buffer.from(\"fake_video_bytes_2\") },\n ],\n { schema: legacyBlobSchema },\n);\nconst legacyBlobTable = await db.createTable(\n \"legacy_videos\",\n legacyBlobData,\n {\n mode: \"overwrite\",\n },\n);\n";
|
||||
|
||||
export const TsLegacyBlobApiSchema = "const legacyBlobSchema = new arrow.Schema([\n new arrow.Field(\"id\", new arrow.Int64()),\n new arrow.Field(\n \"video\",\n new arrow.LargeBinary(),\n true,\n new Map([[\"lance-encoding:blob\", \"true\"]]),\n ),\n]);\n";
|
||||
|
||||
export const TsMultimodalImports = "import * as arrow from \"apache-arrow\";\nimport { Buffer } from \"node:buffer\";\nimport * as lancedb from \"@lancedb/lancedb\";\n";
|
||||
|
||||
export const TsProcessResults = "for (const row of results) {\n const imageBytes = row.image_blob as Uint8Array;\n console.log(\n `Retrieved image: ${row.filename}, Byte length: ${imageBytes.length}`,\n );\n}\n";
|
||||
|
||||
export const TsSearchData = "const queryVector = Array.from({ length: 128 }, (_, i) => (i % 16) / 16);\nconst results = await tbl.search(queryVector).limit(1).toArray();\n";
|
||||
|
||||
export const RsBlobApiIngest = "let blob_rows = vec![\n (1_i64, b\"fake_video_bytes_1\".to_vec()),\n (2_i64, b\"fake_video_bytes_2\".to_vec()),\n];\n\nlet blob_schema = Arc::new(blob_schema);\nlet blob_batch = RecordBatch::try_new(\n blob_schema.clone(),\n vec![\n Arc::new(Int64Array::from_iter_values(blob_rows.iter().map(|row| row.0))),\n Arc::new(LargeBinaryArray::from_iter_values(\n blob_rows.iter().map(|row| row.1.as_slice()),\n )),\n ],\n)\n.unwrap();\nlet blob_reader: Box<dyn arrow_array::RecordBatchReader + Send> =\n Box::new(RecordBatchIterator::new(vec![Ok(blob_batch)].into_iter(), blob_schema));\nlet blob_table = db\n .create_table(\"videos\", blob_reader)\n .mode(CreateTableMode::Overwrite)\n .execute()\n .await\n .unwrap();\n";
|
||||
export const RsBlobApiFetch = "let payloads = blob_table.fetch_blobs(\"image\", &row_ids).await.unwrap();\n";
|
||||
|
||||
export const RsBlobApiSchema = "let blob_metadata = HashMap::from([(\n \"lance-encoding:blob\".to_string(),\n \"true\".to_string(),\n)]);\nlet blob_schema = Schema::new(vec![\n Field::new(\"id\", DataType::Int64, false),\n Field::new(\"video\", DataType::LargeBinary, true).with_metadata(blob_metadata),\n]);\n";
|
||||
export const RsBlobApiFiles = "let batches = blob_table\n .query()\n .with_row_id()\n .execute()\n .await\n .unwrap()\n .try_collect::<Vec<_>>()\n .await\n .unwrap();\nlet row_ids = batches[0]\n .column_by_name(\"_rowid\")\n .unwrap()\n .as_any()\n .downcast_ref::<UInt64Array>()\n .unwrap()\n .values()\n .to_vec();\n\nlet handles = blob_table\n .fetch_blob_files(\"image\", &row_ids[..1])\n .await\n .unwrap();\nlet header = handles[0].as_ref().unwrap().read_range(0..6).await.unwrap();\nhandles[0].as_ref().unwrap().close().await.unwrap();\n";
|
||||
|
||||
export const RsBlobApiImports = "use std::sync::Arc;\n\nuse arrow_array::{\n Array, Int64Array, LargeBinaryArray, RecordBatch, RecordBatchIterator, StringArray, UInt64Array,\n};\nuse arrow_schema::{DataType, Field, Schema};\nuse futures_util::TryStreamExt;\nuse lancedb::blob::{BlobRangeRequest, blob};\nuse lancedb::connect;\nuse lancedb::database::CreateTableMode;\nuse lancedb::query::{ExecutableQuery, QueryBase};\n";
|
||||
|
||||
export const RsBlobApiIngest = "let blob_table = db\n .create_empty_table(\"photos\", blob_schema)\n .mode(CreateTableMode::Overwrite)\n .execute()\n .await\n .unwrap();\n\nlet input_schema = Arc::new(Schema::new(vec![\n Field::new(\"id\", DataType::Int64, false),\n Field::new(\"label\", DataType::Utf8, true),\n Field::new(\"image\", DataType::LargeBinary, true),\n]));\nlet input_batch = RecordBatch::try_new(\n input_schema.clone(),\n vec![\n Arc::new(Int64Array::from(vec![1_i64, 2_i64])),\n Arc::new(StringArray::from(vec![Some(\"door\"), Some(\"window\")])),\n Arc::new(LargeBinaryArray::from_iter_values([\n b\"inline_image_bytes\".as_slice(),\n b\"second_image_bytes\".as_slice(),\n ])),\n ],\n)\n.unwrap();\nlet input_reader: Box<dyn arrow_array::RecordBatchReader + Send> = Box::new(\n RecordBatchIterator::new(vec![Ok(input_batch)].into_iter(), input_schema),\n);\nblob_table.add(input_reader).execute().await.unwrap();\n";
|
||||
|
||||
export const RsBlobApiRanges = "let slices = blob_table\n .fetch_blob_ranges(\n \"image\",\n [\n BlobRangeRequest::new(row_ids[0], 0, 6),\n BlobRangeRequest::new(row_ids[1], 0, 6),\n BlobRangeRequest::new(row_ids[1], 7, 5),\n ],\n )\n .await\n .unwrap();\n\nlet second_handles = blob_table\n .fetch_blob_files(\"image\", &row_ids[1..2])\n .await\n .unwrap();\nlet second_image = second_handles[0].as_ref().unwrap();\nlet parts = second_image.read_ranges(&[0..6, 7..12]).await.unwrap();\nsecond_image.close().await.unwrap();\n";
|
||||
|
||||
export const RsBlobApiSchema = "let blob_schema = Arc::new(Schema::new(vec![\n Field::new(\"id\", DataType::Int64, false),\n Field::new(\"label\", DataType::Utf8, true),\n blob(\"image\", true),\n]));\n";
|
||||
|
||||
export const RsBlobApiSetup = "let db = connect(\"./blob_db\").execute().await.unwrap();\n";
|
||||
|
||||
export const RsCreateDummyData = "let create_dummy_image = |color: u8| -> Vec<u8> {\n let mut png_like = vec![137, 80, 78, 71, 13, 10, 26, 10];\n png_like.push(color);\n png_like\n};\n\nlet data = vec![\n (\n 1_i32,\n \"red_square.png\",\n vec![0.1_f32; 128],\n create_dummy_image(1),\n \"red\",\n ),\n (\n 2_i32,\n \"blue_square.png\",\n vec![0.2_f32; 128],\n create_dummy_image(2),\n \"blue\",\n ),\n];\n";
|
||||
|
||||
export const RsDefineSchema = "let schema = Schema::new(vec![\n Field::new(\"id\", DataType::Int32, false),\n Field::new(\"filename\", DataType::Utf8, false),\n Field::new(\n \"vector\",\n DataType::FixedSizeList(Arc::new(Field::new(\"item\", DataType::Float32, true)), 128),\n false,\n ),\n Field::new(\"image_blob\", DataType::Binary, false),\n Field::new(\"label\", DataType::Utf8, false),\n]);\n";
|
||||
|
||||
export const RsIngestData = "let schema = Arc::new(schema);\nlet image_batch = RecordBatch::try_new(\n schema.clone(),\n vec![\n Arc::new(Int32Array::from_iter_values(data.iter().map(|row| row.0))),\n Arc::new(StringArray::from_iter_values(data.iter().map(|row| row.1))),\n Arc::new(\n FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(\n data.iter()\n .map(|row| Some(row.2.iter().copied().map(Some).collect::<Vec<_>>())),\n 128,\n ),\n ),\n Arc::new(BinaryArray::from_iter_values(\n data.iter().map(|row| row.3.as_slice()),\n )),\n Arc::new(StringArray::from_iter_values(data.iter().map(|row| row.4))),\n ],\n)\n.unwrap();\nlet image_reader: Box<dyn arrow_array::RecordBatchReader + Send> =\n Box::new(RecordBatchIterator::new(vec![Ok(image_batch)].into_iter(), schema.clone()));\nlet table = db\n .create_table(\"images\", image_reader)\n .mode(CreateTableMode::Overwrite)\n .execute()\n .await\n .unwrap();\n";
|
||||
export const RsIngestData = "let schema = Arc::new(schema);\nlet image_batch = RecordBatch::try_new(\n schema.clone(),\n vec![\n Arc::new(Int32Array::from_iter_values(data.iter().map(|row| row.0))),\n Arc::new(StringArray::from_iter_values(data.iter().map(|row| row.1))),\n Arc::new(\n FixedSizeListArray::from_iter_primitive::<Float32Type, _, _>(\n data.iter()\n .map(|row| Some(row.2.iter().copied().map(Some).collect::<Vec<_>>())),\n 128,\n ),\n ),\n Arc::new(BinaryArray::from_iter_values(\n data.iter().map(|row| row.3.as_slice()),\n )),\n Arc::new(StringArray::from_iter_values(data.iter().map(|row| row.4))),\n ],\n)\n.unwrap();\nlet image_reader: Box<dyn arrow_array::RecordBatchReader + Send> = Box::new(\n RecordBatchIterator::new(vec![Ok(image_batch)].into_iter(), schema.clone()),\n);\nlet table = db\n .create_table(\"images\", image_reader)\n .mode(CreateTableMode::Overwrite)\n .execute()\n .await\n .unwrap();\n";
|
||||
|
||||
export const RsMultimodalImports = "use std::collections::HashMap;\nuse std::sync::Arc;\n\nuse arrow_array::types::Float32Type;\nuse arrow_array::{\n BinaryArray, FixedSizeListArray, Int32Array, Int64Array, LargeBinaryArray, RecordBatch,\n RecordBatchIterator, StringArray,\n};\nuse arrow_schema::{DataType, Field, Schema};\nuse futures_util::TryStreamExt;\nuse lancedb::connect;\nuse lancedb::database::CreateTableMode;\nuse lancedb::query::{ExecutableQuery, QueryBase};\n";
|
||||
export const RsLegacyBlobApiIngest = "let legacy_blob_rows = vec![\n (1_i64, b\"fake_video_bytes_1\".to_vec()),\n (2_i64, b\"fake_video_bytes_2\".to_vec()),\n];\n\nlet legacy_blob_schema = Arc::new(legacy_blob_schema);\nlet legacy_blob_batch = RecordBatch::try_new(\n legacy_blob_schema.clone(),\n vec![\n Arc::new(Int64Array::from_iter_values(\n legacy_blob_rows.iter().map(|row| row.0),\n )),\n Arc::new(LargeBinaryArray::from_iter_values(\n legacy_blob_rows.iter().map(|row| row.1.as_slice()),\n )),\n ],\n)\n.unwrap();\nlet legacy_blob_reader: Box<dyn arrow_array::RecordBatchReader + Send> = Box::new(\n RecordBatchIterator::new(vec![Ok(legacy_blob_batch)].into_iter(), legacy_blob_schema),\n);\nlet legacy_blob_table = db\n .create_table(\"legacy_videos\", legacy_blob_reader)\n .mode(CreateTableMode::Overwrite)\n .execute()\n .await\n .unwrap();\n";
|
||||
|
||||
export const RsLegacyBlobApiSchema = "let legacy_blob_metadata = HashMap::from([(\n \"lance-encoding:blob\".to_string(),\n \"true\".to_string(),\n)]);\nlet legacy_blob_schema = Schema::new(vec![\n Field::new(\"id\", DataType::Int64, false),\n Field::new(\"video\", DataType::LargeBinary, true).with_metadata(legacy_blob_metadata),\n]);\n";
|
||||
|
||||
export const RsMultimodalImports = "use std::collections::HashMap;\nuse std::sync::Arc;\n\nuse arrow_array::types::Float32Type;\nuse arrow_array::{\n Array, BinaryArray, FixedSizeListArray, Int32Array, Int64Array, LargeBinaryArray, RecordBatch,\n RecordBatchIterator, StringArray,\n};\nuse arrow_schema::{DataType, Field, Schema};\nuse futures_util::TryStreamExt;\nuse lancedb::connect;\nuse lancedb::database::CreateTableMode;\nuse lancedb::query::{ExecutableQuery, QueryBase};\n";
|
||||
|
||||
export const RsProcessResults = "for batch in &results {\n let filenames = batch\n .column_by_name(\"filename\")\n .unwrap()\n .as_any()\n .downcast_ref::<StringArray>()\n .unwrap();\n let images = batch\n .column_by_name(\"image_blob\")\n .unwrap()\n .as_any()\n .downcast_ref::<BinaryArray>()\n .unwrap();\n\n for row in 0..batch.num_rows() {\n let image_bytes = images.value(row);\n println!(\n \"Retrieved image: {}, Byte length: {}\",\n filenames.value(row),\n image_bytes.len()\n );\n }\n}\n";
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "SQL data types"
|
||||
sidebarTitle: "Data Types"
|
||||
description: "How Lance columns appear to SQL, including the types SQL has no name for."
|
||||
icon: shapes
|
||||
keywords: ["sql types", "arrow", "cast", "pseudo-columns", "vector", "blob", "json"]
|
||||
keywords: ["sql types", "arrow", "cast", "pseudo-columns", "vector", "json"]
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
|
||||
@@ -61,7 +61,7 @@ describes the full mapping.
|
||||
|
||||
## Types SQL has no name for {#no-name}
|
||||
|
||||
Three column kinds you will meet that no SQL type name covers.
|
||||
Some column kinds have no SQL type name.
|
||||
|
||||
### Vectors {#vectors}
|
||||
|
||||
@@ -94,17 +94,6 @@ WHERE json_get_int(data, 'status') = 200
|
||||
`json_get`, its typed variants, and `json_extract` are **index-accelerated** by a
|
||||
JSON scalar index. See [Functions](/sql/scalar-functions#json).
|
||||
|
||||
### Blobs {#blobs}
|
||||
|
||||
A blob column is read either as its binary contents or as a **description**: a
|
||||
pointer plus metadata, without fetching the bytes. That choice changes the
|
||||
column's Arrow type, and therefore the schema your query sees.
|
||||
|
||||
<Warning>
|
||||
The default reads blob columns as descriptions and other binary columns as
|
||||
binary. A query selecting a blob column and expecting bytes gets a struct.
|
||||
</Warning>
|
||||
|
||||
### bfloat16 {#bfloat16}
|
||||
|
||||
Sixteen-bit floats in the bfloat16 layout, stored as `fixed_size_binary(2)`.
|
||||
|
||||
@@ -16,6 +16,7 @@ CREATE TABLE recent AS SELECT * FROM documents WHERE created_at > now() - INTERV
|
||||
```
|
||||
|
||||
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
||||
|
||||
A table inside a [namespace](/sql/ddl/namespace) is written as a dotted path:
|
||||
`prod.events.clicks`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user