mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-21 11:12:27 +00:00
feat(node): add blob v2 fetch and field helpers (#4155)
this PR blob v2 field helpers and reads to the Node SDK.
`blob()` marks a field as blob v2 and lets you set the storage
thresholds. Inputs can be bytes, a URI, or a data/uri struct.
Queries return descriptors. `fetchBlobs()` reads the bytes by row ID,
and `fetchBlobFiles()` gives you lazy handles for full or range reads.
`blobColumns()` lists the blob fields, including nested ones.
Fetch uses the table’s current checkout. It preserves order, duplicates,
and nulls. Holding row IDs across compaction still requires stable row
IDs.
```javascript
const db = await connect("./data");
const video = await readFile("clip.mp4");
const table = await db.createTable(
"videos",
[{ id: 1n, video }],
{
schema: new Schema([
new Field("id", new Int64()),
blob("video"),
]),
},
);
const rows = await table.query().select(["id"]).withRowId().toArray();
const rowIds = rows.map((row) => row._rowid as bigint);
const bytes = await table.fetchBlobs("video", rowIds);
const [handle] = await table.fetchBlobFiles("video", rowIds);
const header = await handle!.readRange(0n, 65536n);
```
### Testing
- cover input validation, thresholds, nested fields, fetch ordering,
nulls, and range reads.
This commit is contained in:
@@ -0,0 +1,55 @@
|
||||
[**@lancedb/lancedb**](../README.md) • **Docs**
|
||||
|
||||
***
|
||||
|
||||
[@lancedb/lancedb](../globals.md) / blob
|
||||
|
||||
# Function: blob()
|
||||
|
||||
```ts
|
||||
function blob(name, options): Field
|
||||
```
|
||||
|
||||
Declares a `lance.blob.v2` column.
|
||||
|
||||
Query results are descriptors, not payload bytes. Use [Table.fetchBlobs](../classes/Table.md#fetchblobs)
|
||||
or [Table.fetchBlobFiles](../classes/Table.md#fetchblobfiles) to read bytes.
|
||||
|
||||
## Parameters
|
||||
|
||||
* **name**: `string`
|
||||
|
||||
* **options**: [`BlobOptions`](../type-aliases/BlobOptions.md) = `{}`
|
||||
|
||||
## Returns
|
||||
|
||||
`Field`
|
||||
|
||||
## Example
|
||||
|
||||
```ts
|
||||
import { readFile } from "node:fs/promises";
|
||||
import { Field, Int64, Schema } from "apache-arrow";
|
||||
import { blob, connect } from "@lancedb/lancedb";
|
||||
|
||||
const db = await connect("./data");
|
||||
const video = await readFile("clip.mp4");
|
||||
const table = await db.createTable(
|
||||
"videos",
|
||||
[{ id: 1n, video }],
|
||||
{
|
||||
schema: new Schema([
|
||||
new Field("id", new Int64()),
|
||||
blob("video"),
|
||||
]),
|
||||
},
|
||||
);
|
||||
|
||||
const rows = await table.query().select(["id"]).withRowId().toArray();
|
||||
const rowIds = rows.map((row) => row._rowid as bigint);
|
||||
const bytes = await table.fetchBlobs("video", rowIds);
|
||||
|
||||
const [handle] = await table.fetchBlobFiles("video", rowIds);
|
||||
const size = handle!.size();
|
||||
const header = await handle!.readRange(0n, size < 65536n ? size : 65536n);
|
||||
```
|
||||
@@ -0,0 +1,22 @@
|
||||
[**@lancedb/lancedb**](../README.md) • **Docs**
|
||||
|
||||
***
|
||||
|
||||
[@lancedb/lancedb](../globals.md) / isBlobField
|
||||
|
||||
# Function: isBlobField()
|
||||
|
||||
```ts
|
||||
function isBlobField(field): boolean
|
||||
```
|
||||
|
||||
Checks for the `lance.blob.v2` extension marker. Does not validate the
|
||||
field's storage type.
|
||||
|
||||
## Parameters
|
||||
|
||||
* **field**: `Field`<`any`>
|
||||
|
||||
## Returns
|
||||
|
||||
`boolean`
|
||||
Reference in New Issue
Block a user