Updates the Rust workspace and Java Lance dependencies to
[v13.1.0-beta.1](https://github.com/lance-format/lance/releases/tag/v13.1.0-beta.1)
and refreshes Cargo.lock. Removes a redundant `pub(crate)` declaration
in the private Node.js blob module to satisfy the required Clippy check.
Validation: `cargo clippy --quiet --workspace --tests --all-features --
-D warnings`, `cargo fmt --all --quiet`, `pnpm build`, and `pnpm test
__test__/blob.test.ts --runInBand` (26 tests passed) in `nodejs`.
---------
Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Closes#4288.
Blob reads in Node can now be cancelled. `BlobFile.read`, `readRange`
and `readRanges`, and `Table.fetchBlobs` and `fetchBlobFiles` take an
optional `{ signal?: AbortSignal }` (new `BlobReadOptions` type). When
the signal aborts, the call rejects with `signal.reason` (an
`AbortError` unless the caller gave a reason) and the native future is
dropped, so the read stops instead of running to completion. A signal
that is already aborted rejects before any read starts.
How it works:
- On the Rust side, these five methods are now sync `#[napi]` methods
that return `env.spawn_future(...)`. The future is wrapped in
`futures::future::abortable`, and the abort handle is registered with
napi's `AbortSignal::on_abort`. napi's `AbortSignal` is `Rc`-based, so
it can't be an argument of an `async fn`, which is why the methods
changed shape. The TypeScript signatures are unchanged apart from the
new optional argument.
- napi-rs installs its own `onabort` on the signal it receives, which
would overwrite a handler the caller set. So the TypeScript wrapper
passes a private `AbortController` signal to the native call for each
request, forwards the caller's abort to it with `addEventListener(..., {
once: true })`, and removes the listener when the call settles.
Tests in `table.test.ts`:
- an already aborted signal rejects on all five methods, and the handle
still reads afterwards
- a custom abort reason is what the call rejects with
- a read with a signal that never aborts returns normally and leaves no
`abort` listener on the signal
- aborting a 32 MiB `readRange` right after it starts rejects with
`AbortError`, and the same handle reads again afterwards
The Node docs under `docs/src/js` are regenerated.
I didn't add a test for where the `read()` cursor ends up after a
mid-flight abort on a local table. The remote `BlobFile` documents that
a cancelled read leaves the cursor unchanged, but I haven't confirmed
the same for the native Lance handle, so the docs don't claim it.
Closes#4300.
The TypeScript `BlobFile` only had `size()`, `read()` and `readRange()`,
while the Rust `BlobFile` behind it also has `seek`, `tell`,
`read_up_to`, `read_ranges` and `close`. This exposes those on the Node
binding:
- `read(maxBytes?)` reads at most `maxBytes` from the cursor (Rust
`read_up_to`). Without the argument it reads to the end, as before.
- `readRanges([{ start, end }, ...])` reads several half-open ranges and
returns one buffer per range, in order. Like `readRange`, it does not
move the cursor.
- `seek(position)` and `tell()` move and report the cursor.
- `close()` releases the handle, and `isClosed()` reports it. Every
read, `seek` and `tell` after `close()` fails with "blob file is already
closed". A second `close()` does nothing. The binding keeps its own
`closed` flag, the same way the Python `BlobFile` does.
Offsets and lengths are `bigint`, like `readRange` and `size()`.
`BlobRange` is exported as a type.
The new tests in `table.test.ts` cover chunked reads with
`read(maxBytes)`, `seek` and `tell`, a negative seek position,
`readRanges` order and cursor, a `readRanges` end past the blob size,
and reads after `close()`. The Node docs under `docs/src/js` are
regenerated with `pnpm run docs`.
I left the `ReadableStream`/async iterator idea from the issue out of
this PR. The AbortSignal work in #4288 touches the same methods, so it
can follow on top of this.
Update the Rust workspace Lance dependencies, Cargo lockfile, and Java
lance-core to
[v12.0.0-beta.18](https://github.com/lance-format/lance/releases/tag/v12.0.0-beta.18).
Fix redundant visibility declarations in the Node.js Rust blob helpers
required by the workspace Clippy check.
Validated with workspace Clippy (all features and tests, warnings
denied), cargo fmt, pnpm build, and 13 targeted Node.js blob tests.
this PR blob v2 field helpers and reads to the Node SDK.
`blob()` marks a field as blob v2 and lets you set the storage
thresholds. Inputs can be bytes, a URI, or a data/uri struct.
Queries return descriptors. `fetchBlobs()` reads the bytes by row ID,
and `fetchBlobFiles()` gives you lazy handles for full or range reads.
`blobColumns()` lists the blob fields, including nested ones.
Fetch uses the table’s current checkout. It preserves order, duplicates,
and nulls. Holding row IDs across compaction still requires stable row
IDs.
```javascript
const db = await connect("./data");
const video = await readFile("clip.mp4");
const table = await db.createTable(
"videos",
[{ id: 1n, video }],
{
schema: new Schema([
new Field("id", new Int64()),
blob("video"),
]),
},
);
const rows = await table.query().select(["id"]).withRowId().toArray();
const rowIds = rows.map((row) => row._rowid as bigint);
const bytes = await table.fetchBlobs("video", rowIds);
const [handle] = await table.fetchBlobFiles("video", rowIds);
const header = await handle!.readRange(0n, 65536n);
```
### Testing
- cover input validation, thresholds, nested fields, fetch ordering,
nulls, and range reads.
Bilal Atique
Drew