fix: handle remote blob fetch request caps (#4309)

## Cause

Remote `fetch_blobs` sent every row ID in one Cloud request, exceeding
the server's 1024-row or 64 MiB byte cap. A single blob over the byte
cap could not be read through this method.

## Fix

- Send at most 1024 row IDs per request and split batches further when
the server reports the blob-byte cap, preserving input order and nulls.
- Resolve one exact dataset version before multi-request reads. If an
initial single request hits the byte cap, resolve the version before any
successful split or Range read; use that version for every resulting
request.
- Read an individual oversized blob through the existing Range route.
- Document the per-request caps and transparent handling in the Rust,
Python, and TypeScript APIs and generated JS docs.

## Validation

- Focused Rust `fetch_blobs` tests: 26 passed, including row-count,
byte-cap, Range fallback, and exact-version assertions.
- Rust clippy and formatting passed.
- Focused Python blob tests: 13 passed; Ruff lint and formatting passed.
- Node build, lint, docs generation, and focused `fetchBlobs` test
passed.

Fixes #4275

<!-- lance-gatekeeper-fix:v1 agent=6ae94d49bf1d4cae057bba72584e7b90
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
lancedb-gatefixer[bot]
2026-09-28 20:31:36 -05:00
committed by GitHub
parent c119f0c8e5
commit 8e3606fbe3
6 changed files with 284 additions and 5 deletions
+4 -2
View File
@@ -1963,8 +1963,10 @@ class Table(ABC):
``_rowid`` values stay valid after compaction when the table has stable
row ids.
Convenience for small payloads. For large values use
:meth:`fetch_blob_files`.
Remote servers limit each request to 1024 row IDs and 64 MiB of blob
bytes. The client splits requests automatically and reads an individual
larger blob through the Range route. This method still materializes all
bytes in memory; for large values use :meth:`fetch_blob_files`.
"""
@abstractmethod