perf: reduce Blob read buffering and expose read window controls (#4979)

Remote Blob reads retain and concatenate intermediate IPC buffers across
batches and HTTP requests, increasing peak memory. Decode fragmented IPC
bodies using their declared length, assemble Blob values once across
batches/requests, and collect large-object Range responses into
known-size buffers. Impossible IPC/Range allocations return errors
instead of panicking. Closing a remote Blob file releases cached bytes
immediately, including while a cross-window read is waiting for the
network.

Rust and Python expose configurable Blob read buffers. Python sync/async
`fetch_blob_files` and `open_blob_file(s)` accept `buffer_size`; the
default is 512 KiB and zero disables prefetch. Explicit ranges bypass
prefetch. New PE batch-endpoint adoption is separate.

Malformed-length and retained-cache regressions fail on the previous
implementation and pass after the correction. Real HTTP tests cover
buffer propagation, close during blocked input, batch/request
aggregation, and large-object fallback. The final extension was rebuilt
before Python validation. Non-Blob Arrow arrays may still require
alignment copies; a contiguous multi-batch result still requires its
final allocation.

## Benchmark

Peak client RSS increment in KiB, using
`python/benchmarks/bench_fetch_blobs_rss.py`, independent clients and
real loopback HTTP. Both extensions were rebuilt from clean, unchanged
source with the same toolchain/configuration; loaded artifact hashes
were checked. Baseline `42f11d13a2755e44b63876b5a5edd93901291150`, after
`0a8a76de5286949d80368f95a5c529adf422c187`.

| Equivalent fetch workload | Payload | Before | After |
| --- | ---: | ---: | ---: |
| Single response, one batch | 48 MiB | 63,812 | 63,896 |
| Multiple responses, one batch per response | 144 MiB | 423,104 |
195,628 |
| Single response, 12 rows per batch | 48 MiB | 130,520 | 71,348 |
| Multiple responses, 12 rows per batch | 144 MiB | 363,144 | 172,748 |
| One oversized row, Range fallback | 70 MiB | 182,816 | 85,548 |

These are one observation per workload and build, not a statistical
distribution or memory upper bound. The single-batch workload shows no
improvement. Cloud/Sophon deployment, credentialed S3 and long-running
stress have not been exercised.
This commit is contained in:
Xuanwo
2026-11-13 02:05:09 +08:00
committed by GitHub
parent 8939ef1f01
commit 10f89b3c32
13 changed files with 1326 additions and 88 deletions
+9
View File
@@ -793,6 +793,15 @@ with `search().with_row_id(True).to_arrow()` and pass the result to the fetch AP
One missing or deleted row ID rejects the entire blob-read batch; the APIs do
not return partial results.
Local and remote `BlobFile` handles prefetch 512 KiB for sequential reads by
default. Pass `buffer_size=0` to `fetch_blob_files`, `open_blob_file`, or
`open_blob_files` for sparse reads, or choose another window size in bytes.
Call `file.set_buffer_size(size)` to change it without moving the cursor;
changing the size discards unused prefetched bytes. Explicit `read_range` and
`read_ranges` calls always read exactly the requested intervals and leave the
cursor unchanged. Remote prefetch consumes the HTTP response completely before
returning, so idle handles do not hold request concurrency slots or connections.
To construct inline Blob V2 arrays for ingestion in record batches, use
[blob_array][lancedb.schema.blob_array]. It accepts bytes and `None`, preserving
the distinction between an empty blob (`b""`) and a null blob: