mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
perf: reduce Blob read buffering and expose read window controls (#4979)
Remote Blob reads retain and concatenate intermediate IPC buffers across batches and HTTP requests, increasing peak memory. Decode fragmented IPC bodies using their declared length, assemble Blob values once across batches/requests, and collect large-object Range responses into known-size buffers. Impossible IPC/Range allocations return errors instead of panicking. Closing a remote Blob file releases cached bytes immediately, including while a cross-window read is waiting for the network. Rust and Python expose configurable Blob read buffers. Python sync/async `fetch_blob_files` and `open_blob_file(s)` accept `buffer_size`; the default is 512 KiB and zero disables prefetch. Explicit ranges bypass prefetch. New PE batch-endpoint adoption is separate. Malformed-length and retained-cache regressions fail on the previous implementation and pass after the correction. Real HTTP tests cover buffer propagation, close during blocked input, batch/request aggregation, and large-object fallback. The final extension was rebuilt before Python validation. Non-Blob Arrow arrays may still require alignment copies; a contiguous multi-batch result still requires its final allocation. ## Benchmark Peak client RSS increment in KiB, using `python/benchmarks/bench_fetch_blobs_rss.py`, independent clients and real loopback HTTP. Both extensions were rebuilt from clean, unchanged source with the same toolchain/configuration; loaded artifact hashes were checked. Baseline `42f11d13a2755e44b63876b5a5edd93901291150`, after `0a8a76de5286949d80368f95a5c529adf422c187`. | Equivalent fetch workload | Payload | Before | After | | --- | ---: | ---: | ---: | | Single response, one batch | 48 MiB | 63,812 | 63,896 | | Multiple responses, one batch per response | 144 MiB | 423,104 | 195,628 | | Single response, 12 rows per batch | 48 MiB | 130,520 | 71,348 | | Multiple responses, 12 rows per batch | 144 MiB | 363,144 | 172,748 | | One oversized row, Range fallback | 70 MiB | 182,816 | 85,548 | These are one observation per workload and build, not a statistical distribution or memory upper bound. The single-batch workload shows no improvement. Cloud/Sophon deployment, credentialed S3 and long-running stress have not been exercised.
This commit is contained in:
@@ -793,6 +793,15 @@ with `search().with_row_id(True).to_arrow()` and pass the result to the fetch AP
|
||||
One missing or deleted row ID rejects the entire blob-read batch; the APIs do
|
||||
not return partial results.
|
||||
|
||||
Local and remote `BlobFile` handles prefetch 512 KiB for sequential reads by
|
||||
default. Pass `buffer_size=0` to `fetch_blob_files`, `open_blob_file`, or
|
||||
`open_blob_files` for sparse reads, or choose another window size in bytes.
|
||||
Call `file.set_buffer_size(size)` to change it without moving the cursor;
|
||||
changing the size discards unused prefetched bytes. Explicit `read_range` and
|
||||
`read_ranges` calls always read exactly the requested intervals and leave the
|
||||
cursor unchanged. Remote prefetch consumes the HTTP response completely before
|
||||
returning, so idle handles do not hold request concurrency slots or connections.
|
||||
|
||||
To construct inline Blob V2 arrays for ingestion in record batches, use
|
||||
[blob_array][lancedb.schema.blob_array]. It accepts bytes and `None`, preserving
|
||||
the distinction between an empty blob (`b""`) and a null blob:
|
||||
|
||||
Reference in New Issue
Block a user