fix: read legacy blob columns through blob APIs (#4649)

Legacy blob v1 columns were rejected by LanceDB's blob validation and
omitted from `blob_columns()`, even though the pinned Lance reader
already supports them. Accept both blob versions in read validation and
discovery so `fetch_blobs`, `fetch_blob_files`, and `fetch_blob_ranges`
can read legacy payloads through the existing native reader.

Keep list-nested blobs excluded from row-based reads and document how to
fetch legacy blobs using explicit row IDs. Regression coverage checks
sync and async APIs, independently written Lance datasets, ordering and
duplicates across fragments, null and empty blobs, lazy seeking, ranges,
and deleted rows.

Fixes #4644

Validation:

- Both new Python regressions failed on the original implementation with
the reported legacy-column rejection and empty discovery result, then
passed with the fix.
- Python blob suite: 202 passed, 1 skipped.
- Remote blob tests: 29 passed.
- Focused Rust blob unit tests with `remote`: 13 passed.
- Rust blob integration suite with `remote`: 59 passed, including legacy
storage 2.0 and 2.1.
- Rust compile and Clippy checks with `remote`, tests, and examples
passed.
- Python package Ruff lint and format checks passed; Rust formatting
passed.
- Repository-wide `ruff check .` reports 49 existing errors in unchanged
`docs/web-tests` examples; the Python package is clean.

<!-- lance-gatekeeper-fix:v1 agent=02b7edcbfa8ff0e7fad4615159249b3e
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
This commit is contained in:
lancedb-gatefixer[bot]
2026-10-27 19:56:50 +08:00
committed by GitHub
co-authored by Xuanwo
parent 83b7275137
commit c167b99fa0
7 changed files with 220 additions and 39 deletions
+3
View File
@@ -581,6 +581,9 @@ pylance, LanceDB uses a matching `lance.blob.v2` extension type so blob columns
still work. Native table queries return descriptors. Call
[`fetch_blob_files`][lancedb.table.Table.fetch_blob_files] for lazy reads or
[`fetch_blobs`][lancedb.table.Table.fetch_blobs] for eager bytes.
Local tables support these fetch APIs for both legacy blob v1 columns
(`lance-encoding:blob`) and blob v2 columns. For legacy columns, obtain row IDs
with `search().with_row_id(True).to_arrow()` and pass the result to the fetch API.
One missing or deleted row ID rejects the entire blob-read batch; the APIs do
not return partial results.