mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-29 06:56:15 +00:00
fix: read legacy blob columns through blob APIs (#4649)
Legacy blob v1 columns were rejected by LanceDB's blob validation and omitted from `blob_columns()`, even though the pinned Lance reader already supports them. Accept both blob versions in read validation and discovery so `fetch_blobs`, `fetch_blob_files`, and `fetch_blob_ranges` can read legacy payloads through the existing native reader. Keep list-nested blobs excluded from row-based reads and document how to fetch legacy blobs using explicit row IDs. Regression coverage checks sync and async APIs, independently written Lance datasets, ordering and duplicates across fragments, null and empty blobs, lazy seeking, ranges, and deleted rows. Fixes #4644 Validation: - Both new Python regressions failed on the original implementation with the reported legacy-column rejection and empty discovery result, then passed with the fix. - Python blob suite: 202 passed, 1 skipped. - Remote blob tests: 29 passed. - Focused Rust blob unit tests with `remote`: 13 passed. - Rust blob integration suite with `remote`: 59 passed, including legacy storage 2.0 and 2.1. - Rust compile and Clippy checks with `remote`, tests, and examples passed. - Python package Ruff lint and format checks passed; Rust formatting passed. - Repository-wide `ruff check .` reports 49 existing errors in unchanged `docs/web-tests` examples; the Python package is clean. <!-- lance-gatekeeper-fix:v1 agent=02b7edcbfa8ff0e7fad4615159249b3e generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com> Co-authored-by: Xuanwo <github@xuanwo.io>
This commit is contained in:
co-authored by
Xuanwo
parent
83b7275137
commit
c167b99fa0
@@ -581,6 +581,9 @@ pylance, LanceDB uses a matching `lance.blob.v2` extension type so blob columns
|
||||
still work. Native table queries return descriptors. Call
|
||||
[`fetch_blob_files`][lancedb.table.Table.fetch_blob_files] for lazy reads or
|
||||
[`fetch_blobs`][lancedb.table.Table.fetch_blobs] for eager bytes.
|
||||
Local tables support these fetch APIs for both legacy blob v1 columns
|
||||
(`lance-encoding:blob`) and blob v2 columns. For legacy columns, obtain row IDs
|
||||
with `search().with_row_id(True).to_arrow()` and pass the result to the fetch API.
|
||||
One missing or deleted row ID rejects the entire blob-read batch; the APIs do
|
||||
not return partial results.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user