mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-24 04:32:27 +00:00
fix: preserve sliced child validity during struct merge (#4261)
## Root cause The pinned Lance Arrow crate rebuilt a sliced struct child with its original Arrow offset but supplied a validity bitmap containing only the visible rows. Arrow requires the bitmap to cover the offset as well, so filtered vector searches involving nullable nested fields could panic with `null_bit_buffer size too small`. ## Fix - Carry an offset-aligned validity bitmap when propagating parent nulls to a sliced child. - Vendor the pinned `lance-arrow` crate until a Lance release includes the fix; include it in Maturin source distributions. - Add unit regressions for sliced children and parent bitmaps, plus a record-batch merge regression. ## Validation - The issue's 8,600-row Python reproducer completes its filtered flat dot search and returns 8,448 rows without a panic. - The targeted Rust merge regression and two focused Python query tests pass. - Rust formatting, Ruff formatting and lint, the vendored crate tests, and the workspace Cargo checks passed. The Maturin source archive includes the patched crate. Fixes #4260 <!-- lance-gatekeeper-fix:v1 agent=e6045f701d9b708ee26528ba36353aba generation=1 --> --------- Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
@@ -22,6 +22,8 @@ bytes.workspace = true
|
||||
lancedb.workspace = true
|
||||
datafusion-common.workspace = true
|
||||
lance-core.workspace = true
|
||||
# The workspace path dependency includes the patched crate in Maturin sources.
|
||||
lance-arrow.workspace = true
|
||||
lance-namespace.workspace = true
|
||||
lance-namespace-impls.workspace = true
|
||||
lance-io.workspace = true
|
||||
|
||||
Reference in New Issue
Block a user