fix: preserve sliced child validity during struct merge (#4261)

## Root cause

The pinned Lance Arrow crate rebuilt a sliced struct child with its
original Arrow offset but supplied a validity bitmap containing only the
visible rows. Arrow requires the bitmap to cover the offset as well, so
filtered vector searches involving nullable nested fields could panic
with `null_bit_buffer size too small`.

## Fix

- Carry an offset-aligned validity bitmap when propagating parent nulls
to a sliced child.
- Vendor the pinned `lance-arrow` crate until a Lance release includes
the fix; include it in Maturin source distributions.
- Add unit regressions for sliced children and parent bitmaps, plus a
record-batch merge regression.

## Validation

- The issue's 8,600-row Python reproducer completes its filtered flat
dot search and returns 8,448 rows without a panic.
- The targeted Rust merge regression and two focused Python query tests
pass.
- Rust formatting, Ruff formatting and lint, the vendored crate tests,
and the workspace Cargo checks passed. The Maturin source archive
includes the patched crate.

Fixes #4260

<!-- lance-gatekeeper-fix:v1 agent=e6045f701d9b708ee26528ba36353aba
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
lancedb-gatefixer[bot]
2026-10-23 01:26:28 +08:00
committed by GitHub
parent 66b5b5bbdc
commit 624defd109
19 changed files with 7994 additions and 2 deletions
+2
View File
@@ -22,6 +22,8 @@ bytes.workspace = true
lancedb.workspace = true
datafusion-common.workspace = true
lance-core.workspace = true
# The workspace path dependency includes the patched crate in Maturin sources.
lance-arrow.workspace = true
lance-namespace.workspace = true
lance-namespace-impls.workspace = true
lance-io.workspace = true