fix(python): report unsplittable IVF partition errors (#3846)

## Summary

- add Python regression coverage for an IVF build that cannot form all
requested non-empty partitions
- verify hierarchical k-means returns an actionable RuntimeError instead
of panicking or silently creating a degenerate index
- exercise the current Lance v10.1.0-beta.1 dependency, which contains
the upstream error-return fix

## Root cause

Hierarchical k-means previously guarded a shortfall in generated
clusters with only a debug assertion. Debug builds panicked, while
release builds could silently publish an index with many empty
partitions. The upstream Lance fix now returns a descriptive error and
is already included in the dependency pinned on main; this test locks in
propagation through the LanceDB Python API.

## Validation

- uv run --extra tests pytest python/tests/test_index.py -q (24 passed)
- uv run --extra tests pytest
python/tests/test_index.py::test_create_ivf_index_reports_unsplittable_partitions
-q (1 passed)
- python/.venv/bin/ruff format python/python/tests/test_index.py
- python/.venv/bin/ruff check .
- git diff --check

Fixes #3649

<!-- lance-gatekeeper-fix:v1 agent=a4d34448a9d350a3e2e659f33f5db6f2
generation=1 -->

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
lancedb-gatefixer[bot]
2026-08-06 16:13:54 +08:00
committed by GitHub
parent 7357d63e87
commit c3176a47ce
+25
View File
@@ -372,6 +372,31 @@ async def test_create_vector_index(some_table: AsyncTable):
assert stats.num_indices == 1
@pytest.mark.asyncio
async def test_create_ivf_index_reports_unsplittable_partitions(db_async):
dim = 8
num_partitions = 300 # More than 256 selects hierarchical k-means.
base_vectors = [[float(row == column) for column in range(dim)] for row in range(5)]
vectors = pa.array(base_vectors * 200, pa.list_(pa.float32(), dim))
table = await db_async.create_table(
"unsplittable_partitions",
pa.table({"vector": vectors}),
)
error_pattern = (
rf"Cannot create {num_partitions} IVF partitions: k-means could only form"
)
with pytest.raises(RuntimeError, match=error_pattern):
await table.create_index(
"vector",
config=IvfFlat(
distance_type="dot",
num_partitions=num_partitions,
max_iterations=10,
),
)
@pytest.mark.asyncio
async def test_create_4bit_ivfpq_index(some_table: AsyncTable):
# Can create