fix(python): support Polars 1.32 table scans (#3801)

## Root cause

`Table.to_polars()` disabled PyArrow predicate pushdown by selecting the
non-PyArrow Polars scan callback. Polars 1.32.3 invokes that callback
with `batch_size` both positionally and through its partial, so
collecting the returned lazy frame raises `TypeError:
_scan_pyarrow_dataset_impl() got multiple values for argument
batch_size`.

## Fix

- Keep the compatible PyArrow callback path.
- Add an identity `map_batches` barrier so predicates stay in Polars
instead of reaching the LanceDB adapter as unsupported PyArrow
expressions.
- Extend the tested Polars range through 1.32.3 and retain lazy-frame
regression coverage.

## Validation

- `python/tests/test_table.py::test_polars` with Polars 1.32.3
- `python/tests/test_table.py::test_polars` with the locked Polars 1.3.0
baseline
- `ruff format --check` on the changed Python files
- `ruff check .`
- `uv lock --check`

Fixes #2619

<!-- lance-gatekeeper-fix:v1 agent=0d42bcda944ac42765b25f2c19ff729f
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
This commit is contained in:
lancedb-gatefixer[bot]
2026-08-07 17:32:17 +08:00
committed by GitHub
parent dd5cb4d805
commit 564e5d0d56
4 changed files with 26 additions and 7 deletions
+1 -1
View File
@@ -1998,7 +1998,7 @@ requires-dist = [
{ name = "pillow", marker = "extra == 'clip'", specifier = ">=12.1.1" },
{ name = "pillow", marker = "extra == 'embeddings'", specifier = ">=12.1.1" },
{ name = "pillow", marker = "extra == 'siglip'", specifier = ">=12.1.1" },
{ name = "polars", marker = "extra == 'tests'", specifier = ">=0.19,<=1.3.0" },
{ name = "polars", marker = "extra == 'tests'", specifier = ">=0.19,<=1.32.3" },
{ name = "pre-commit", marker = "extra == 'dev'", specifier = ">=3.5.0" },
{ name = "pyarrow", specifier = ">=16" },
{ name = "pyarrow", marker = "extra == 'tests'", specifier = "<25" },