fix(python): expose optimize compaction options (#3870)

## Summary
- expose typed compaction options on synchronous and asynchronous table
optimization
- translate validated Python options into native Lance compaction
settings
- document the row-based steady-state limit and first-pass/source-budget
caveats
- carry the Lance beta.15 fragment-exclusion option through the typed
Python API

## Root cause
The native Python optimize binding always constructed default compaction
settings, and the public AsyncTable API offered no way to lower the
fragment row target. Wide-row fragments therefore remained compaction
candidates on every later optimize call, repeatedly rewriting the
accumulated table.

## Limitations
This is an explicit row-based mitigation, not byte-native task sizing or
a hard memory cap. Callers must derive target_rows_per_fragment from
expected row size. Existing oversized fragments are rewritten whole on
the first pass; max_bytes_per_file limits output file size rather than
compaction memory; and source budgets admit whole planned tasks, so a
budget below the first task performs no compaction. Splitting tasks by a
byte budget or returning a planner error requires a Lance-side change.

## Validation
- cargo check --release -p lancedb-python
- cargo clippy --quiet -p lancedb-python
- cargo fmt --all --check
- ruff format --check python tests
- ruff check python tests
- uv run --no-sync pytest python/tests/test_table.py -k
optimize_compaction -q (16 passed)

Fixes #2325

<!-- lance-gatekeeper-fix:v1 agent=9b58eee0c79040b19540e259af0e676c
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
This commit is contained in:
lancedb-gatefixer[bot]
2026-10-27 03:59:52 +08:00
committed by GitHub
co-authored by Xuanwo
parent 311ded06d0
commit 3352bf49df
14 changed files with 548 additions and 16 deletions
+2
View File
@@ -329,6 +329,8 @@ code that used the former remote-only argument order.
::: lancedb.table.Table
::: lancedb.table.CompactionOptions
::: lancedb.table.FragmentStatistics
::: lancedb.table.FragmentSummaryStats