Function dependency typos currently reach the environment build before
failing. Validate pip requirements with
`packaging.requirements.Requirement` and conda requirements with
Rattler's MatchSpec parser when constructing a function definition,
before source packaging or registration. Errors identify the original
entry and its index.
```python
@lancedb.function(pip=["numpy", "torch>=2.0,,<3"])
def score(value: float) -> float:
return value
# ValueError: invalid pip[1] requirement 'torch>=2.0,,<3': ...
```
Preserves pip extras, markers, named references, bare archive/git URLs,
and conda channel/build specifications. The existing native extension
exposes the conda parser, so no additional Python package is required.
Adds regression coverage for invalid and supported forms, including the
deprecated `udf` alias.
Fixes#4773.
Python job polling now retries transient connection, send, and read failures with capped exponential backoff under the caller's original timeout. If the deadline expires while disconnected, `wait()` raises `ConnectionError` with the job ID, reconnect guidance, and the original cause; after recovery, ordinary timeouts remain `TimeoutError`. Terminal job errors, cancellation, and malformed responses continue to surface immediately.
Validation: exact-head CI passed across Rust all-features, Python on Linux, macOS, and Windows, NodeJS, deny, and packaging checks. The hosted macOS wheel suite passed 3049 tests (162 skipped, 70 deselected), including sync and async recovery, timeout reuse, cancellation, terminal errors, and error preservation.
Updates Rust and Java Lance dependencies to
[v15.0.0-beta.6](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.6),
pins Python tests to published pylance 15.0.0b6, and refreshes both
lockfiles.
Aligns Arrow 59, DataFusion 55, and PyO3 0.29; adapts execution plans
and MemWAL directory handling, updates the ABI metadata test, and
removes the obsolete Lance Arrow patch now fixed upstream.
Validation: workspace clippy with all features/tests and warnings
denied, cargo fmt, Python/CI Ruff checks, uv lock check, 16 CI-script
tests, and targeted Python regressions (54 passed, 28 skipped);
repository-wide Ruff retains existing documentation-example failures,
and the upstream macOS wheel build failed while Linux and Windows wheels
are published.
---------
Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Published SDK builds currently expose local SQL computed columns and
materialized-view execution that are not ready for release. Add the
opt-in `experimental-functions` Cargo feature to the Rust core and
forward it through the Python and Node bindings; published packages
leave it disabled.
Local declarations and refreshes now return `NotSupported` before
mutating tables or spawning jobs, including prepared materialized-view
creation. Remote Cloud/Enterprise client operations, Python UDF
packaging, ordinary eager `add_columns`, and access to existing data
remain available. Stored metadata and write protections stay active, and
shared server commit helpers remain usable.
CI covers both feature configurations. Existing local execution tests
run with the feature enabled; default-build regressions cover early
rejection, unchanged table versions, existing metadata protections, and
normal eager transforms. API and source-build documentation explain the
opt-in.
Validation: Rust library tests pass with the feature disabled (1,869
tests) and enabled (1,947 tests), plus three default-build integration
regressions. Python and TypeScript bindings were rebuilt and tested in
both configurations, including mocked remote requests; Rust formatting,
Python package lint, TypeScript lint/build, and generated docs were
checked. Repository-wide Ruff still reports 48 pre-existing errors in
unchanged `docs/web-tests` files.
Consumers that provide a local Lance checkout cannot resolve the
dependency graph because the SDK directly selects a vendored
`lance-arrow`, producing two path packages with the same name and
version. The Rust core now declares its upstream Git dependency
separately so the consuming workspace can override that source
consistently. Python continues inheriting the vendored workspace
dependency, keeping the sliced-struct validity fix in Maturin source
distributions. Standalone SDK builds still select that same fix through
the existing root patch.
Adds a downstream consumer CI regression that resolves a local Lance
override and checks that exactly one canonical `lance-arrow` package is
selected. The Lance version updater also updates the core declaration in
stable, preview and local modes. Public APIs and query behavior are
unchanged.
The workspace declaration lint excludes path dependencies because the
Python binding intentionally uses the bundled source. Duplicate and
unused external declarations remain errors; advisory, license,
banned-package and source checks continue covering the full dependency
graph.
---------
Co-authored-by: Lance Release <lance-dev@lancedb.com>
## Root cause
The pinned Lance Arrow crate rebuilt a sliced struct child with its
original Arrow offset but supplied a validity bitmap containing only the
visible rows. Arrow requires the bitmap to cover the offset as well, so
filtered vector searches involving nullable nested fields could panic
with `null_bit_buffer size too small`.
## Fix
- Carry an offset-aligned validity bitmap when propagating parent nulls
to a sliced child.
- Vendor the pinned `lance-arrow` crate until a Lance release includes
the fix; include it in Maturin source distributions.
- Add unit regressions for sliced children and parent bitmaps, plus a
record-batch merge regression.
## Validation
- The issue's 8,600-row Python reproducer completes its filtered flat
dot search and returns 8,448 rows without a panic.
- The targeted Rust merge regression and two focused Python query tests
pass.
- Rust formatting, Ruff formatting and lint, the vendored crate tests,
and the workspace Cargo checks passed. The Maturin source archive
includes the patched crate.
Fixes#4260
<!-- lance-gatekeeper-fix:v1 agent=e6045f701d9b708ee26528ba36353aba
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Jack Ye
Lance Release
lancedb-gatefixer[bot]
Xuanwo
