Upgrade the Rust Lance dependency, Java lance-core, and Python test
dependency from beta.17 to the officially published
[v15.0.0-beta.18](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.18)
(`079fd77683a7acf056b0912f6ecda938e39ee120`). This provides a compatible
SDK for the newer Lance source already adopted downstream. Rust registry
versions and checksums are unchanged; the Python lock uses the six
published beta.18 wheels.
### Compatibility
The adopted release includes [Lance
#10863](https://github.com/lance-format/lance/pull/10863):
`FilteredReadOptionsProto.has_fragment_ids` distinguishes an explicit
empty fragment selection. Older executors ignore this field, so upgrade
executors before relying on the corrected empty-selection behavior. Rust
callers constructing this generated struct need the new field. [Lance
#10112](https://github.com/lance-format/lance/pull/10112) also replaces
the internal `hamming_distance_batch` API with
`hamming_distance_batch_into`.
The Linux Python matrix timeout is raised from 30 to 60 minutes: the
Python 3.13 cold native build and both test passes completed, but the
job hit its limit during cleanup. Build profiles, test commands, and
coverage are unchanged.
### Validation
- Three native FRI/ANN regressions passed: deferred-compaction refined
RQ, deferred index remapping, and approximate-query builder.
- `cargo clippy --quiet --locked --features remote --tests --examples --
-D warnings` passed with pinned Rust 1.97.0.
- Locked metadata, formatting, Python Makefile-scope Ruff checks, and
`uv lock --check --offline` passed.
- YAML, whitespace, end-of-file, and typo checks passed for the
timeout-only workflow change.
### Publication and CI
Java beta.18 is now published. After [Lance
#10968](https://github.com/lance-format/lance/pull/10968) merged, the
[normal publication
workflow](https://github.com/lance-format/lance/actions/runs/43146352505)
built all three JNI targets and published successfully; the Maven jar is
available. The upstream Python publication run was cancelled after its
Windows ARM job hit the one-hour timeout; six other wheels were
published and are the actual artifacts recorded in this lock.
Cross-platform SDK CI is still required. No performance claim is made by
this dependency update.
Update the Rust workspace, Java dependency, and Python test dependency
to [Lance
v15.0.0-beta.17](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.17).
This includes the FRI cache and vector-index changes required by current
Sophon and retains the RaBitQ prewarm fixes. Lockfiles use the published
source and test wheels without unrelated registry version bumps.
The dependency dry run also exposed CI resource limits. Use the existing
larger x64 runner for GNU Node and compatibility-wheel builds; allow 60
minutes for Mac Python tests, 120 minutes for the compatibility wheel,
and 150 minutes for the Mac release wheel. Disable the CLI toolchain
action's automatic target cache so only the subsequent downloads-only
cache runs. Release optimization, the compatibility wheel's manylinux
container and x86-64-v2 baseline, and test coverage stay unchanged.
Validation:
- Three native SDK regressions pass: refined RQ queries after deferred
compaction, deferred index remapping, and ANN approximation planning.
- Pinned Rust 1.97.0: `cargo clippy --quiet --locked --features remote
--tests --examples -- -D warnings` passes.
- Locked offline Rust metadata, Python lock validation, Ruff
lint/format, and diff checks pass. The Python lock contains six
published beta.17 wheels.
- Repository YAML, whitespace and typo checks pass for the workflow
changes. Product-source checks are reused for the subsequent CI-only
commits.
- At head `07033dc6`, all 65 applicable pre-review CI checks pass,
including the compatibility wheel, Mac CLI, GNU Node, Mac Python tests,
release wheels, and Java checks. Six checks are skipped by tag/main-only
conditions or draft-only app behavior; the automated review and security
checks run after the Ready transition.
The upstream Java publisher succeeded and its beta.17 Maven artifact is
available. Six upstream Python wheel builds/uploads succeeded; its
Windows ARM job was cancelled. These checks do not establish Sophon
performance or aggregate-memory acceptance.
Update the Rust workspace, Java lance-core, and Python test dependency
to [Lance
v15.0.0-beta.16](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.16).
Synchronize `python/uv.lock` with the published beta.16 wheels for
Linux, macOS, and Windows.
Run the native Linux x64 wheel build on an eight-core runner, matching
the existing ARM build allocation, after its optimized build repeatedly
exceeded the job deadline on the standard runner.
The dependency update passed Clippy, formatting, and CI-script checks;
the Python lockfile refresh/check passes. Fresh CI is running for the
runner adjustment. Java dependency-resolution failures caused solely by
publication lag are accepted for this upgrade.
---------
Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
Update Rust and Java Lance dependencies to
[v15.0.0-beta.15](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.15),
pin Python tests to published pylance 15.0.0b15, and refresh both
lockfiles; no compatibility fixes were needed.
Validation passed: workspace Clippy with all features/tests and warnings
denied, cargo fmt, uv lock --check, and all 35 CI-script tests.
The [upstream wheel
workflow](https://github.com/lance-format/lance/actions/runs/42669437482)
canceled Windows ARM64 and manylinux_2_28 x86_64 builds after an hour,
so the lockfile contains the five published variants (Linux x86_64
remains covered by manylinux2014); Windows ARM64 is unavailable until
upstream publishes its wheel and the lockfile is refreshed.
Update the Rust workspace Lance crates and Python test dependency to
[Lance
v15.0.0-beta.14](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.14),
which includes JSON cache reuse across request stores and stable-row-ID
JSON index support. The Python lock contains all seven published wheels.
The independent Java module retains beta.8, as in the previous beta.13
update; its separate beta.14 publisher repair is outside this Rust
consumer upgrade.
Preserve all CI coverage while reclaiming completed workspace lancedb
artifacts before the package-level feature-test build, which otherwise
exhausted the Linux runner disk. Give only the Linux ARM64 release-wheel
job two hours after its one-hour timeout, retaining the same runner,
fat-LTO release profile and build flags.
Validation: 2,280 Rust tests passed (79 existing ignores), strict
workspace/all-feature Clippy, Rust formatting, and remote example checks
passed on unchanged Rust source and dependency lock. Python dependency
guards, lock validation, locked sync, and the published macOS wheel
import passed. The new Windows ARM64 wheel was downloaded and its
SHA-256 verified. Workflow YAML/shell checks and independent review
passed. Root Python Ruff/format checks still fail on unchanged
documentation examples; no suppressions or unrelated cleanup were added.
The CI repairs require a complete new run before readiness. This
dependency update does not establish corrected Sophon performance;
downstream native acceptance and the paired S3 replay remain separate.
Update the Rust workspace Lance dependencies to
[v15.0.0-beta.13](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.13),
synchronize the Python test dependency to published pylance 15.0.0b13,
and refresh both lockfiles; no compatibility fixes were needed.
The Java SDK stays on `lance-core` 15.0.0-beta.8, as in the beta.9 and
beta.10 updates: Maven Central has no `lance-core` artifact for
15.0.0-beta.9 through beta.13.
Validation passed: workspace Clippy with all features and warnings
denied, Cargo formatting, Python lockfile checks, and all 35 CI-script
tests. Repository-wide Ruff reports 163 diagnostics, all verified
identical on unchanged HEAD; unrelated formatter changes were reverted.
---------
Co-authored-by: Vivek <vbharathan2008@gmail.com>
Lance beta.11 calibrates lower `search_effort` tiers by metric and
requested `k`. With the Rust dependency and Python/Node execution tests
already on main via #4881, document the tiers across Rust, Python, Node
and the search guide, extend the remote request matrix to cover `0.1`
and `0.25`, and align the Python test package with beta.11.
`0.1` and `0.25` target empirical mean partition-routing recall of 80%
and 90% for L2, cosine and dot through `k=100000`. These are not
per-query or final Recall@k guarantees. Default Auto is unchanged;
Hamming and larger `k` remain uncalibrated, and remote queries require
corresponding server support. This PR makes no additional performance
claim. See [Lance
#10695](https://github.com/lance-format/lance/pull/10695) for
measurements and limitations.
Validation: both strict workspace Clippy configurations, Rust
formatting, 13 Rust effort tests, 208 Python query tests, 41 Node query
tests, native binding builds, TypeScript, Node lint/generated docs,
Python lint/format, full lockfile check and two dependency guards passed
on `a8b09170`. All validated Rust/Python/Node source and dependency
files are byte-identical at `2a82d07c`; the only subsequent change
restores the Java POM to main. Root Ruff has 59 existing diagnostics and
six formatting files, all byte-identical to the base; the Python package
passes.
The Java REST/Namespace client has no changed API here, and its
dependency remains identical to main. The separate [Java beta
publication failure](https://github.com/lance-format/lance/issues/10773)
is not claimed resolved or validated. Official SDK release, automatic
mirror, Sophon adoption and measured consumer acceptance remain
follow-up stages.
## Summary
Bumps dependencies that Aikido flags with critical advisories, where a
patched version is available without code changes.
| Package | Where | From → To | Ships to users |
|---|---|---|---|
| jackson (BOM) | `java/pom.xml` | 2.18.11 → 2.18.12 | Yes, via the
published pom |
| zerovec / zerovec-derive | `Cargo.lock` | 0.11.6 → 0.11.8 / 0.11.3 →
0.11.7 | Yes, linked into the wheels and Node binaries |
| pillow | `python/uv.lock` | 12.3.0 → 13.0.0 | No, dev lockfile |
| pyarrow | `python/uv.lock` | 24.0.0 → 26.0.1 | No, dev lockfile |
| pyjwt, urllib3, anyio, cryptography | `python/uv.lock` | → 2.15.1,
2.8.0, 4.16.0, 50.0.2 | No, dev lockfile |
| pillow | `docs/web-tests/blob-v2/py/requirements.txt` | 12.1.1 →
13.0.0 | No |
pyarrow was held at 24.0.0 by a `pyarrow<25` pin in the `tests` extra.
It came in with #3675 without a stated reason, so this PR removes it.
pillow 13 and pyarrow 26 require Python 3.11+, so the lockfile keeps
pillow 12.3.0 and pyarrow 25.0.1 on Python 3.10, and those remain
flagged.
The sharp findings in the Node examples and docs are not addressed here.
They come from the `@huggingface/transformers@3.0.2` optional
dependency, which needs a major upgrade to 4.3+.
## Evidence
With the pyarrow pin removed and pyarrow 26.0.1 installed, the Python
suite (`-m "not slow and not s3_test"`) passed locally on Python 3.12:
3319 passed, 164 skipped. That run was on main as of #4879; this branch
is based on a slightly newer main.
## Merge Risk
**Door:** Two-way. Patch and minor version bumps that a revert undoes.
**Blast Radius:** Java consumers (jackson patch release), native
binaries (zerovec patch release), Python test environments.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Update the Rust workspace Lance dependencies to
[v15.0.0-beta.10](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.10),
which contains the released MinHash signature scanning API needed by
downstream consumers. Pin Python tests to the published `15.0.0b10`
wheels, including macOS, and refresh both lockfiles. Java remains on
beta.8.
No SDK source compatibility changes were needed. All 25 Lance Git
packages resolve to release commit
`f56a08d40b5630ce1124535c4c3b1406d5ceedf1`; third-party package versions
are unchanged.
Validation on macOS ARM64 with pinned Rust 1.97.0: strict workspace
Clippy with default and all features, Rust formatting, 32 CI-script
tests, three count tests, one WAL/vector regression, and the Python
lockfile check passed. The native test binary emitted a macOS
compact-unwind size warning; both test commands completed successfully
without changing linker settings. Downstream release and integrated
Sophon acceptance remain separate.
Update Rust Lance dependencies to
[v15.0.0-beta.9](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.9),
including the scoped stable-row-ID conversion fix. Python tests use
published beta.9 wheels on Linux/Windows while macOS retains beta.4.
Java stays on beta.8, matching the original base, so this update does
not depend on beta.9 Maven publication or macOS Java packaging.
The dependency updater preserves workspace inheritance, the WAL test
wrapper uses the new StoredBatch module, and Linux/remote Rust CI uses
its installed mold linker. The Python lockfile also records the
already-declared OCI extra. CLI CI now caches Cargo downloads under a
separate key instead of restoring compiled targets: the previous runner
exhausted its disk during cache restoration, before compilation. Its
test matrix and commands are unchanged.
This branch includes #4820's row-freshness changes: successful rows
become current while skipped rows retain their values and remain
eligible for retry. Repair identities, replay handling and Restore
fencing protect publication completion.
BREAKING CHANGE: `StalenessPlan::row_dirty` is a new public field. Rust
callers using exhaustive struct literals must initialize it.
Validation:
- All reported non-Java checks passed on
`f5fa66fcbcf31a8eb188935f728735560c77c873`, including Rust
lint/Linux/remote, Python, NodeJS and CLI checks. Current head
`c660bc0ac9fd5920ad4b64be9c19f732ef4c56a5` preserves those product
sources, manifests, lockfiles and compiler/test commands; only the Java
pin restoration and CLI cache configuration differ. Current-head CI is
tracked separately.
- The complete Java directory is identical to the original PR base. The
scope correction reuses [Xuanwo's prepared and validated
commit](https://github.com/lancedb/lancedb/pull/4814#issuecomment-6626088466);
no new Java build was run.
- CLI workflow YAML parses successfully, its inputs match the pinned
rust-cache action schema, and the existing jobs, matrix and commands are
unchanged. Hosted execution must confirm the cache-restoration repair.
- Existing validation includes the updater regressions and 32 CI-script
tests, dependency-policy checks, count/WAL coverage, and #4820's 42
freshness tests, strict core Clippy and Rust doctest. Root Ruff
previously reported unchanged documentation-example issues;
Python-package Ruff passed.
This PR adopts the existing released Lance beta.9. A subsequent official
SDK release and normal Sophon dependency adoption remain separate
delivery steps.
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
Adds `OCIGenAIEmbeddings`, registered as `"oci-genai"`, embedding text with the Cohere models hosted on Oracle Cloud Infrastructure Generative AI (default `cohere.embed-v4.0`) via the `oci` SDK, alongside the existing Bedrock, Gemini and watsonx functions.
- Auth delegated to the OCI SDK: `API_KEY` / `SECURITY_TOKEN` profiles from the OCI config file, or `INSTANCE_PRINCIPAL` / `RESOURCE_PRINCIPAL`. No credentials are stored on the function or in table metadata; `compartment_id` can come from `OCI_COMPARTMENT_ID` so the OCID need not be baked into metadata either.
- Endpoint derived from `region` (default `us-chicago-1`) or overridden with `service_endpoint`; dedicated AI cluster endpoint OCIDs use `DedicatedServingMode`.
- Asymmetric Cohere input types (`SEARCH_DOCUMENT` for the source column, `SEARCH_QUERY` for queries); `truncate` and `output_dimensions` (embed-v4.0: 256/512/1024/1536) forwarded.
- Batched at the service limit of 96 inputs; empty inputs are not sent and come back as `None`.
- `ndims()` uses `output_dimensions` when set, otherwise a single cached probe request, so there is no model→dims table to keep in sync with the catalog.
- SDK retries disabled in favour of LanceDB's `max_retries` backoff; 401/403 service errors carry `status_code` so they surface without retrying, with an actionable message (expired session token, IAM policy, region, model).
- `oci>=2.165.1` added to the `embeddings` extra (the floor where `EmbedTextDetails.output_dimensions` exists).
Tests: 22 mocked unit tests (auth paths, batching, input types, config validation, retry behaviour, pickling, table-metadata round trip) and a live test skipped unless `OCI_COMPARTMENT_ID` is set, which passes against us-chicago-1 for embed-v4.0 at 1536 and 512 dims and embed-multilingual-v3.0 at 1024. Verified end to end with a real LanceDB table (`LanceModel` with `SourceField`/`VectorField`, 6 rows, two semantic queries ranked correctly). `ruff format --check`, `ruff check`, pyright and typos are clean.
Docs: a page under `docs/src/embeddings/available_embedding_models/text_embedding_functions/` in the shape of the remaining `voyageai_embedding.md`; the function is exported from `lancedb.embeddings` so the API reference picks it up. The hosted docs' available-models table lives outside this repo and would need a separate update.
## Problem
The Python test extra pins `pylance==15.0.0b7` for every platform. On macOS, pip cannot resolve that pin: the configured `lance-format` index publishes b7 wheels for Linux and Windows but not macOS, while public PyPI returns HTTP 404 for the version and the `lancedb` index contains no 15.x release. The macOS Python dependency installation consequently cannot resolve the pin.
## Change
- Keep the exact `pylance==15.0.0b7` test pin on non-macOS platforms.
- Use `pylance==15.0.0b4` on macOS. This is the newest release in this range with a macOS wheel (`cp310-abi3-macosx_11_0_arm64`).
- Refresh `python/uv.lock` for the platform-specific resolutions.
The production Rust/Lance inputs are unchanged: `Cargo.lock` is byte-identical to `main` and remains on Lance `15.0.0-beta.7`. No tests, assertions, coverage, or CI checks are removed or weakened.
## Validation
- Confirmed the wheel identity from the configured index and matched its published SHA-256 to `uv.lock`; wheel metadata reports `Name: pylance`, `Version: 15.0.0b4`.
- Verified the b4 APIs used by the Python tests (`blob_field`, `blob_array`, `BlobType`, `Blob`, `dataset`, `write_dataset`, `FFILanceTableProvider`, and `DirectoryNamespace`). The Python test suite does not reference the b7-only `mark_sstables_as_compacted`, `mem_wal_dir_id`, `mem_wal_index_details`, `clustering_id`, `LanceFragment`, `IndexSegment`, `UpdateConfig`, or `max_source_fragments` changes.
- Verified scalar read/write interoperability in both directions between b4 and the beta.7 LanceDB native binding, plus beta.7 reads of a b4-written blob dataset with byte and null fidelity.
- `uv lock --check` with both configured Fury indexes and `unsafe-best-match`.
- CI dependency tests.
- Ruff 0.9.9 format and lint checks.
- Targeted Python tests: 911 passed, 22 skipped.
- Full Python suite (`-m "not slow and not s3_test"`): 2858 passed, 157 skipped, 70 deselected.
Updates Rust workspace and Java Lance dependencies to
[v15.0.0-beta.7](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.7),
and pins Python tests to the published `pylance==15.0.0b7` release with
refreshed Cargo and uv lockfiles. No compatibility fixes were required.
Validation: workspace Clippy with all features and warnings denied,
Cargo formatting, uv lock consistency, and all 18 CI-script tests pass;
repository-wide Ruff diagnostics are pre-existing.
---------
Co-authored-by: Lu Qiu <lu@lancedb.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Updates Rust and Java Lance dependencies to
[v15.0.0-beta.6](https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.6),
pins Python tests to published pylance 15.0.0b6, and refreshes both
lockfiles.
Aligns Arrow 59, DataFusion 55, and PyO3 0.29; adapts execution plans
and MemWAL directory handling, updates the ABI metadata test, and
removes the obsolete Lance Arrow patch now fixed upstream.
Validation: workspace clippy with all features/tests and warnings
denied, cargo fmt, Python/CI Ruff checks, uv lock check, 16 CI-script
tests, and targeted Python regressions (54 passed, 28 skipped);
repository-wide Ruff retains existing documentation-example failures,
and the upstream macOS wheel build failed while Linux and Windows wheels
are published.
---------
Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Upgrade Lance to 15.0.0-beta.3 so `Dataset::concat_data_file_parts`
returns a `Send` future and downstream workers can run it on Tokio. The
upstream fix is https://github.com/lance-format/lance/pull/10370.
Keep the vendored Arrow validity fix; upstream Arrow sources are
unchanged between beta.2 and beta.3.
**Blocked on the corrected Lance release:**
https://github.com/lance-format/lance/pull/10370 restores the `Send`
contract required by downstream Function workers. Refresh this upgrade
with that release before merging.
Upgrade the Rust, Java, and Python Lance dependencies to
https://github.com/lance-format/lance/releases/tag/v15.0.0-beta.2. This
brings in bounded vector-index consolidation without retraining
(https://github.com/lance-format/lance/pull/10335) and deletion
filtering for index entries whose fragment coverage was pruned
(https://github.com/lance-format/lance/pull/10325).
Synchronize the vendored `lance-arrow` with the release and retain its
offset-safe validity patch. The existing `nullable_struct_merge`
regression still panics with the unpatched release, so removing that
patch would regress nullable nested queries. Refresh both lockfiles,
including the `zstd-sys` version required by the new Lance encoding
dependency.
Update the custom memtable bitmap test adapter to train on row
addresses, matching the new bitmap contract; remove its obsolete row-ID
conversion.
Preserve the Blob APIs' existing missing-row error messages when newer
Lance rejects a missing fragment earlier in the read path. Retain
upstream attribution in vendored sources while satisfying the repository
license-header check.
Validation of the compatibility fixes: all 58 Rust Blob integration
tests and 219 Python Blob tests pass (one Python test skipped). Strict
workspace Clippy, formatting, and diff checks pass.
Fixes#4248.
The preview index still contains pylance 1.0.0b6. uv's default
first-index resolution can select it, or backtrack to an old LanceDB
release when installing the extra. Meanwhile, the extra still permits
pylance 5 even though current main bundles Lance 14.
Raise the optional dependency to `pylance>=14.0.0b12`, the version
already pinned in the test extra, and keep the lockfile metadata
consistent. Document both Fury indexes and uv's `unsafe-best-match`
strategy in the Python README and the source-owned quickstart
(`docs/web/quickstart.mdx`). Explain the index trust requirement and
recommend installing the extra to enforce the version floor.
Add CI checks that reject older Lance majors and ensure the pylance and
tests extras remain compatible. Run these checks when either dependency
manifest changes, so a future Lance major bump cannot silently leave the
Python minimum behind.
### Validation
- Recreated the reported install on Ubuntu 26.04 / CPython 3.12.15 with
uv 0.12.23: `lancedb==0.40.0b5` plus the old index selects
`pylance==1.0.0b6`. Reproduced both the missing `blob_array` import and
the unexpected `blob_handling` scanner argument using a real blob table.
- Resolved the modified `python/pyproject.toml` with the pylance extra
against both indexes: selects `pylance==14.0.0b18`, matching the current
Rust pin.
- Ran both documented pip and uv installations in isolated environments:
`lancedb==0.41.0b7`, `pylance==14.0.0b18`; blob-array construction and a
blob-bytes round trip through `to_pandas(blob_mode="bytes")` pass. Both
package dependency checks pass.
- New regression fails on the original dependency bound (27
rejected-version cases incorrectly accepted), then passes with the fix.
Full CI-script suite: 3 tests pass.
- `uv lock --check` with both indexes passes; focused Ruff checks pass;
`git diff --check` passes.
- Repository-wide Ruff reports 49 existing errors in untouched
`docs/web-tests` files. Unrelated formatting changes were discarded.
Runtime smoke checks used released native wheels; source dependency
resolution was tested directly from the modified pyproject. No Rust or
Python runtime code changes.
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
Lance supports `search_effort`, but LanceDB cannot pass it through its
query APIs. Expose the parameter in Rust, Python sync/async and
TypeScript, including remote requests and hybrid query conversion, using
the [Lance
v14.0.0-beta.13](https://github.com/lance-format/lance/releases/tag/v14.0.0-beta.13)
dependency already on main.
The default is `None` (Auto, internally 0.5). Any explicit effort,
including 0.5, overrides fixed `nprobes` in either setter order;
explicit minimum/maximum bounds remain effective. Invalid values are
rejected in the Rust core. Hybrid conversion now retains the vector
probe parameters and handles an inferred FTS column without panicking.
Namespace queries with explicit effort execute locally because the
namespace wire model cannot carry it. MemWAL LSM queries reject it
explicitly because Lance's LSM scanner does not yet expose this option.
Maturin is included in the dev extra so the documented build command
works in the project environment.
BREAKING CHANGE: The public Rust `VectorQueryRequest` struct now
includes `search_effort: Option<f64>`. Downstream code that initializes
every field explicitly must add `search_effort: None` or use
`..Default::default()`. Exhaustive destructuring patterns must include
the new field or `..`. Query-builder calls and struct literals that
already use `..Default::default()` require no source changes. The new
effort setters in Python and TypeScript are additive. This PR does not
change the persisted file format.
Using explicit effort with remote tables requires server support for
`search_effort` and its precedence over `nprobes`; the client wire tests
do not establish that a deployed server supports it. Server-side support
is tracked in lancedb/sophon#8607.
Validation on AWS after merging current main (Lance beta.13 / SDK
beta.5):
- 14 Rust regressions and the API doctest pass.
- 60 parameter regressions, the full 47-test namespace suite, and three
API doctests pass. The parameter selection includes some namespace
cases.
- 40 TypeScript query tests and 54 remote tests pass; build, lint and
generated docs pass.
- Rust Clippy, Python package lint/format, Pyright and lockfile checks
pass. Repository-wide Ruff reports 49 lint errors and 11 formatting
failures in unchanged `docs/web-tests` files; those files are
byte-identical to main and outside this change.
All 1,103 Git-tracked files match the AWS validation checkout. No
benchmark scripts or result artifacts are included, and no performance
improvement is claimed.
Uses lance-format/lance#9798. Server-side support is in
lancedb/sophon#8607.
Keep IVF probe fields unset by default so Lance supplies adaptive
behavior. Preserve `nprobes`, `minimum_nprobes`, and `maximum_nprobes`
independently through Rust, Python, namespace, and remote request
construction, with compatibility fields serialized only when explicitly
configured.
The final Rust namespace and LSM propagation update depends on the Lance
release containing lance-format/lance#9184.
Updates the Rust workspace Lance dependencies and Java lance-core from
14.0.0-beta.1 to
[v14.0.0-beta.6](https://github.com/lance-format/lance/releases/tag/v14.0.0-beta.6),
and refreshes Cargo.lock. No compatibility fixes were required.
Validation passed: `cargo clippy --quiet --workspace --tests
--all-features -- -D warnings`, `cargo fmt --all --quiet`, and `git diff
--check`.
---------
Co-authored-by: Daniel Rammer <hamersaw@protonmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds `TypeSafeReranker`, which reranks vector, FTS, and hybrid results
with the [TypeSafe System One
API](https://docs.typesafe.ai/introduction).
Each result is scored independently: TypeSafe reads `{"query",
"document"}` and answers a yes/no (noul) question, and the probability
of yes becomes `_relevance_score`. Unlike listwise LLM rerankers, the
score is an absolute probability, so it is comparable across queries and
can be thresholded. The question's `instructions` and `true`/`false`
`criteria` are configurable, since domain-specific criteria are what
make this kind of scoring work well ([TypeSafe's re-ranking
cookbook](https://docs.typesafe.ai/cookbooks/rerank_typesafe)).
The API takes one state per request, so the reranker sends one request
per result on a thread pool bounded by `max_concurrency`. It
deliberately does not use the background event loop: rerankers are
called synchronously from inside the async query APIs, where `LOOP.run`
would deadlock.
TypeSafe scores for the same pair vary slightly between calls, so
results with close scores can swap places when a search is repeated. The
shared reranker test helper now takes `deterministic=False` for this
case: it still checks result sizes and descending scores, but not that
two identical searches return the same order.
The SDK is imported only when the client is created and questions are
sent as plain dicts, so the new tests run in CI with a fake client and
without `typesafe-sdk` installed. The live-API test is skipped without
`TYPESAFE_API_KEY`. Ranking quality has not been compared with other API
rerankers.
Closes#4045
## What changed
- replace the unpublished pylance 9.0.0rc1 test-extra pin with the
published 9.0.0 release
- restore dependency resolution for editable installs using the tests
extra
## Validation
- downloaded pylance==9.0.0 from PyPI with pip --no-deps
- parsed python/pyproject.toml with tomllib and verified the tests extra
- git diff --check
---------
Co-authored-by: Xuanwo <github@xuanwo.io>
## Summary
Add SQL execution to remote LanceDB connections. On the standard
synchronous connection, `execute_query` waits for the initial result
stream and returns its Arrow reader. `execute_query_async` is called
without Python `await` and immediately returns a query handle for status
inspection, streaming, or cancellation. Local databases report that SQL
is not supported.
The transport and query lifecycle live in Rust. Python exposes
native-backed synchronous and asynchronous connection methods and query
wrappers; it does not use PyArrow's Flight client.
## User experience
The standard synchronous connection supports both direct reads and
background query execution:
```python
db = lancedb.connect(
"db://analytics",
api_key="ldb_...",
sql_host_override="grpc+tls://sql.example.com:10026",
)
# Direct execution waits only until the initial result stream is available.
# Later batches continue streaming as the query progresses.
reader = db.execute_query(
"SELECT * FROM events",
default_namespace_path=["production"],
)
for batch in reader:
print(batch.num_rows)
# Background execution returns a query handle immediately. Despite the
# `_async` suffix, no Python `await` is needed on a synchronous connection.
query = db.execute_query_async("SELECT * FROM events")
print(query.id)
description = db.describe_query(query.id)
print(description.status)
print(description.progress)
print(description.expires_at)
# Start reading as soon as the service advertises partial results. The reader
# continues polling and yields newly available record batches until the query
# and all result endpoints are complete.
reader = query.reader()
for batch in reader:
print(batch.num_rows)
# Or cancel a different still-running query. Its status becomes "cancelling"
# while the server is still working, then "cancelled" once confirmed.
cancelled_query = db.execute_query_async("SELECT * FROM large_events")
cancelled_query.cancel()
```
The less commonly used asynchronous connection exposes the same
operations as coroutines:
```python
async_db = await lancedb.connect_async(
"db://analytics",
api_key="ldb_...",
sql_host_override="grpc+tls://sql.example.com:10026",
)
query = await async_db.execute_query_async("SELECT * FROM events")
async for batch in await query.reader():
print(batch.num_rows)
```
The UUIDv7 query id is scoped to the connection that submitted it. The
connection retains lightweight shared query state used by
`query.describe()` and `db.describe_query(query.id)`; the id does not
encode SQL or a Flight continuation token and is not a cross-connection
resume token. Abandoned state has bounded retention, and terminal state
remains available briefly.
Unqualified table names use the connected database and the `public`
namespace by default. `default_namespace_path` accepts a list such as
`["production", "events"]`. SQL can still use qualified names to
reference other databases and namespaces available to the deployment.
## Design
- Uses Arrow Flight `PollFlightInfo` for submission and long polling,
`DoGet` for results, and `CancelFlightInfo` for cancellation. Each
`PollInfo.info` is treated as the cumulative set of currently available
endpoints, so advertised tickets are consumed once and batches can be
delivered before execution is complete.
- Serializes result completion and cancellation into one lifecycle. A
server-accepted request reports `cancelling` and wakes blocked
status/result work; a later retry can confirm `cancelled`. Result
retrieval is rejected after cancellation is accepted, while cancellation
after a result was already delivered is a no-op.
- Assigns a time-ordered UUIDv7 connection-scoped query id and retains
only shared evolving lifecycle state, keeping SQL, Flight continuation
tokens, and Arrow result data out of public ids and the registry.
- Leaves admission control to the server while honoring server
expiration and a local fallback retention window for abandoned entries.
- Retains terminal ids for five minutes so they remain available for
connection-level description.
- Keeps one lazily initialized SQL client on each remote database
connection and attaches fresh authentication, routing, namespace, and
request metadata to every operation.
- Applies the configured overall timeout to each execution, description,
reader, and cancellation operation. A result reader carries one absolute
deadline from `reader()` through the end of streaming; connect and read
timeouts continue to bound their individual phases.
- Returns a bounded, backpressured, single-consumer Arrow stream rather
than collecting the full result in memory. Dropping the reader stops
downloading but does not implicitly cancel the server query.
- Preserves typed schemas for empty result sets through the stream
schema.
- Accepts Flight result messages up to 1 GiB so a valid row containing a
large blob, string, or vector is not rejected by tonic's 4 MiB default
receive limit.
- Supports the Python client first while keeping the authoritative
implementation in the Rust core.
## Problem
`uv run ... maturin develop` synchronizes the project as an editable
package before running the command. Maturin's editable build otherwise
uses the release profile, which enables the repository's fat LTO
configuration during local bootstrap.
## Behavior
Editable Python builds now explicitly use Cargo's dev profile. The
minimum maturin version is raised to 1.10, where `editable-profile`
support was introduced.
## Summary
- expose `memory_limit` and `num_workers` on the Python FTS
configuration for local builds
- forward both build-only settings to the Lance inverted-index builder
- add an end-to-end regression proving the configured memory budget
reaches the native build
## Root cause
LanceDB 0.26.1 pinned Lance 1.0.1. That Lance version used an FTS
partition-merge path whose retained data made memory grow with merge
progress on very large indexes. Upstream Lance
[#5754](https://github.com/lance-format/lance/pull/5754) changed
partition merging to stream its inputs, reducing peak memory by about
25%. Lance [#6174](https://github.com/lance-format/lance/pull/6174) then
removed the old merge phase, compressed posting lists during
construction, reduced indexing memory by about 60%, and introduced a
total build `memory_limit` for bounded workers.
Current `main` pins Lance 11.0.0-beta.3, which contains those
architectural fixes. This PR does not duplicate or claim the upstream
leak fix; it addresses the remaining Python API gap.
## This repair
LanceDB Python did not expose the native FTS builder resource controls.
`memory_limit` now sets the total local-build budget in MiB, divided
among effective workers, and `num_workers` controls build parallelism.
Both are build-only settings and do not affect remote builds or
persisted index configuration.
## Validation
- `cargo check --quiet --features remote --tests --examples`
- `cargo fmt --all`
- `uv run --project python --extra tests --extra dev ruff check .`
- `uv run --project python --extra tests --extra dev ruff format --check
python/python/lancedb/index.py python/python/tests/test_fts.py`
- `uv run --project python --extra tests pytest python/tests/test_fts.py
-q` (51 passed)
Fixes#2923
<!-- lance-gatekeeper-fix:v1 agent=a1ceedf74531e0212cb6f1ebf9390a26
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
LanceDB's Python SDK now requires Pydantic `>=2.7.4,<3` and uses the v2
APIs throughout. This removes dual-version behavior from schema
conversion, query serialization, embedding models, and Function wire
models while preserving their existing public and canonical-wire
behavior.
The minimum-dependencies CI job pins Pydantic 2.7.4 so the declared
compatibility floor remains covered.
## Root cause
`Table.to_polars()` disabled PyArrow predicate pushdown by selecting the
non-PyArrow Polars scan callback. Polars 1.32.3 invokes that callback
with `batch_size` both positionally and through its partial, so
collecting the returned lazy frame raises `TypeError:
_scan_pyarrow_dataset_impl() got multiple values for argument
batch_size`.
## Fix
- Keep the compatible PyArrow callback path.
- Add an identity `map_batches` barrier so predicates stay in Polars
instead of reaching the LanceDB adapter as unsupported PyArrow
expressions.
- Extend the tested Polars range through 1.32.3 and retain lazy-frame
regression coverage.
## Validation
- `python/tests/test_table.py::test_polars` with Polars 1.32.3
- `python/tests/test_table.py::test_polars` with the locked Polars 1.3.0
baseline
- `ruff format --check` on the changed Python files
- `ruff check .`
- `uv lock --check`
Fixes#2619
<!-- lance-gatekeeper-fix:v1 agent=0d42bcda944ac42765b25f2c19ff729f
generation=1 -->
---------
Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
## Summary
- publish the PEP 561 `py.typed` marker so downstream type checkers
consume the inline public annotations
- add a Pyright contract test that distinguishes synchronous `connect`
from awaited `connect_async`
- verify the marker is present in the installed package
## Root cause
The public Python module already annotated `lancedb.connect` as
synchronous and `lancedb.connect_async` as asynchronous. The private
native `_lancedb.connect` stub is intentionally awaitable because it
backs `connect_async`. However, the distribution did not include a PEP
561 marker, so downstream tools such as mypy could ignore the public
inline annotations and expose misleading or incomplete type information.
## Validation
- `python/.venv/bin/ruff format --check python/python/tests/test_db.py
python/python/type_tests/connect.py`
- `python/.venv/bin/ruff check .`
- `cd python && .venv/bin/pytest
python/tests/test_db.py::test_package_includes_pep_561_marker -q`
- `cd python && .venv/bin/pyright --pythonpath .venv/bin/python`
- downstream mypy contract check for both public connection functions
Fixes#2159
<!-- lance-gatekeeper-fix:v1 agent=b07901451487187fc03f61890d3aa6bb
generation=1 -->
Co-authored-by: lancedb-gatefixer[bot] <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
BREAKING CHANGE: splits generated by the permutation data loader will
not be the same, due to a change in hash function.
Updates the Lance dependencies and Java lance-core to
[v9.0.0-rc.1](https://github.com/lance-format/lance/releases/tag/v9.0.0-rc.1).
Includes the required DataFusion 54 and Lance file-reader compatibility
updates.
---------
Co-authored-by: Will Jones <willjones127@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds client-side support for analyze_plan distributed metrics modes
across Rust, Python, and TypeScript clients. Defaults to aggregate for
backward compatibility and sends the remote distributed_metrics
parameter only when a non-default mode is requested.
Bridges Lance's internal `metrics`-crate instrumentation (object store
request counts, bytes, latency, errors, and throttles) into
OpenTelemetry, in both the Python and Node bindings, with a shared
adapter in the Rust core. This is the LanceDB counterpart to
lance-format/lance#7537.
## Rust core (`rust/lancedb`)
Two new, **off-by-default** features:
- `metrics` — re-exports the [`metrics`](https://docs.rs/metrics) crate
as `lancedb::metrics` and turns on Lance's object-store instrumentation.
Install any `metrics`-compatible recorder to collect them.
- `metrics-otel` — adds `lancedb::metrics_otel`, a pull-based adapter
that installs a process-global recorder aggregating into lock-free
cumulative storage and exposes a snapshot/catalog API
(`register_metrics_recorder`, `metrics_catalog`, `snapshot_metrics`,
`MetricPoint`/`MetricValue`/`MetricKind`/`MetricDescription`). Both
bindings build on this.
## Python
`lancedb.otel.instrument_lancedb_metrics()` registers each metric as an
OpenTelemetry observable instrument on the given (or global)
`MeterProvider`. Available via the `otel` extra (`pip install
lancedb[otel]`), which pulls in only `opentelemetry-api` — the
application supplies and configures the SDK.
## Node
`instrumentLanceDbMetrics()` provides the equivalent wiring against
`@opentelemetry/api`. This is the only public entry point; the
underlying recorder/catalog/snapshot functions stay internal.
Because OpenTelemetry has no asynchronous histogram instrument,
histograms are exported Prometheus-style as `<name>_bucket` (with an
`le` attribute), `<name>_count`, and `<name>_sum`. Only `_sum` carries
the histogram's unit; `_bucket` and `_count` observe cumulative counts
and are unitless. The adapter is enabled by default in the Python and
Node builds, and off by default in the Rust crate.
## Notes
- Requires Lance ≥ `v9.0.0-beta.19`, which ships the object-store
metrics APIs (upstream lance-format/lance#7537, now merged). `main` is
already on beta.19, so this is a single feature commit with no
dependency bump.
- Tests: 8 Rust unit tests, 3 Python tests, 2 Node tests, all covering
the end-to-end object-store-metrics → OpenTelemetry path.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This follows the Rust-side Tantivy removal by deleting the remaining
Python Tantivy runtime, tests, and packaging references.
It also turns the legacy Python-only Tantivy parameters into explicit
errors and stops reading legacy `_indices/fts` directories so Python FTS
is fully native-only.
## Summary
- migrate gemini-text embedding provider from deprecated
google.generativeai to google.genai
- update Python embedding extra dependency to google-genai
- update default model name to gemini-embedding-001
- adapt embed calls to Client().models.embed_content(...)
- apply lint fixes from CI
## Related
- Closes#3191
1. Refactored every client (Rust core, Python, Node/TypeScript) so
“namespace” usage is explicit: code now keeps namespace paths
(namespace_path) separate from namespace clients (namespace_client).
Connections propagate the client, table creation routes through it, and
managed versioning defaults are resolved from namespace metadata. Python
gained LanceNamespaceDBConnection/async counterparts, and the
namespace-focused tests were rewritten to match the clarified API
surface.
2. Synchronized the workspace with Lance 5.0.0-beta.3 (see
https://github.com/lance-format/lance/pull/6186 for the upstream
namespace refactor), updating Cargo/uv lockfiles and ensuring all
bindings align with the new namespace semantics.
3. Added a namespace-backed code path to lancedb.connect() via new
keyword arguments (namespace_client_impl, namespace_client_properties,
plus the existing pushdown-ops flag). When those kwargs are supplied,
connect() delegates to connect_namespace, so users can opt into
namespace clients without changing APIs. (The async helper will gain
parity in a later change)
## Summary
- Update dependencies across Rust, Python, Node.js, Java, Docker, and
docs
- Pin unpinned dependency lower bounds to prevent silent downgrades
- Bump CI actions to current major versions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Without this fix, if user directly use the native table to do operations
like `add_columns`, even if it is configured to use namespace db
connection, it is not really propagated through.
The fix is to bring lancedb's python binding up to date and do a similar
implementation as https://github.com/lance-format/lance/pull/5968, and
make sure the namespace is fully propagated through all the related
calls.
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
## Summary
- bump all Lance crates to 1.0.0-beta.14 via ci/set_lance_version.py
- refresh Cargo.lock to capture new transitive requirements
- verified `cargo clippy --workspace --tests --all-features -- -D
warnings` and `cargo fmt --all`
Triggered by refs/tags/v1.0.0-beta.14
---------
Co-authored-by: Jack Ye <yezhaoqin@gmail.com>
Based on https://github.com/lancedb/lance/pull/4984
1. Bump to 1.0.0-beta.2
2. Use DirectoryNamespace in lance to perform all testing in python and
rust for much better coverage
3. Refactor `ListingDatabase` to be able to accept location and
namespace. This is because we have to leverage listing database (local
lancedb connection) for using namespace, namespace only resolves the
location and storage options but we don't want to bind all the way to
rust since user will plug-in namespace from python side. And thus
`ListingDatabase` needs to be able to accept location and namespace that
are created from namespace connection.
4. For credentials vending, we also pass storage options provider all
the way to rust layer, and the rust layer calls back to the python
function to fetch next storage option. This is exactly the same thing we
did in pylance.
## Description of changes
Fixes#2698
This PR uses
[`typing.override`](https://docs.python.org/3/library/typing.html#typing.override)
in favor of the [`overrides`](https://pypi.org/project/overrides/)
dependency when possible. As of Python 3.12, the standard library offers
`typing.override` to perform a static check on overridden methods.
### Motivation
Currently, `overrides` is incompatible with Python 3.14. As a result,
any package that attempts to import `overrides` using Python 3.14+ will
raise an `AttributeError`. An
[issue](https://github.com/mkorpela/overrides/issues/127) has been
raised and a [pull
request](https://github.com/mkorpela/overrides/pull/133) has been
submitted to the GitHub repo for the `overrides` project. But the
maintainer has been unresponsive.
To ensure readiness for Python 3.14, this package (and any other package
directly depending on `overrides`) should consider using
`typing.override` instead.
### Impact
The standard library added `typing.override` as of 3.12. As a result,
this change will affect only users of Python 3.12+. Previous versions
will continue to rely on `overrides`. Notably, the standard library
implementation is slightly different than that of `overrides`. A
thorough discussion of those differences is shown in [PEP
698](https://peps.python.org/pep-0698/), and it is also summarized
nicely by the maintainer of `overrides`
[here](https://github.com/mkorpela/overrides/issues/126#issuecomment-2401327116).
There are 2 main ways that switching from `overrides` to
`typing.override` will have an impact on developers of this repo.
1. `typing.override` does not implement any runtime checking. Instead,
it provides information to type checkers.
2. The stdlib does not provide a mixin class to enforce override
decorators on child classes. (Their reasoning for this is explained in
[the PEP](https://peps.python.org/pep-0698/).) This PR disables that
behavior entirely by replacing the `EnforceOverrides`.
Fixed the issue on lance-namespace side to avoid pinning to a specific
lance version. This should fix the issue of the increased release
artifact size and build time.
This PR integrates `lancedb` with `lance-namespace` so that users can
use LanceDB client to access Lance tables in any catalog services. In
general, we expect most of the logic to be delegated to the existing
`LanceDBConnection` and `LanceTable`, but the namespace implemenation
will control how table is created, dropped, and describe where the table
is stored with any related storage options like access credentials.
The implementation currently only supports a 1 level namespace that
directly contains tables. We will introduce nested namespace support in
a separated PR.
Users are expected to use it in the following way:
```python
>>> import lancedb
>>> import pyarrow as pa
>>> # Connect using GlueNamespace
>>> db = lancedb.connect_namespace("glue", {"catalog_id": "123456789012"})
>>> # Create a table with schema
>>> schema = pa.schema([
... pa.field("id", pa.int64()),
... pa.field("vector", pa.list_(pa.float32(), 2))
... ])
>>> table = db.create_table("my_table", schema=schema)
>>> # List tables
>>> db.table_names()
['my_table']
```
### Summary
This PR adds **SigLIP** (Sigmoid Loss Image Pretraining) as a new
embedding model in the LanceDB embedding registry. SigLIP improves
image-text alignment performance using sigmoid-based contrastive loss
and offers robust zero-shot generalization.
Fixes#2498
### What’s Implemented
#### 1. `SigLIP` Embedding Class
* Added `SigLIP` support under `python/lancedb/embeddings/siglip.py`
* Implements:
* `compute_source_embeddings`
* `_batch_generate_embeddings`
* Normalization logic
* Batch-wise progress logging for image embedding
#### 2. Registry Integration
* Registered `SigLIP` in `embeddings/__init__.py`
* `SigLIP` now usable via `connect(..., embedding="siglip")`
#### 3. Evaluation Benchmark Support
* Added SigLIP to `test_embeddings_slow.py` for side-by-side
benchmarking with OpenCLIP and ImageBind
### New Test Methods
#### `test_siglip`
* End-to-end test to verify embeddings table creation and vector shape
for SigLIP

#### `test_siglip_vs_openclip_vs_imagebind_benchmark_full`
* Benchmarks:
* **Recall\@1 / 5 / 10**
* **mAP (Mean Average Precision)**
* **Embedding & Search Latency**
* Dimensionality reporting

### Notes
* SigLIP outputs 768D embeddings (vs 512D for OpenCLIP)
* Benchmark shows competitive performance despite higher dimensionality
* I'm still new to contributing to open-source and learning as I go.
Please feel free to suggest any improvements — I'm happy to make
changes!
This patch fix can not build on python3.9 dev
the reason is that for ibm-watsonx-ai the min version is py3.10
more can check on `pyoven` https://pyoven.org/package/ibm-watsonx-ai/
also fix tiny md lint
---------
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Other embedding integrations such as Cohere and OpenAI already send
requests in batches. We should do that for Ollama too to improve
throughput.
The Ollama [`.embed`
API](https://github.com/ollama/ollama-python/blob/63ca74762284100b2f0ad207bc00fa3d32720fbd/ollama/_client.py#L359-L378)
was added in version 0.3.0 (almost a year ago) so I updated the version
requirement in pyproject.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
- **Bug Fixes**
- Improved compatibility with newer versions of the "ollama" package by
requiring version 0.3.0 or higher.
- Enhanced embedding generation to process batches of texts more
efficiently and reliably.
- **Refactor**
- Improved type consistency and clarity for embedding-related methods.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->