Will Jones
bf15655c83
Python was versioned and tagged separately from the Rust, Java, and Node.js SDKs, and had drifted three minor versions ahead (0.36 vs 0.33). Users had no way to tell which Python version corresponded to which Rust or Node release, and the gap had no meaning behind it. This unifies the two tracks so there is one version and one tag for all four SDKs. ## Version The shared version is set to `0.37.0-beta.0`. Python continues its own sequence (highest published: 0.36 → 0.37) while Rust, Java, and Node.js jump 0.33 → 0.37 to meet it. Picking Python's next minor means Python users see no discontinuity at all, and only the other SDKs skip forward. Note that `main` trails the `release/v0.32` branch on both lines (main is at 0.32.0-beta.3 / 0.35.0-beta.3; the release branch carries 0.33.0-beta.0 / 0.36.0-beta.0), so 0.37 is chosen to clear the highest tag on either branch. Every index stays monotonic: | index | publishes | last published | next | |---|---|---|---| | PyPI | stable only | 0.34.0 | 0.37.0 | | Fury | previews | 0.36.0b0 | 0.37.0-beta.1 | | npm | both | 0.33.0-beta.0 | 0.37.0-beta.1 | | crates.io | stable only | 0.31.0 | 0.37.0 | | Maven | both | 0.33.0-beta.0 | 0.37.0-beta.1 | A one-time jump for three SDKs, versus explaining the offset indefinitely. ## Mechanism * `python/.bumpversion.toml` is removed. `python/Cargo.toml` — the source of the Python package version, since `pyproject.toml` declares `dynamic = ["version"]` — becomes a tracked file of the root config. Its `cargo update -p lancedb-python` pre-commit hook is dropped as redundant: `ci/update_lockfiles.sh` already refreshes every workspace member version in `Cargo.lock`. * `pypi-publish.yml` triggers on `v*` instead of `python-v*`, so one tag releases all four packages. `ci/bump_version.sh` and `make-release-commit.yml` lose their now-dead tag-prefix and per-language plumbing, including the `python` / `other` dispatch inputs. * The two byte-identical GH release jobs in `npm-publish.yml` and `pypi-publish.yml` are replaced by a single `gh-release.yml`. One release per tag, named `LanceDB vX.Y.Z`, instead of separate "Python LanceDB" and "Node/Rust LanceDB" releases for the same commit. The trade-off: there is no longer a way to ship a Python-only patch without also releasing crates.io, Maven, and npm. That is the cost of making drift structurally impossible. ## Beta releases marked "Latest" (#3666) Both GH release jobs used: ```yaml prerelease: ${{ contains('beta', github.ref) }} ``` The arguments are reversed. `contains(search, item)` asks whether *`search`* contains *`item`*, so this evaluated "does the literal string `'beta'` contain `refs/tags/python-v0.35.0-beta.2`?" — always `false`. Every beta was published as a full release, and GitHub awards "Latest" to the newest non-prerelease. The new workflow derives the flag from the parsed version rather than the raw ref, and sets `make_latest` explicitly: ```yaml prerelease: ${{ steps.extract_version.outputs.prerelease }} make_latest: ${{ steps.extract_version.outputs.prerelease == 'false' }} ``` npm was never affected (`--tag preview` uses correct bash), and PyPI already excludes pre-releases from resolution. This only fixes releases published from here on. Already-published betas need a one-time backfill: ```shell gh api --paginate /repos/lancedb/lancedb/releases \ --jq '.[] | select(.prerelease == false) | select(.tag_name | test("beta")) | .id' \ | xargs -I{} gh api -X PATCH /repos/lancedb/lancedb/releases/{} -F prerelease=true ``` ## Verification Ran `ci/bump_version.sh` end-to-end against this branch with the release tooling installed: * `preview` → tags `v0.37.0-beta.1` (previous tag `v0.33.0-beta.0` detected, `pre_n` bump) * `stable` → tags `v0.37.0` * Both paths update `.bumpversion.toml`, `rust/lancedb/Cargo.toml`, `nodejs/Cargo.toml`, `python/Cargo.toml`, `nodejs/package.json`, the 7 `nodejs/npm/*/package.json` files, both Java poms, and `docs/src/java/java.md` together * `check_breaking_changes.py` resolves the last stable as `v0.31.0`, so the minor-version gate passes All five touched workflows parse as valid YAML and the pre-commit hooks pass. ## Notes for review * This targets `main` only, so it takes effect at the next release-branch cut. The in-flight `release/v0.32` branch still carries `v0.33.0-beta.0` / `python-v0.36.0-beta.0`; if we want the imminent stable to be 0.37.0, this needs to be applied there too. * Historical `python-v*` tags are left alone. The changelog builder scans `^v`, which does not match them, so the first unified release's notes will compute `fromTag` from the Rust/Node line only — a one-time gap in the Python-side changelog. * Pre-existing and not addressed here: `ci/update_lockfiles.sh --amend` amends the commit that `bump-my-version` has already tagged, so the lockfile update lands outside the tag on stable releases. Fixes #3666
LanceDB Python SDK
A Python library for LanceDB.
Installation
pip install lancedb
Pre-Haswell x86_64 hosts: lancedb-compat
The default lancedb wheel targets x86-64-haswell (AVX2 + FMA + F16C) for full performance on modern hardware. Pre-Haswell hosts — Intel Sandy Bridge / Ivy Bridge / Westmere; AMD Bulldozer / Piledriver / Steamroller — don't have AVX2 and crash with Illegal instruction at import lancedb.
For those hosts, install the lancedb-compat package instead:
pip install lancedb-compat
Same Python API (import lancedb works as usual). The compat wheel is compiled at the x86-64-v2 baseline (Nehalem-class) and uses runtime SIMD dispatch in the embedded lance crate to pick the right kernel tier (scalar / AVX / AVX+FMA / AVX2+FMA / AVX-512) at load time, so it still goes fast on modern hardware while running cleanly on the pre-Haswell silicon. Use lance.simd_info() from Python to verify which tier was selected.
lancedb and lancedb-compat install to the same lancedb/ namespace and conflict at install time. Pick one. To switch, pip uninstall lancedb first, then pip install lancedb-compat (or vice-versa).
If you need a custom baseline (or lancedb-compat isn't yet published for your platform), build from source with the override:
RUSTFLAGS="-C target-cpu=x86-64-v2" maturin build --release
pip install ./target/wheels/lancedb-*.whl
Preview Releases
Stable releases are created about every 2 weeks. For the latest features and bug fixes, you can install the preview release. These releases receive the same level of testing as stable releases, but are not guaranteed to be available for more than 6 months after they are released. Once your application is stable, we recommend switching to stable releases.
pip install --pre --extra-index-url https://pypi.fury.io/lancedb/ lancedb
Usage
Basic Example
import lancedb
db = lancedb.connect('<PATH_TO_LANCEDB_DATASET>')
table = db.open_table('my_table')
results = table.search([0.1, 0.3]).limit(20).to_list()
print(results)
Development
See CONTRIBUTING.md for information on how to contribute to LanceDB.