## Summary
The Python wheel builds used different protoc versions: v24.4 on Linux,
v21.12 on Windows, and whatever Homebrew shipped on macOS. None of them
verified the download. A new composite action,
`.github/actions/setup-protoc`, now holds one pin, v36.2, with a SHA-256
for each platform's zip.
- macOS and Windows: the action downloads protoc, checks its hash, and
installs it.
- Linux: protoc has to be installed inside the manylinux container, so
the build reads the version and hash from the action's outputs and
checks the hash there.
protobuf does not publish checksum files for protoc. The pinned hashes
are the digests GitHub recorded for the v36.2 release assets. Older
releases such as v24.4 have none, which is why this moves to v36.2.
protoc only runs at build time, through `prost-build` in the Lance
crates. The Linux protoc binary is statically linked, so it runs in the
manylinux 2_17 container.
The action is also added to the path triggers of `pypi-publish.yml` and
`python.yml`, so a change to the pin runs the wheel builds.
Follow-up to
https://github.com/lancedb/lancedb/pull/4522#issuecomment-6160848344.
The sdist job in that PR can use the same action once this merges.
## Evidence
Running the action's install script locally on linux-aarch_64 installs
`libprotoc 36.2`, and a wrong hash fails with `protoc checksum
mismatch`. `cargo check -p lancedb` with `PROTOC` set to protoc 36.2
builds the Lance protos. The PyPI Publish dry run on this PR covers the
Linux, macOS and Windows builds.
## Merge Risk
**Door:** Two-way. CI only.
**Blast Radius:** Python wheel builds.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The python-v0.32.0 publish run failed on every build matrix entry. Three
independent issues:
1. **Mac and Windows**: `pypa/gh-action-pypi-publish` only runs on
Linux, but was being called inline from each build job.
2. **Linux (all arches)**: `pypa/gh-action-pypi-publish` derives its
docker image name from `github.action_repository`, which is empty when
the action is invoked from inside a composite action
(actions/runner#2473 — pypa's own `action.yml` references this bug). It
falls back to `github.repository`, generating
`docker://ghcr.io/lancedb/lancedb:<tag>`, which doesn't exist →
`denied`. Only the ARM matrix entry surfaced this because it failed
first and cancel-cascaded the rest.
3. **Windows**: `upload-artifact` in `build_windows_wheel` pointed at
`python\target\wheels`, but maturin writes to the workspace-root
`target/wheels`. The artifact was always empty. Also, `pypi-publish.yml`
passed a `vcpkg_token` input that the composite doesn't declare.
## Changes
- Build jobs (linux/mac/windows) now upload their wheels as
`actions/upload-artifact` artifacts.
- New Linux `publish` job downloads all wheel artifacts and runs the
Fury or PyPA publish step directly (not via a composite), so
`github.action_repository` resolves correctly.
- Delete the unused `upload_wheel` composite action.
- Drop the broken upload-artifact step inside `build_windows_wheel`.
- Remove the bogus `vcpkg_token` input.
- Fury upload now loops over all wheels instead of just the first.
- Bump `actions/checkout`, `actions/upload-artifact`,
`actions/download-artifact` to current major versions (Node 24) to clear
deprecation warnings.
- Bump Windows job timeout 60 → 90 minutes; previous run was
cancel-timing-out on a 60m cap.
- Use `rust-lld` as the Windows MSVC linker via
`CARGO_TARGET_X86_64_PC_WINDOWS_MSVC_LINKER`. `link.exe` is
single-threaded and the long pole on Windows builds.
Fixes#3445
## Test plan
- [x] Open this PR — `paths` filter triggers a dry-run build on all
three platforms.
- [x] Verify all three builds produce wheels.
- [x] Confirm the `pypa/gh-action-pypi-publish` container actually
starts (the actions/runner#2473 bug) via the `publish-dry-run` job
pointed at TestPyPI.
- [x] **REMOVE BEFORE MERGE**: drop the `publish-dry-run` job and the
now-redundant `actions/upload-artifact` runs on PRs (currently always-on
so the dry-run has wheels to publish).
- [ ] After merge, cherry-pick onto `python-v0.32.0` and force-push the
tag to re-trigger the publish.
This changes `lancedb` from a "pure python" setuptools project to a
maturin project and adds a rust lancedb dependency.
The async python client is extremely minimal (only `connect` and
`Connection.table_names` are supported). The purpose of this PR is to
get the infrastructure in place for building out the rest of the async
client.
Although this is not technically a breaking change (no APIs are
changing) it is still a considerable change in the way the wheels are
built because they now include the native shared library.
Will JonesandClaude Opus 5.5
Lei Xu
Jack Ye
Weston Pace