mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-15 08:12:26 +00:00
Will JonesandClaude Opus 5.5
eabec507a8
## Summary The Python wheel builds used different protoc versions: v24.4 on Linux, v21.12 on Windows, and whatever Homebrew shipped on macOS. None of them verified the download. A new composite action, `.github/actions/setup-protoc`, now holds one pin, v36.2, with a SHA-256 for each platform's zip. - macOS and Windows: the action downloads protoc, checks its hash, and installs it. - Linux: protoc has to be installed inside the manylinux container, so the build reads the version and hash from the action's outputs and checks the hash there. protobuf does not publish checksum files for protoc. The pinned hashes are the digests GitHub recorded for the v36.2 release assets. Older releases such as v24.4 have none, which is why this moves to v36.2. protoc only runs at build time, through `prost-build` in the Lance crates. The Linux protoc binary is statically linked, so it runs in the manylinux 2_17 container. The action is also added to the path triggers of `pypi-publish.yml` and `python.yml`, so a change to the pin runs the wheel builds. Follow-up to https://github.com/lancedb/lancedb/pull/4522#issuecomment-6160848344. The sdist job in that PR can use the same action once this merges. ## Evidence Running the action's install script locally on linux-aarch_64 installs `libprotoc 36.2`, and a wrong hash fails with `protoc checksum mismatch`. `cargo check -p lancedb` with `PROTOC` set to protoc 36.2 builds the Lance protos. The PyPI Publish dry run on this PR covers the Linux, macOS and Windows builds. ## Merge Risk **Door:** Two-way. CI only. **Blast Radius:** Python wheel builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
92 lines
4.0 KiB
YAML
92 lines
4.0 KiB
YAML
# We create a composite action to be re-used both for testing and for releasing
|
|
name: build-linux-wheel
|
|
description: "Build a manylinux wheel for lance"
|
|
inputs:
|
|
python-minor-version:
|
|
description: "10, 11, 12, 13"
|
|
required: true
|
|
args:
|
|
description: "--release"
|
|
required: false
|
|
default: ""
|
|
arm-build:
|
|
description: "Build for arm64 instead of x86_64"
|
|
# Note: this does *not* mean the host is arm64, since we might be cross-compiling.
|
|
required: false
|
|
default: "false"
|
|
manylinux:
|
|
description: "The manylinux version to build for"
|
|
required: false
|
|
default: "2_17"
|
|
package-name:
|
|
description: "Override [project] name in python/pyproject.toml (e.g. 'lancedb-compat'). Default keeps 'lancedb'."
|
|
required: false
|
|
default: "lancedb"
|
|
rustflags:
|
|
description: "RUSTFLAGS for the build container, as a single whitespace-free token (e.g. '-Ctarget-cpu=x86-64-v2'). Empty leaves RUSTFLAGS unset, keeping the defaults from .cargo/config.toml."
|
|
required: false
|
|
default: ""
|
|
runs:
|
|
using: "composite"
|
|
steps:
|
|
- name: Get protoc version
|
|
id: protoc
|
|
uses: ./.github/actions/setup-protoc
|
|
with:
|
|
install: "false"
|
|
- name: CONFIRM ARM BUILD
|
|
shell: bash
|
|
env:
|
|
ARM_BUILD: ${{ inputs.arm-build }}
|
|
run: |
|
|
echo "ARM BUILD: $ARM_BUILD"
|
|
- name: Patch package name for variant build
|
|
if: ${{ inputs.package-name != 'lancedb' }}
|
|
shell: bash
|
|
env:
|
|
PACKAGE_NAME: ${{ inputs.package-name }}
|
|
run: |
|
|
# Swap the [project] name so this build produces e.g. lancedb-compat
|
|
# wheels. The package still installs files under the lancedb/
|
|
# namespace -- import lancedb still works after pip install.
|
|
sed -i.bak 's/^name = "lancedb"$/name = "'"$PACKAGE_NAME"'"/' python/pyproject.toml
|
|
rm -f python/pyproject.toml.bak
|
|
grep '^name = ' python/pyproject.toml
|
|
- name: Build x86_64 Manylinux wheel
|
|
if: ${{ inputs.arm-build == 'false' }}
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
maturin-version: "1.12.4"
|
|
command: build
|
|
working-directory: python
|
|
docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/' -e PROTOC=/usr/local/bin/protoc -e PROTOC_VERSION=${{ steps.protoc.outputs.version }} -e PROTOC_SHA256=${{ steps.protoc.outputs.linux-x86_64-sha256 }} ${{ inputs.rustflags != '' && format('-e RUSTFLAGS={0}', inputs.rustflags) || '' }}"
|
|
target: x86_64-unknown-linux-gnu
|
|
manylinux: ${{ inputs.manylinux }}
|
|
args: ${{ inputs.args }}
|
|
before-script-linux: |
|
|
set -e
|
|
curl -fsSL https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-x86_64.zip -o /tmp/protoc.zip
|
|
echo "${PROTOC_SHA256} /tmp/protoc.zip" | sha256sum -c -
|
|
unzip /tmp/protoc.zip -d /usr/local
|
|
rm /tmp/protoc.zip
|
|
/usr/local/bin/protoc --version
|
|
- name: Build Arm Manylinux Wheel
|
|
if: ${{ inputs.arm-build == 'true' }}
|
|
uses: PyO3/maturin-action@v1
|
|
with:
|
|
maturin-version: "1.12.4"
|
|
command: build
|
|
working-directory: python
|
|
docker-options: "-e PIP_EXTRA_INDEX_URL='https://pypi.fury.io/lance-format/ https://pypi.fury.io/lancedb/' -e PROTOC=/usr/local/bin/protoc -e PROTOC_VERSION=${{ steps.protoc.outputs.version }} -e PROTOC_SHA256=${{ steps.protoc.outputs.linux-aarch_64-sha256 }} ${{ inputs.rustflags != '' && format('-e RUSTFLAGS={0}', inputs.rustflags) || '' }}"
|
|
target: aarch64-unknown-linux-gnu
|
|
manylinux: ${{ inputs.manylinux }}
|
|
args: ${{ inputs.args }}
|
|
before-script-linux: |
|
|
set -e
|
|
yum install -y clang
|
|
curl -fsSL https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-aarch_64.zip -o /tmp/protoc.zip
|
|
echo "${PROTOC_SHA256} /tmp/protoc.zip" | sha256sum -c -
|
|
unzip /tmp/protoc.zip -d /usr/local
|
|
rm /tmp/protoc.zip
|
|
/usr/local/bin/protoc --version
|