mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-20 18:52:28 +00:00
Tobias
1ad97b148b
Closes #4246. lancedb publishes no source distribution, so a platform without a prebuilt wheel (#4104, #4239) has nothing to install. This adds an sdist to the Python release. ## The sdist job The release workflow builds an sdist with the maturin version of the wheel jobs. On a tag, the publish job uploads it with the wheels: to PyPI, and to Fury for beta tags. The job also runs on the pull request dry run, which now starts on a change to `python/pyproject.toml`. ## Checks - The job builds a wheel from the sdist with `--locked` and smoke-tests it in a fresh venv. - `ci/check_sdist.py` fails if a tracked file of the Python package is missing from the tarball. - `twine check --strict` checks that the metadata renders. `pydistcheck` enforces PyPI's 100 MiB size limit. - The unit tests of both check scripts run in the CI scripts workflow. ## Cargo.lock maturin ships the repository lockfile, which pins crates that only the dropped `nodejs` member uses, so `cargo --locked` fails (PyO3/maturin#2609). The job first runs `cargo metadata --locked` on the repository workspace, so a lockfile that has drifted from `Cargo.toml` fails there. It then removes the nodejs-only pins and repacks the sdist. `ci/check_cargo_lock.py` fails if the prune changes any package it keeps, or any dependency of one. The published file is the repacked one. ## Behavior change After the first release with an sdist, pip on a platform with no current wheel builds the newest version from source, which needs a Rust toolchain and `protoc`. Before, pip installed the last version with a wheel for that platform.
43 lines
1.2 KiB
Python
43 lines
1.2 KiB
Python
"""Fail if the lancedb sdist lacks a git-tracked file of the Python package.
|
|
|
|
Usage: python ci/check_sdist.py <sdist> (run from the repository root)
|
|
"""
|
|
|
|
import subprocess
|
|
import sys
|
|
import tarfile
|
|
from pathlib import Path
|
|
|
|
# Where git tracks the package, and where maturin puts it under the sdist root.
|
|
TRACKED_PACKAGE = "python/python/lancedb"
|
|
SDIST_PACKAGE = "python/lancedb"
|
|
|
|
|
|
def main() -> None:
|
|
sdist = Path(sys.argv[1])
|
|
root = sdist.name.removesuffix(".tar.gz")
|
|
listing = subprocess.run(
|
|
["git", "ls-files", "-z", "--", TRACKED_PACKAGE],
|
|
check=True,
|
|
capture_output=True,
|
|
text=True,
|
|
).stdout
|
|
tracked = [p for p in listing.split("\0") if p]
|
|
if not tracked:
|
|
sys.exit(f"error: git tracks no files under {TRACKED_PACKAGE}")
|
|
|
|
with tarfile.open(sdist) as tar:
|
|
members = {m.name for m in tar if m.isfile()}
|
|
missing = [
|
|
p
|
|
for p in tracked
|
|
if f"{root}/{SDIST_PACKAGE}{p.removeprefix(TRACKED_PACKAGE)}" not in members
|
|
]
|
|
if missing:
|
|
sys.exit(f"error: {sdist.name} lacks tracked sources: {', '.join(missing)}")
|
|
print(f"{sdist.name} OK: all {len(tracked)} tracked sources present")
|
|
|
|
|
|
if __name__ == "__main__":
|
|
main()
|