mirror of
https://github.com/lancedb/lancedb.git
synced 2026-08-18 03:58:26 +00:00
ci(nodejs): move Windows builds to larger runner and use ThinLTO (#3634)
The `build - aarch64-pc-windows-msvc` node build job (and, marginally, the x86_64 one) had started hitting `rustc-LLVM ERROR: out of memory` while linking the `lancedb-nodejs` cdylib — most recently surfaced by #3526, which adds the goosefs backend (and its tonic/prost gRPC subtree) to the default node binary. The peak-memory step is the fat-LTO codegen (`lto=fat`, `codegen-units=1` from `.cargo/config.toml`), which merges the whole crate graph into a single LLVM module and runs single-threaded. It therefore neither parallelizes across cores nor fits in the 16 GB of the standard `windows-latest` runner as the dependency graph grows. This PR: - Moves both `*-pc-windows-msvc` node build jobs to `windows-2025-8x-x64` (more memory + cores). - Overrides the release profile to ThinLTO for just these jobs, via `CARGO_PROFILE_RELEASE_LTO=thin` / `CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16` in `pre_build`. ThinLTO parallelizes the cross-module optimization across the runner's cores and keeps peak memory well under the limit. Scoped so Python wheels and Rust release builds keep fat LTO. The larger runner alone would clear the OOM but waste the added cores on the single-threaded fat-LTO tail; ThinLTO is what makes the extra cores actually reduce wall-clock and gives durable memory headroom for future dependency growth. Tradeoff: ThinLTO can leave a small runtime-perf gap vs fat LTO for the node native binary, but it recovers most of it and is a common release configuration. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -103,7 +103,7 @@ jobs:
|
||||
features: fp16kernels
|
||||
pre_build: brew install protobuf
|
||||
- target: x86_64-pc-windows-msvc
|
||||
host: windows-latest
|
||||
host: windows-2025-8x-x64
|
||||
features: ","
|
||||
pre_build: |-
|
||||
choco install --no-progress protoc ninja nasm
|
||||
@@ -111,12 +111,21 @@ jobs:
|
||||
# There is an issue where choco doesn't add nasm to the path
|
||||
export PATH="$PATH:/c/Program Files/NASM"
|
||||
nasm -v
|
||||
# Fat LTO of the cdylib is single-threaded and the peak-memory
|
||||
# step of the build, and had started hitting rustc-LLVM OOM on the
|
||||
# Windows runners. ThinLTO parallelizes it across the runner's
|
||||
# cores and keeps peak memory well under the limit.
|
||||
export CARGO_PROFILE_RELEASE_LTO=thin
|
||||
export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
|
||||
- target: aarch64-pc-windows-msvc
|
||||
host: windows-latest
|
||||
host: windows-2025-8x-x64
|
||||
features: ","
|
||||
pre_build: |-
|
||||
choco install --no-progress protoc
|
||||
rustup target add aarch64-pc-windows-msvc
|
||||
# See ThinLTO note on the x86_64-pc-windows-msvc target above.
|
||||
export CARGO_PROFILE_RELEASE_LTO=thin
|
||||
export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16
|
||||
- target: x86_64-unknown-linux-gnu
|
||||
host: ubuntu-latest
|
||||
features: fp16kernels
|
||||
|
||||
Reference in New Issue
Block a user