From 0d9c87a0792a5ff53b949dc7e18300ea62cda753 Mon Sep 17 00:00:00 2001 From: Will Jones Date: Fri, 10 Jul 2026 08:50:43 -0700 Subject: [PATCH] ci(nodejs): move Windows builds to larger runner and use ThinLTO (#3634) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `build - aarch64-pc-windows-msvc` node build job (and, marginally, the x86_64 one) had started hitting `rustc-LLVM ERROR: out of memory` while linking the `lancedb-nodejs` cdylib — most recently surfaced by #3526, which adds the goosefs backend (and its tonic/prost gRPC subtree) to the default node binary. The peak-memory step is the fat-LTO codegen (`lto=fat`, `codegen-units=1` from `.cargo/config.toml`), which merges the whole crate graph into a single LLVM module and runs single-threaded. It therefore neither parallelizes across cores nor fits in the 16 GB of the standard `windows-latest` runner as the dependency graph grows. This PR: - Moves both `*-pc-windows-msvc` node build jobs to `windows-2025-8x-x64` (more memory + cores). - Overrides the release profile to ThinLTO for just these jobs, via `CARGO_PROFILE_RELEASE_LTO=thin` / `CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16` in `pre_build`. ThinLTO parallelizes the cross-module optimization across the runner's cores and keeps peak memory well under the limit. Scoped so Python wheels and Rust release builds keep fat LTO. The larger runner alone would clear the OOM but waste the added cores on the single-threaded fat-LTO tail; ThinLTO is what makes the extra cores actually reduce wall-clock and gives durable memory headroom for future dependency growth. Tradeoff: ThinLTO can leave a small runtime-perf gap vs fat LTO for the node native binary, but it recovers most of it and is a common release configuration. Co-authored-by: Claude Opus 4.8 (1M context) --- .github/workflows/npm-publish.yml | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/.github/workflows/npm-publish.yml b/.github/workflows/npm-publish.yml index 3ebee5d96..5707b085a 100644 --- a/.github/workflows/npm-publish.yml +++ b/.github/workflows/npm-publish.yml @@ -103,7 +103,7 @@ jobs: features: fp16kernels pre_build: brew install protobuf - target: x86_64-pc-windows-msvc - host: windows-latest + host: windows-2025-8x-x64 features: "," pre_build: |- choco install --no-progress protoc ninja nasm @@ -111,12 +111,21 @@ jobs: # There is an issue where choco doesn't add nasm to the path export PATH="$PATH:/c/Program Files/NASM" nasm -v + # Fat LTO of the cdylib is single-threaded and the peak-memory + # step of the build, and had started hitting rustc-LLVM OOM on the + # Windows runners. ThinLTO parallelizes it across the runner's + # cores and keeps peak memory well under the limit. + export CARGO_PROFILE_RELEASE_LTO=thin + export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 - target: aarch64-pc-windows-msvc - host: windows-latest + host: windows-2025-8x-x64 features: "," pre_build: |- choco install --no-progress protoc rustup target add aarch64-pc-windows-msvc + # See ThinLTO note on the x86_64-pc-windows-msvc target above. + export CARGO_PROFILE_RELEASE_LTO=thin + export CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 - target: x86_64-unknown-linux-gnu host: ubuntu-latest features: fp16kernels