mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-24 20:52:29 +00:00
fix(node): support Apache Arrow 19 through 21 and run the library on each
Builds on #4359 (which added the Arrow 19-21 test aliases) and makes the package genuinely work across the whole peer range: * Resolve the library's own `apache-arrow` to 21.2.0 (devDependency) so the main CI jobs build, type-check and test on the newest release, and widen the peer range to `>=15.0.0 <22.0.0` instead of an exact pin. * Add a CI job that re-installs with `apache-arrow` resolved to 19.0.1 and 20.0.0 and runs the suite, covering what a consumer on those versions gets. * Arrow 21 overrides `Symbol.hasInstance` on Schema/Field/Table/RecordBatch/ Data so `instanceof` matches objects from another copy of the library, but not on DataType. The sanitizer's short-circuits therefore returned foreign schemas unchanged and Arrow's builders then failed with "Unrecognized type 'NONE'". Check the real prototype chain instead. * Sanitize the types added in Arrow 20/21 (LargeList, Utf8View, BinaryView, IntervalMonthDayNano), looked up dynamically so the library still compiles and runs on older Arrow releases, with a clear error when the installed Arrow lacks the type. Carry `variadicBuffers` through `Data`. * Loosen `FieldLike` so a branded DataType from another Arrow copy still type-checks; export the new `DataTypeLike`. * Share the Arrow version list between test files, exercise embedding function types per version in the registry tests, and add sanitize and end-to-end coverage for the new column types. * Enable `skipLibCheck`: flatbuffers 25 (via Arrow 21) ships declarations that need TypeScript 5.7+. * Record the new dev dependencies in the third-party license inventory. Co-Authored-By: Vincent Koc <vincentkoc@ieee.org> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Vincent Koc
Claude Fable 5.1
parent
8dfd371031
commit
003b87bfa0
@@ -164,6 +164,59 @@ jobs:
|
||||
python ci/mock_openai.py &
|
||||
cd nodejs/examples
|
||||
node_modules/.bin/jest --testEnvironment jest-environment-node-single-context --verbose
|
||||
arrow-compat:
|
||||
# The main jobs resolve `apache-arrow` to the version pinned in
|
||||
# devDependencies (the newest supported release). This job re-runs the
|
||||
# suite with the library itself running on each older Arrow release the
|
||||
# peer range still accepts, which is what a consumer on that version gets.
|
||||
name: Linux (Arrow ${{ matrix.arrow-version }})
|
||||
timeout-minutes: 30
|
||||
strategy:
|
||||
matrix:
|
||||
arrow-version: [ "19.0.1", "20.0.0" ]
|
||||
runs-on: "ubuntu-22.04"
|
||||
defaults:
|
||||
run:
|
||||
shell: bash
|
||||
working-directory: nodejs
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
fetch-depth: 0
|
||||
lfs: true
|
||||
- uses: pnpm/action-setup@v6
|
||||
with:
|
||||
version: 11.1.1
|
||||
- uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: 24
|
||||
cache: 'pnpm'
|
||||
cache-dependency-path: nodejs/pnpm-lock.yaml
|
||||
- uses: Swatinem/rust-cache@v2
|
||||
with:
|
||||
save-if: ${{ github.ref == 'refs/heads/main' }}
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
sudo apt update
|
||||
sudo apt install -y protobuf-compiler libssl-dev
|
||||
- name: Build
|
||||
run: |
|
||||
pnpm install --frozen-lockfile
|
||||
# No `--` separator: pnpm forwards it literally, which would
|
||||
# make napi-rs treat `--profile ci` as a cargo passthrough arg.
|
||||
pnpm build:debug --profile ci
|
||||
- name: Resolve apache-arrow to ${{ matrix.arrow-version }}
|
||||
run: |
|
||||
npm pkg set devDependencies.apache-arrow=${{ matrix.arrow-version }}
|
||||
pnpm install --no-frozen-lockfile
|
||||
# `require("apache-arrow/package.json")` is blocked by the package's exports map.
|
||||
node -p "'apache-arrow resolved to ' + JSON.parse(require('fs').readFileSync('node_modules/apache-arrow/package.json')).version"
|
||||
pnpm tsc
|
||||
- name: Test
|
||||
env:
|
||||
# Newer @smithy/core uses dynamic ESM imports.
|
||||
NODE_OPTIONS: "--experimental-vm-modules"
|
||||
run: node_modules/.bin/jest --verbose
|
||||
macos:
|
||||
timeout-minutes: 30
|
||||
# macos-15 ships a newer linker; the older macos-14 linker fails to insert
|
||||
|
||||
Reference in New Issue
Block a user