fix(node): support Apache Arrow 19 through 21 and run the library on each

Builds on #4359 (which added the Arrow 19-21 test aliases) and makes the
package genuinely work across the whole peer range:

* Resolve the library's own `apache-arrow` to 21.2.0 (devDependency) so
  the main CI jobs build, type-check and test on the newest release, and
  widen the peer range to `>=15.0.0 <22.0.0` instead of an exact pin.
* Add a CI job that re-installs with `apache-arrow` resolved to 19.0.1 and
  20.0.0 and runs the suite, covering what a consumer on those versions
  gets.
* Arrow 21 overrides `Symbol.hasInstance` on Schema/Field/Table/RecordBatch/
  Data so `instanceof` matches objects from another copy of the library,
  but not on DataType. The sanitizer's short-circuits therefore returned
  foreign schemas unchanged and Arrow's builders then failed with
  "Unrecognized type 'NONE'". Check the real prototype chain instead.
* Sanitize the types added in Arrow 20/21 (LargeList, Utf8View, BinaryView,
  IntervalMonthDayNano), looked up dynamically so the library still compiles
  and runs on older Arrow releases, with a clear error when the installed
  Arrow lacks the type. Carry `variadicBuffers` through `Data`.
* Loosen `FieldLike` so a branded DataType from another Arrow copy still
  type-checks; export the new `DataTypeLike`.
* Share the Arrow version list between test files, exercise embedding
  function types per version in the registry tests, and add sanitize and
  end-to-end coverage for the new column types.
* Enable `skipLibCheck`: flatbuffers 25 (via Arrow 21) ships declarations
  that need TypeScript 5.7+.
* Record the new dev dependencies in the third-party license inventory.

Co-Authored-By: Vincent Koc <vincentkoc@ieee.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Dan Tasse
2026-10-01 11:21:59 -05:00
co-authored by Vincent Koc Claude Fable 5.1
parent 8dfd371031
commit 003b87bfa0
25 changed files with 1876 additions and 1558 deletions
+3 -19
View File
@@ -3,16 +3,10 @@
import * as apiArrow from "apache-arrow";
import * as arrow15 from "apache-arrow-15";
import * as arrow16 from "apache-arrow-16";
import * as arrow17 from "apache-arrow-17";
import * as arrow18 from "apache-arrow-18";
import * as arrow19 from "apache-arrow-19";
import * as arrow20 from "apache-arrow-20";
import * as arrow21 from "apache-arrow-21";
import * as tmp from "tmp";
import { arrowVersions } from "./arrow_versions";
import { connect } from "../lancedb";
import {
EmbeddingFunction,
@@ -21,16 +15,6 @@ import {
} from "../lancedb/embedding";
import { getRegistry, register } from "../lancedb/embedding/registry";
const arrowVersions = [
arrow15,
arrow16,
arrow17,
arrow18,
arrow19,
arrow20,
arrow21,
];
describe.each(arrowVersions)("LanceSchema", (arrow) => {
test("should preserve input order", async () => {
const schema = LanceSchema({
@@ -253,7 +237,7 @@ describe("Registry.setVar", () => {
return data.map(() => [1, 2, 3]);
}
embeddingDataType() {
return new arrow18.Float32() as apiArrow.Float;
return new apiArrow.Float32();
}
protected getSensitiveKeys() {
return ["secretKey"];