mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-29 06:56:15 +00:00
fix(node): support Apache Arrow 19 through 21 and run the library on each
Builds on #4359 (which added the Arrow 19-21 test aliases) and makes the package genuinely work across the whole peer range: * Resolve the library's own `apache-arrow` to 21.2.0 (devDependency) so the main CI jobs build, type-check and test on the newest release, and widen the peer range to `>=15.0.0 <22.0.0` instead of an exact pin. * Add a CI job that re-installs with `apache-arrow` resolved to 19.0.1 and 20.0.0 and runs the suite, covering what a consumer on those versions gets. * Arrow 21 overrides `Symbol.hasInstance` on Schema/Field/Table/RecordBatch/ Data so `instanceof` matches objects from another copy of the library, but not on DataType. The sanitizer's short-circuits therefore returned foreign schemas unchanged and Arrow's builders then failed with "Unrecognized type 'NONE'". Check the real prototype chain instead. * Sanitize the types added in Arrow 20/21 (LargeList, Utf8View, BinaryView, IntervalMonthDayNano), looked up dynamically so the library still compiles and runs on older Arrow releases, with a clear error when the installed Arrow lacks the type. Carry `variadicBuffers` through `Data`. * Loosen `FieldLike` so a branded DataType from another Arrow copy still type-checks; export the new `DataTypeLike`. * Share the Arrow version list between test files, exercise embedding function types per version in the registry tests, and add sanitize and end-to-end coverage for the new column types. * Enable `skipLibCheck`: flatbuffers 25 (via Arrow 21) ships declarations that need TypeScript 5.7+. * Record the new dev dependencies in the third-party license inventory. Co-Authored-By: Vincent Koc <vincentkoc@ieee.org> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Vincent Koc
Claude Fable 5.1
parent
8dfd371031
commit
003b87bfa0
@@ -3,16 +3,10 @@
|
||||
|
||||
import * as apiArrow from "apache-arrow";
|
||||
|
||||
import * as arrow15 from "apache-arrow-15";
|
||||
import * as arrow16 from "apache-arrow-16";
|
||||
import * as arrow17 from "apache-arrow-17";
|
||||
import * as arrow18 from "apache-arrow-18";
|
||||
import * as arrow19 from "apache-arrow-19";
|
||||
import * as arrow20 from "apache-arrow-20";
|
||||
import * as arrow21 from "apache-arrow-21";
|
||||
|
||||
import * as tmp from "tmp";
|
||||
|
||||
import { arrowVersions } from "./arrow_versions";
|
||||
|
||||
import { connect } from "../lancedb";
|
||||
import {
|
||||
EmbeddingFunction,
|
||||
@@ -21,16 +15,6 @@ import {
|
||||
} from "../lancedb/embedding";
|
||||
import { getRegistry, register } from "../lancedb/embedding/registry";
|
||||
|
||||
const arrowVersions = [
|
||||
arrow15,
|
||||
arrow16,
|
||||
arrow17,
|
||||
arrow18,
|
||||
arrow19,
|
||||
arrow20,
|
||||
arrow21,
|
||||
];
|
||||
|
||||
describe.each(arrowVersions)("LanceSchema", (arrow) => {
|
||||
test("should preserve input order", async () => {
|
||||
const schema = LanceSchema({
|
||||
@@ -253,7 +237,7 @@ describe("Registry.setVar", () => {
|
||||
return data.map(() => [1, 2, 3]);
|
||||
}
|
||||
embeddingDataType() {
|
||||
return new arrow18.Float32() as apiArrow.Float;
|
||||
return new apiArrow.Float32();
|
||||
}
|
||||
protected getSensitiveKeys() {
|
||||
return ["secretKey"];
|
||||
|
||||
Reference in New Issue
Block a user