mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-16 07:36:50 +00:00
Existing Rust and Python blob range calls must keep working when only the SDK is upgraded. Add negotiated batched range reads across Rust, Python sync/async and TypeScript, preserving ordered NULLs, duplicates and branch/version reads with splitting and reassembly at 1,024 ranges or 64 MiB per request. Only a successful describe response advertising `x-lancedb-blob-ranges: 1` selects the batched endpoint. Missing or unknown capability values select the existing descriptor-query plus BlobFile Range GET route before the operation. Endpoint failures never switch protocols. HTTP 400 preserves the server message and request ID as Rust `InvalidInput` / Python `ValueError`; other errors retain their existing semantics. The operation binds both numeric version and immutable snapshot identity from its producing query or first range response. Every later partition and oversized-range piece sends that original identity and requires an equal response identity before decoding. Same-name table recreation therefore fails explicitly instead of returning mixed old/new payloads; no partial array is returned on later failure. Targets `main`. The read-buffering prerequisite #4979, safe-write prerequisite #4977 and deferred-query identity prerequisite #4973 are merged into main; #5000 also supplies the merged SQL timeout-test prerequisite. Server support is [Sophon #10219](https://github.com/lancedb/sophon/pull/10219), including #10211. Exact SDK `1497b508324662180086a4d1e0a6eb9e0b3970db` and server `f39712c2c150b220eb831e00f6663c9eef32dce5` passed a real local SDK → REST → PE experiment: one Rust harness test, 14 assertion groups, 38 REST range requests and 34 PE RPCs. All four sync/async × warm/cold table-recreation cases failed explicitly before the second descriptor lookup or PE dispatch, with no mixed payload or partial result. The other groups retained 1,025-range success, 70 MiB reassembly, branch/version reads, overwrite snapshot consistency, error behavior and TypeScript buffer ownership. Bindings were rebuilt and their loaded hashes verified. This is local interoperability evidence, not Cloud, mixed-fleet or deployed header-forwarding acceptance. The server dependency upgrade, External/cache/URI-policy acceptance and validated rollout remain required. Enforcing PE support must precede QN and SDK enablement; identity does not retain versions or freeze External objects. This PR does not claim deployed support or complete ENT-3100/GA acceptance. Published documentation retains the Early access badge and released Enterprise instructions, including explicit fetches for copying. The new Enterprise `blob_mode="bytes"` guidance, snapshot-bound copy example, future streaming restrictions and GA wording remain source-only MDX drafts until release. The correction distinguishing released Enterprise SQL bytes from SDK descriptors remains visible. Documentation validation checked comment boundaries, visible instructions, Python example syntax and whitespace; the site was not built. The main integration keeps these release gates unchanged. Main integration at `0491d9bdcc65ce5d52ea93e1083c89e951ea4f9d` normally merges main `9b13e8f72530c8b1950c09685f6920711154a539`, including its Lance beta.18 dependency. Focused current-source validation passed: 493 remote-table/query/Blob/IPC/write-progress Rust tests, 23 shared create/write-body tests, 59 native Blob integration tests and 7 public HTTP range tests. Rebuilt Python passed 468 selected tests (one optional PyAV test skipped), and rebuilt TypeScript passed 194 selected tests across four suites; lint/format, TypeScript compilation and API documentation generation passed. Shared tests include actual HTTP multipart failures, acknowledgment boundaries, dictionary compaction and native Blob compaction. The 598 unrelated Jest cases excluded by the focused shared-suite selector were not run. The retained 14-group joint REST/PE experiment used the earlier exact candidates stated above. Operation-identity/range/IPC code is unchanged, but dependencies and shared write interfaces changed with main; the new local checks validate the SDK combination and do not constitute a fresh final-head SDK/server joint, Cloud or rollout run. Early access and every remaining server/External/cache/policy/dependency/rollout condition remain in force. CI snapshot on this head (2026-11-12 18:30 UTC): `build-no-lock` failed in the standalone `ci/tests/test_build_no_lock.py` fixture at the retained-rlib assertion, before the actual SDK no-lock build. The fixture, helper and Rust workflow are byte-identical to main; the fixture does not load SDK source. The underlying CI/environment cause is not diagnosed here. Other jobs remained pending/running at the snapshot, so CI is not reported as passing. Failure: https://github.com/lancedb/lancedb/actions/runs/43280277756/job/131399418284.
590 lines
19 KiB
TypeScript
590 lines
19 KiB
TypeScript
// SPDX-License-Identifier: Apache-2.0
|
|
// SPDX-FileCopyrightText: Copyright The LanceDB Authors
|
|
|
|
import { createHash } from "crypto";
|
|
import { getEventListeners } from "events";
|
|
import { createReadStream } from "fs";
|
|
import * as http from "http";
|
|
import { AddressInfo } from "net";
|
|
import {
|
|
Table as ArrowTable,
|
|
Binary,
|
|
Field,
|
|
LargeBinary,
|
|
Struct,
|
|
Uint8,
|
|
Uint32,
|
|
Uint64,
|
|
Utf8,
|
|
tableToIPC,
|
|
vectorFromArray,
|
|
} from "apache-arrow";
|
|
import { BlobRangeRequest, Connection, Table, connect } from "../lancedb";
|
|
|
|
jest.setTimeout(30000);
|
|
|
|
const IDENTITY = `v1:${"a".repeat(64)}`;
|
|
const MIB = 1024 * 1024;
|
|
type Range = {
|
|
// biome-ignore lint/style/useNamingConvention: Matches the server JSON contract.
|
|
row_id: number;
|
|
offset: number;
|
|
length: number;
|
|
};
|
|
type Request = {
|
|
column: string;
|
|
branch: string | null;
|
|
version: number | null;
|
|
ranges: Range[];
|
|
identity?: string;
|
|
};
|
|
|
|
function payload(row: number, offset: number, length: number, version: number) {
|
|
const bytes = Buffer.alloc(length);
|
|
for (let i = 0; i < length; i++) {
|
|
bytes[i] = (row * 17 + ((offset + i) % 251) + version) % 251;
|
|
}
|
|
return bytes;
|
|
}
|
|
|
|
async function withRangeServer(
|
|
run: (db: Connection, calls: Request[], paths: string[]) => Promise<void>,
|
|
fault?: string | number,
|
|
capability: string | null = "1",
|
|
onStall?: () => void,
|
|
) {
|
|
let takes = 0;
|
|
const calls: Request[] = [];
|
|
const paths: string[] = [];
|
|
const server = http.createServer((req, res) => {
|
|
req.setTimeout(15000);
|
|
paths.push(req.url ?? "");
|
|
if (
|
|
req.url === "/v1/table/test/describe/" ||
|
|
req.url === "/v1/table/test/branches/list/"
|
|
) {
|
|
res.writeHead(200, {
|
|
"Content-Type": "application/json",
|
|
"phalanx-version": "1.13.0-beta.0",
|
|
...(req.url.endsWith("/describe/") && capability !== null
|
|
? { "x-lancedb-blob-ranges": capability }
|
|
: {}),
|
|
});
|
|
const response = req.url.endsWith("branches/list/")
|
|
? {
|
|
branches: {
|
|
experiment: {
|
|
parentBranch: null,
|
|
parentVersion: 1,
|
|
createAt: 1,
|
|
manifestSize: 1,
|
|
},
|
|
},
|
|
}
|
|
: { version: 8, schema: { fields: [] } };
|
|
res.end(JSON.stringify(response));
|
|
return;
|
|
}
|
|
if (capability !== "1" && req.method === "GET") {
|
|
const url = new URL(req.url!, "http://localhost");
|
|
expect(url.searchParams.get("branch")).toBe("experiment");
|
|
expect(url.searchParams.get("version")).toBe("3");
|
|
const [start, end] = req.headers.range!.slice(6).split("-").map(Number);
|
|
const bytes = Buffer.from("alpha").subarray(start, end + 1);
|
|
res
|
|
.writeHead(206, {
|
|
"Content-Range": `bytes ${start}-${end}/5`,
|
|
"Content-Length": bytes.length,
|
|
"x-lancedb-version": "3",
|
|
})
|
|
.end(bytes);
|
|
return;
|
|
}
|
|
expect(req.method).toBe("POST");
|
|
expect(req.url).toBe(
|
|
capability === "1"
|
|
? "/v1/table/test/fetch_blob_ranges/"
|
|
: "/v1/table/test/query/",
|
|
);
|
|
const chunks: Buffer[] = [];
|
|
let size = 0;
|
|
req.on("data", (chunk: Buffer) => {
|
|
size += chunk.length;
|
|
expect(size).toBeLessThanOrEqual(256 * 1024);
|
|
chunks.push(chunk);
|
|
});
|
|
req.on("end", () => {
|
|
if (capability !== "1") {
|
|
const body = JSON.parse(Buffer.concat(chunks).toString()) as {
|
|
branch: string;
|
|
version: number;
|
|
filter: string;
|
|
};
|
|
expect(body.branch).toBe("experiment");
|
|
expect(body.version).toBe(3);
|
|
takes++;
|
|
const ids = (body.filter.match(/\d+/g) ?? [])
|
|
.map(Number)
|
|
.filter((id) => [10, 20, 40].includes(id))
|
|
.reverse();
|
|
const descriptor = new Struct([
|
|
new Field("kind", new Uint8(), false),
|
|
new Field("position", new Uint64(), false),
|
|
new Field("size", new Uint64(), false),
|
|
new Field("blob_id", new Uint32(), false),
|
|
new Field("blob_uri", new Utf8(), false),
|
|
]);
|
|
const arrow = new ArrowTable({
|
|
image: vectorFromArray(
|
|
ids.map((id) =>
|
|
id === 20
|
|
? null
|
|
: {
|
|
kind: 0,
|
|
position: 0n,
|
|
size: id === 40 ? 0n : 5n,
|
|
// biome-ignore lint/style/useNamingConvention: Arrow descriptor field.
|
|
blob_id: 0,
|
|
// biome-ignore lint/style/useNamingConvention: Arrow descriptor field.
|
|
blob_uri: "",
|
|
},
|
|
),
|
|
descriptor,
|
|
),
|
|
_rowid: vectorFromArray(ids.map(BigInt), new Uint64()),
|
|
});
|
|
expect(takes).toBeGreaterThan(0);
|
|
res
|
|
.writeHead(200, {
|
|
"Content-Type": "application/vnd.apache.arrow.stream",
|
|
"x-lancedb-version": "3",
|
|
})
|
|
.end(tableToIPC(arrow, "stream"));
|
|
return;
|
|
}
|
|
const body = JSON.parse(Buffer.concat(chunks).toString()) as Request;
|
|
body.identity = req.headers["x-lancedb-query-snapshot"] as
|
|
| string
|
|
| undefined;
|
|
calls.push(body);
|
|
expect(body.column).toBe("image");
|
|
expect(body.branch).toBe("experiment");
|
|
expect(body.ranges.length).toBeLessThanOrEqual(1024);
|
|
expect(
|
|
body.ranges.reduce((sum, range) => sum + range.length, 0),
|
|
).toBeLessThanOrEqual(64 * MIB);
|
|
const version = body.version ?? (calls.length === 1 ? 7 : 8);
|
|
const failure =
|
|
calls.length > 1 || String(fault).startsWith("first-")
|
|
? fault
|
|
: undefined;
|
|
if (failure === "recreated" || failure === "gone") {
|
|
res
|
|
.writeHead(failure === "recreated" ? 400 : 404)
|
|
.end("query snapshot identity mismatch: source gone or replaced");
|
|
return;
|
|
}
|
|
if (failure === "stall" && calls.length === 2) {
|
|
res.writeHead(200, {
|
|
"Content-Type": "application/vnd.apache.arrow.stream",
|
|
"x-lancedb-version": "7",
|
|
"x-lancedb-query-snapshot": IDENTITY,
|
|
});
|
|
res.flushHeaders();
|
|
onStall?.();
|
|
return;
|
|
}
|
|
if (typeof failure === "number") {
|
|
res.writeHead(failure).end("ordinary server failure");
|
|
return;
|
|
}
|
|
if (failure === "disconnect") {
|
|
req.socket.destroy();
|
|
return;
|
|
}
|
|
let values = body.ranges.map((range) =>
|
|
range.row_id === 20
|
|
? null
|
|
: payload(range.row_id, range.offset, range.length, version),
|
|
);
|
|
if (failure === "count") values = [];
|
|
if (failure === "length") values = [Buffer.alloc(0)];
|
|
const arrow = new ArrowTable({
|
|
[failure === "column" ? "wrong" : "value"]: vectorFromArray(
|
|
values,
|
|
failure === "type" ? new Binary() : new LargeBinary(),
|
|
),
|
|
});
|
|
let ipc = tableToIPC(arrow, "stream");
|
|
if (failure === "eos") ipc = ipc.subarray(0, ipc.length - 8);
|
|
if (failure === "body") ipc = ipc.subarray(0, Math.floor(ipc.length / 2));
|
|
res.writeHead(200, {
|
|
"Content-Type": "application/vnd.apache.arrow.stream",
|
|
connection: "close",
|
|
"Content-Length": ipc.length + (failure === "transport" ? 10 : 0),
|
|
...(failure === "version" ? {} : { "x-lancedb-version": version }),
|
|
...(["missing-identity", "first-missing-identity"].includes(
|
|
String(failure),
|
|
)
|
|
? {}
|
|
: {
|
|
"x-lancedb-query-snapshot":
|
|
failure === "wrong-identity"
|
|
? `v1:${"b".repeat(64)}`
|
|
: failure === "first-invalid-identity"
|
|
? "broken"
|
|
: IDENTITY,
|
|
}),
|
|
});
|
|
// Fragment the generated IPC payload across actual HTTP writes.
|
|
for (let i = 0; i < ipc.length; i += 8192)
|
|
res.write(ipc.subarray(i, i + 8192));
|
|
res.end();
|
|
});
|
|
});
|
|
server.listen(0, "127.0.0.1");
|
|
await new Promise<void>((resolve) => server.once("listening", resolve));
|
|
const port = (server.address() as AddressInfo).port;
|
|
const db = await connect("db://dev", {
|
|
apiKey: "fake",
|
|
hostOverride: `http://127.0.0.1:${port}`,
|
|
clientConfig: {
|
|
retryConfig: { retries: 2, connectRetries: 2, readRetries: 2 },
|
|
blobRequestConcurrency: 1,
|
|
timeoutConfig: { timeout: 20 },
|
|
},
|
|
});
|
|
try {
|
|
await run(db, calls, paths);
|
|
} finally {
|
|
await new Promise<void>((resolve) => {
|
|
server.closeAllConnections();
|
|
server.close(() => resolve());
|
|
});
|
|
}
|
|
}
|
|
|
|
async function open(db: Connection, version?: number): Promise<Table> {
|
|
return db.openTable("test", [], { branch: "experiment", version });
|
|
}
|
|
|
|
describe("fetchBlobRanges", () => {
|
|
it.each([null, "0", "2"])(
|
|
"uses old routes before data requests for capability %s",
|
|
async (capability) => {
|
|
await withRangeServer(
|
|
async (db, calls, paths) => {
|
|
// Reopening exercises both the initial and cached database describe paths.
|
|
for (let i = 0; i < 2; i++) {
|
|
const table = await open(db, 3);
|
|
const requests = [
|
|
{ rowId: 10n, offset: 1n, length: 3n },
|
|
{ rowId: 20n, offset: 0n, length: 1n },
|
|
{ rowId: 40n, offset: 0n, length: 0n },
|
|
{ rowId: 10n, offset: 5n, length: 0n },
|
|
{ rowId: 10n, offset: 1n, length: 3n },
|
|
];
|
|
expect(await table.fetchBlobRanges("image", requests)).toEqual([
|
|
Buffer.from("lph"),
|
|
null,
|
|
Buffer.alloc(0),
|
|
Buffer.alloc(0),
|
|
Buffer.from("lph"),
|
|
]);
|
|
for (const row of [
|
|
{ rowId: 10n, offset: 4n, length: 2n },
|
|
{ rowId: 99n, offset: 0n, length: 1n },
|
|
])
|
|
await expect(
|
|
table.fetchBlobRanges("image", [row]),
|
|
).rejects.toThrow();
|
|
const before = paths.length;
|
|
expect(await table.fetchBlobRanges("image", [])).toEqual([]);
|
|
expect(paths).toHaveLength(before);
|
|
}
|
|
expect(calls).toHaveLength(0);
|
|
expect(paths.some((path) => path.includes("fetch_blob_ranges"))).toBe(
|
|
false,
|
|
);
|
|
},
|
|
undefined,
|
|
capability,
|
|
);
|
|
},
|
|
);
|
|
|
|
it("renegotiates advertised support when reopening a cached table", async () => {
|
|
await withRangeServer(async (db, calls) => {
|
|
for (let i = 0; i < 2; i++) {
|
|
const table = await open(db);
|
|
expect(
|
|
await table.fetchBlobRanges("image", [
|
|
{ rowId: 10n, offset: 0n, length: 1n },
|
|
]),
|
|
).toEqual([payload(10, 0, 1, i === 0 ? 7 : 8)]);
|
|
}
|
|
expect(calls).toHaveLength(2);
|
|
});
|
|
});
|
|
|
|
it("cancels while waiting for a later response body and releases the only permit", async () => {
|
|
let markStalled!: () => void;
|
|
const stalled = new Promise<void>((resolve) => {
|
|
markStalled = resolve;
|
|
});
|
|
await withRangeServer(
|
|
async (db, calls) => {
|
|
const table = await open(db);
|
|
const controller = new AbortController();
|
|
const pending = table.fetchBlobRanges(
|
|
"image",
|
|
Array.from({ length: 2049 }, () => ({
|
|
rowId: 10n,
|
|
offset: 0n,
|
|
length: 1n,
|
|
})),
|
|
{ signal: controller.signal },
|
|
);
|
|
const rejected = expect(pending).rejects.toMatchObject({
|
|
name: "AbortError",
|
|
});
|
|
await stalled;
|
|
controller.abort();
|
|
await rejected;
|
|
expect(getEventListeners(controller.signal, "abort")).toHaveLength(0);
|
|
expect(calls).toHaveLength(2);
|
|
expect(
|
|
await table.fetchBlobRanges("image", [
|
|
{ rowId: 10n, offset: 0n, length: 1n },
|
|
]),
|
|
).toEqual([payload(10, 0, 1, 8)]);
|
|
expect(calls).toHaveLength(3);
|
|
},
|
|
"stall",
|
|
"1",
|
|
() => markStalled(),
|
|
);
|
|
});
|
|
it.each([undefined, 3])(
|
|
"preserves order, nulls, duplicates and snapshot (checkout %s)",
|
|
async (version) => {
|
|
await withRangeServer(async (db, calls, paths) => {
|
|
const table = await open(db, version);
|
|
const requests: BlobRangeRequest[] = Array.from(
|
|
{ length: 1025 },
|
|
(_, i) => ({
|
|
rowId: i % 9 === 0 ? 20 : 10 + (i % 7),
|
|
offset: BigInt(i),
|
|
length: BigInt(i % 4),
|
|
}),
|
|
);
|
|
requests.push(
|
|
{ rowId: 10n, offset: 0n, length: 0n },
|
|
{ rowId: 10n, offset: 2n, length: 3n },
|
|
{ rowId: 10n, offset: 2n, length: 3n },
|
|
);
|
|
const result = await table.fetchBlobRanges("image", requests);
|
|
expect(result).toEqual(
|
|
requests.map((range) =>
|
|
Number(range.rowId) === 20
|
|
? null
|
|
: payload(
|
|
Number(range.rowId),
|
|
Number(range.offset),
|
|
Number(range.length),
|
|
version ?? 7,
|
|
),
|
|
),
|
|
);
|
|
expect(calls.map((body) => body.version)).toEqual([
|
|
version ?? null,
|
|
version ?? 7,
|
|
]);
|
|
expect(
|
|
await table.fetchBlobRanges("image", [
|
|
{ rowId: 10n, offset: 0n, length: 1n },
|
|
]),
|
|
).toEqual([payload(10, 0, 1, version ?? 8)]);
|
|
expect(calls[2].version).toBe(version ?? null);
|
|
expect(
|
|
paths.filter((path) => path.endsWith("fetch_blob_ranges/")),
|
|
).toHaveLength(3);
|
|
});
|
|
},
|
|
);
|
|
|
|
it("partitions the byte budget and reassembles one range larger than 64 MiB", async () => {
|
|
await withRangeServer(async (db, calls) => {
|
|
const table = await open(db);
|
|
const requests = [
|
|
{ rowId: 10n, offset: 3n, length: BigInt(65 * MIB) },
|
|
{ rowId: 30n, offset: 7n, length: 2n },
|
|
];
|
|
const result = await table.fetchBlobRanges("image", requests);
|
|
for (const [i, range] of requests.entries()) {
|
|
expect(result[i]?.length).toBe(Number(range.length));
|
|
expect(
|
|
createHash("sha256")
|
|
.update(result[i] as Buffer)
|
|
.digest("hex"),
|
|
).toBe(
|
|
createHash("sha256")
|
|
.update(
|
|
payload(
|
|
Number(range.rowId),
|
|
Number(range.offset),
|
|
Number(range.length),
|
|
7,
|
|
),
|
|
)
|
|
.digest("hex"),
|
|
);
|
|
}
|
|
expect(calls).toHaveLength(2);
|
|
expect(calls.map((body) => body.version)).toEqual([null, 7]);
|
|
});
|
|
});
|
|
|
|
it.each([
|
|
"type",
|
|
"column",
|
|
"count",
|
|
"length",
|
|
"eos",
|
|
"body",
|
|
"transport",
|
|
"version",
|
|
"disconnect",
|
|
401,
|
|
403,
|
|
404,
|
|
429,
|
|
500,
|
|
])("rejects the whole call after a later %s failure", async (fault) => {
|
|
await withRangeServer(async (db, calls, paths) => {
|
|
const table = await open(db);
|
|
await expect(
|
|
table.fetchBlobRanges(
|
|
"image",
|
|
Array.from({ length: 1025 }, () => ({
|
|
rowId: 10n,
|
|
offset: 0n,
|
|
length: 1n,
|
|
})),
|
|
),
|
|
).rejects.toThrow();
|
|
expect(calls).toHaveLength(2);
|
|
expect(
|
|
paths.every(
|
|
(path) =>
|
|
path.endsWith("describe/") ||
|
|
path.endsWith("branches/list/") ||
|
|
path.endsWith("fetch_blob_ranges/"),
|
|
),
|
|
).toBe(true);
|
|
}, fault);
|
|
});
|
|
|
|
it.each(["missing-identity", "wrong-identity", "recreated", "gone"])(
|
|
"retains original identity through row and byte partitions (%s)",
|
|
async (fault) => {
|
|
for (const huge of [false, true]) {
|
|
await withRangeServer(async (db, calls) => {
|
|
const table = await open(db);
|
|
const requests = huge
|
|
? [{ rowId: 10n, offset: 0n, length: BigInt(64 * MIB + 1) }]
|
|
: Array.from({ length: 1025 }, () => ({
|
|
rowId: 10n,
|
|
offset: 0n,
|
|
length: 1n,
|
|
}));
|
|
await expect(
|
|
table.fetchBlobRanges("image", requests),
|
|
).rejects.toThrow();
|
|
expect(calls.map((call) => call.identity)).toEqual([
|
|
undefined,
|
|
IDENTITY,
|
|
]);
|
|
expect(calls.map((call) => call.version)).toEqual([null, 7]);
|
|
}, fault);
|
|
}
|
|
},
|
|
);
|
|
|
|
it.each(["first-missing-identity", "first-invalid-identity"])(
|
|
"requires a first authoritative bulk identity (%s)",
|
|
async (fault) => {
|
|
await withRangeServer(async (db, calls, paths) => {
|
|
const table = await open(db);
|
|
const before = paths.length;
|
|
await expect(
|
|
table.fetchBlobRanges(
|
|
"image",
|
|
Array.from({ length: 1025 }, () => ({
|
|
rowId: 10n,
|
|
offset: 0n,
|
|
length: 1n,
|
|
})),
|
|
),
|
|
).rejects.toThrow(/snapshot identity/);
|
|
expect(calls).toHaveLength(1);
|
|
expect(paths).toHaveLength(before + 1);
|
|
}, fault);
|
|
},
|
|
);
|
|
|
|
it("validates integers, overflow and cancellation and leaves empty calls at zero I/O", async () => {
|
|
await withRangeServer(async (db, calls, paths) => {
|
|
const table = await open(db);
|
|
const before = paths.length;
|
|
expect(await table.fetchBlobRanges("image", [])).toEqual([]);
|
|
for (const request of [
|
|
{ rowId: -1, offset: 0n, length: 1n },
|
|
{ rowId: 1.5, offset: 0n, length: 1n },
|
|
{ rowId: Number.MAX_SAFE_INTEGER + 1, offset: 0n, length: 1n },
|
|
{ rowId: -1n, offset: 0n, length: 1n },
|
|
{ rowId: 1n << 64n, offset: 0n, length: 1n },
|
|
{ rowId: 10n, offset: -1n, length: 1n },
|
|
{ rowId: 10n, offset: 0n, length: -1n },
|
|
{ rowId: 10n, offset: 0n, length: 1n << 64n },
|
|
{ rowId: 20n, offset: (1n << 64n) - 1n, length: 1n },
|
|
]) {
|
|
await expect(
|
|
table.fetchBlobRanges("image", [request]),
|
|
).rejects.toThrow();
|
|
}
|
|
// Runtime type validation must also reject a number passed as an offset.
|
|
await expect(
|
|
table.fetchBlobRanges("image", [
|
|
{ rowId: 10n, offset: 0 as unknown as bigint, length: 1n },
|
|
]),
|
|
).rejects.toThrow();
|
|
const controller = new AbortController();
|
|
controller.abort(new Error("cancelled by caller"));
|
|
await expect(
|
|
table.fetchBlobRanges(
|
|
"image",
|
|
[{ rowId: 10n, offset: 0n, length: 1n }],
|
|
{ signal: controller.signal },
|
|
),
|
|
).rejects.toThrow("cancelled by caller");
|
|
expect(paths).toHaveLength(before);
|
|
expect(calls).toHaveLength(0);
|
|
});
|
|
});
|
|
|
|
it("records the loaded native artifact", async () => {
|
|
const artifacts = Object.keys(require.cache).filter(
|
|
(path) => path.endsWith(".node") && path.includes("lancedb"),
|
|
);
|
|
expect(artifacts.length).toBeGreaterThan(0);
|
|
for (const path of artifacts) {
|
|
const hash = createHash("sha256");
|
|
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
|
console.log(
|
|
`loaded native artifact=${path} sha256=${hash.digest("hex")}`,
|
|
);
|
|
}
|
|
});
|
|
});
|