mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-18 08:34:00 +00:00
fix(node): normalize record batches before IPC serialization (#4871)
Passing a record batch created by an ESM `apache-arrow` import to `RRFReranker.rerankHybrid` throws an IPC footer parse error. The SDK's CommonJS Arrow writer receives data types from another module instance. Normalize single-record-batch input with the existing table sanitizer before writing IPC. Extend the Arrow 15–21 round-trip test with caller-created batches and nulls, and add a native ESM reranker regression covering 64-bit row IDs, scores, and an empty FTS side. Validation: - Reproduced with published `0.42.0-beta.3`; CommonJS and explicitly normalized ESM controls pass. - Same native module and inputs: all 8 targeted cases fail with the parent serializer and pass with this change. - Local native build, TypeScript build, lint, and formatting pass. - Current-head CI passes on Linux Node 22/24/26, macOS, and Arrow 19/20. The macOS run passes 1,643 tests; Linux Node 24 also passes docs and example checks.
This commit is contained in:
@@ -1386,7 +1386,9 @@ export async function fromBufferToRecordBatch(
|
||||
export async function fromRecordBatchToBuffer(
|
||||
batch: RecordBatch,
|
||||
): Promise<Buffer> {
|
||||
const writer = new RecordBatchFileWriter().writeAll([batch]);
|
||||
// Arrow writers require data types from their own library instance.
|
||||
const table = sanitizeTable({ schema: batch.schema, batches: [batch] });
|
||||
const writer = new RecordBatchFileWriter().writeAll(table);
|
||||
return Buffer.from(await writer.toUint8Array());
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user