fix(node): normalize record batches before IPC serialization (#4871)

Passing a record batch created by an ESM `apache-arrow` import to
`RRFReranker.rerankHybrid` throws an IPC footer parse error. The SDK's
CommonJS Arrow writer receives data types from another module instance.

Normalize single-record-batch input with the existing table sanitizer
before writing IPC. Extend the Arrow 15–21 round-trip test with
caller-created batches and nulls, and add a native ESM reranker
regression covering 64-bit row IDs, scores, and an empty FTS side.

Validation:

- Reproduced with published `0.42.0-beta.3`; CommonJS and explicitly
normalized ESM controls pass.
- Same native module and inputs: all 8 targeted cases fail with the
parent serializer and pass with this change.
- Local native build, TypeScript build, lint, and formatting pass.
- Current-head CI passes on Linux Node 22/24/26, macOS, and Arrow 19/20.
The macOS run passes 1,643 tests; Linux Node 24 also passes docs and
example checks.
This commit is contained in:
Yang Cen
2026-11-05 19:50:50 +08:00
committed by GitHub
parent 181ac2b567
commit 9ba1920e0e
3 changed files with 41 additions and 4 deletions
+3 -1
View File
@@ -1386,7 +1386,9 @@ export async function fromBufferToRecordBatch(
export async function fromRecordBatchToBuffer(
batch: RecordBatch,
): Promise<Buffer> {
const writer = new RecordBatchFileWriter().writeAll([batch]);
// Arrow writers require data types from their own library instance.
const table = sanitizeTable({ schema: batch.schema, batches: [batch] });
const writer = new RecordBatchFileWriter().writeAll(table);
return Buffer.from(await writer.toUint8Array());
}