feat(python): pass merge_insert sources as scannables

Python's `execute_merge_insert` now takes a `PyScannable` (like `add()`),
and the source collection threshold is exposed as a
`source_collect_threshold_bytes` kwarg on `execute()` and a
`merge_insert_source_collect_threshold_bytes` connection option.

Node passes the decoded IPC batches as a `Vec<RecordBatch>`, so the
merge join sees their exact row count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Will Jones
2026-10-10 17:34:51 +00:00
co-authored by Claude Opus 5.5
parent 300d494d6e
commit 2c2bb9a974
10 changed files with 157 additions and 11 deletions
+10
View File
@@ -1081,6 +1081,16 @@ describe("merge insert", () => {
expect(res).toEqual(expected);
});
test("empty source with delete by source", async () => {
const empty = arrow.makeEmptyTable(await table.schema());
const res = await table
.mergeInsert("a")
.whenNotMatchedBySourceDelete()
.execute(empty);
expect(res.numDeletedRows).toBe(3);
expect(await table.countRows()).toBe(0);
});
test("timeout", async () => {
const newData = [
{ a: 2, b: "x" },