feat(nodejs): expose Function refresh publication intervals (#4957)

Expose `publicationIntervalRows` on TypeScript refresh options and
forward it as `publication_interval_rows`. Accept positive safe integers
and preserve whole-fragment publication when omitted.

```typescript
const job = await table.refreshColumnAsync("embedding", {
  publicationIntervalRows: 10000,
});
await job.wait();
```

Document the threshold semantics and add coverage for serialization,
omission, invalid values, safe-integer boundaries, and Embedded
rejection.

Closes ENT-3755.
This commit is contained in:
Jack Ye
2026-11-11 07:34:54 -08:00
committed by GitHub
parent c1fee36045
commit e9774ed168
7 changed files with 103 additions and 6 deletions
+6
View File
@@ -1197,6 +1197,11 @@ LanceDB Enterprise it is the server's backfill job.
`resourceBudget` for registered Functions. All are LanceDB Enterprise
only; a LanceDB Embedded refresh rejects every option other than
`batchSize`, `gpuClass` and `gpuCount`.
`publicationIntervalRows` opts into publishing completed Function split
ranges once their accumulated unpublished rows reach the threshold.
It must be a positive safe integer and requires non-Blob Function outputs.
Omit it for whole-fragment publication. Publications can exceed the
threshold, and remaining rows publish at completion.
#### Returns
@@ -1212,6 +1217,7 @@ const job = await table.refreshColumnAsync("embedding", {
memoryBytes: 16 * 1024 ** 3,
},
maxConcurrency: 4,
publicationIntervalRows: 10000,
errorPolicy: "skip",
maxFailedFraction: 0.01,
rowTimeoutSeconds: 30,
@@ -117,6 +117,19 @@ Worker count for an expression column; Function columns reject it.
***
### publicationIntervalRows?
```ts
optional publicationIntervalRows: number;
```
Row-count threshold for publishing completed Function split ranges.
Must be a positive safe integer. Omit for whole-fragment publication.
Actual publications can be larger; remaining rows publish at completion.
Requires LanceDB Enterprise with non-Blob Function outputs.
***
### resourceBudget?
```ts