feat(node): expose targetPartitionSize on IVF index options (#4594)

The Node SDK's vector index options only take `numPartitions`. The Rust index builders (`target_partition_size` in `rust/lancedb/src/index/vector.rs`) and the Python SDK already support a target partition size, so this adds `targetPartitionSize` to `IvfPqOptions`, `IvfRqOptions`, `IvfFlatOptions`, `HnswPqOptions` and `HnswSqOptions` and passes it to their builders. When both are set, `numPartitions` is used, as it is for Rust and Python callers.

The reason to want it: an index built from a partition count records no target partition size, so `table.optimize()` splits and joins it against the index type's default and can merge small partitions down to one or two. Building with a target records it in the index details, and optimize keeps to it. https://github.com/lance-format/lance/pull/10154 fixes this on the Lance side for indexes built from a partition count; this lets Node users set the size directly.

The new tests build IvfPq, IvfFlat, IvfRq, IvfHnswPq and IvfHnswSq indexes with `targetPartitionSize` and check the size recorded in the index details, and check that `numPartitions` wins when both are set. Each option's doc comment states that index type's default target. Docs regenerated with `pnpm run docs`.

docs/src/js/classes/Table.md

@@ -385,6 +385,13 @@ await table.createIndex("vector", {
 });
 ```
 
+```ts
+// Or size the partitions by row count instead of choosing how many.
+await table.createIndex("vector", {
+  config: lancedb.Index.ivfPq({ targetPartitionSize: 4096 }),
+});
+```
+
 ```ts
 // Or create a Scalar index
 await table.createIndex("my_float_col");

docs/src/js/interfaces/HnswPqOptions.md

@@ -100,6 +100,8 @@ optional numPartitions: number;
 
 The number of IVF partitions to create.
 
+When set, this takes precedence over `targetPartitionSize`.
+
 For HNSW, we recommend a small number of partitions. Setting this to 1 works
 well for most tables. For very large tables, training just one HNSW graph
 will require too much memory. Each partition becomes its own HNSW graph, so
@@ -147,3 +149,23 @@ number of vectors used to train the index is `sample_rate * num_partitions`.
 
 Increasing this value might improve the quality of the index but in most cases the
 default should be sufficient.
+
+***
+
+### targetPartitionSize?
+
+```ts
+optional targetPartitionSize: number;
+```
+
+The target number of rows in each IVF partition.
+
+This is an alternative to `numPartitions`. When it is set and
+`numPartitions` is not, the number of partitions is derived by dividing
+the number of rows by this value. When both are set, `numPartitions` is
+used and this value is ignored. When neither is set, the target is
+1,048,576 rows.
+
+Larger partitions mean fewer of them, which makes the first part of the
+search (picking the right partitions) faster and the second part
+(searching within a partition) slower.

docs/src/js/interfaces/HnswSqOptions.md

@@ -100,6 +100,8 @@ optional numPartitions: number;
 
 The number of IVF partitions to create.
 
+When set, this takes precedence over `targetPartitionSize`.
+
 For HNSW, we recommend a small number of partitions. Setting this to 1 works
 well for most tables. For very large tables, training just one HNSW graph
 will require too much memory. Each partition becomes its own HNSW graph, so
@@ -126,3 +128,23 @@ number of vectors used to train the index is `sample_rate * num_partitions`.
 
 Increasing this value might improve the quality of the index but in most cases the
 default should be sufficient.
+
+***
+
+### targetPartitionSize?
+
+```ts
+optional targetPartitionSize: number;
+```
+
+The target number of rows in each IVF partition.
+
+This is an alternative to `numPartitions`. When it is set and
+`numPartitions` is not, the number of partitions is derived by dividing
+the number of rows by this value. When both are set, `numPartitions` is
+used and this value is ignored. When neither is set, the target is
+1,048,576 rows.
+
+Larger partitions mean fewer of them, which makes the first part of the
+search (picking the right partitions) faster and the second part
+(searching within a partition) slower.

docs/src/js/interfaces/IvfFlatOptions.md

@@ -82,8 +82,9 @@ optional numPartitions: number;
 The number of IVF partitions to create.
 
 This value should generally scale with the number of rows in the dataset.
-By default the number of partitions is the square root of the number of
-rows.
+When set, this takes precedence over `targetPartitionSize`. When omitted,
+the partition count is derived from `targetPartitionSize`, which defaults
+to 4096 rows for this index type.
 
 If this value is too large then the first part of the search (picking the
 right partition) will be slow.  If this value is too small then the second
@@ -110,3 +111,23 @@ Increasing this value might improve the quality of the index but in most cases t
 default should be sufficient.
 
 The default value is 256.
+
+***
+
+### targetPartitionSize?
+
+```ts
+optional targetPartitionSize: number;
+```
+
+The target number of rows in each IVF partition.
+
+This is an alternative to `numPartitions`. When it is set and
+`numPartitions` is not, the number of partitions is derived by dividing
+the number of rows by this value. When both are set, `numPartitions` is
+used and this value is ignored. When neither is set, the target is
+4096 rows.
+
+Larger partitions mean fewer of them, which makes the first part of the
+search (picking the right partitions) faster and the second part
+(searching within a partition) slower.

docs/src/js/interfaces/IvfPqOptions.md

@@ -92,8 +92,9 @@ optional numPartitions: number;
 The number of IVF partitions to create.
 
 This value should generally scale with the number of rows in the dataset.
-By default the number of partitions is the square root of the number of
-rows.
+When set, this takes precedence over `targetPartitionSize`. When omitted,
+the partition count is derived from `targetPartitionSize`, which defaults
+to 8192 rows for this index type.
 
 If this value is too large then the first part of the search (picking the
 right partition) will be slow.  If this value is too small then the second
@@ -141,3 +142,23 @@ Increasing this value might improve the quality of the index but in most cases t
 default should be sufficient.
 
 The default value is 256.
+
+***
+
+### targetPartitionSize?
+
+```ts
+optional targetPartitionSize: number;
+```
+
+The target number of rows in each IVF partition.
+
+This is an alternative to `numPartitions`. When it is set and
+`numPartitions` is not, the number of partitions is derived by dividing
+the number of rows by this value. When both are set, `numPartitions` is
+used and this value is ignored. When neither is set, the target is
+8192 rows.
+
+Larger partitions mean fewer of them, which makes the first part of the
+search (picking the right partitions) faster and the second part
+(searching within a partition) slower.

docs/src/js/interfaces/IvfRqOptions.md

@@ -72,8 +72,9 @@ optional numPartitions: number;
 The number of IVF partitions to create.
 
 This value should generally scale with the number of rows in the dataset.
-By default the number of partitions is the square root of the number of
-rows.
+When set, this takes precedence over `targetPartitionSize`. When omitted,
+the partition count is derived from `targetPartitionSize`, which defaults
+to 4096 rows for this index type.
 
 If this value is too large then the first part of the search (picking the
 right partition) will be slow. If this value is too small then the second
@@ -100,3 +101,23 @@ Increasing this value might improve the quality of the index but in most cases t
 default should be sufficient.
 
 The default value is 256.
+
+***
+
+### targetPartitionSize?
+
+```ts
+optional targetPartitionSize: number;
+```
+
+The target number of rows in each IVF partition.
+
+This is an alternative to `numPartitions`. When it is set and
+`numPartitions` is not, the number of partitions is derived by dividing
+the number of rows by this value. When both are set, `numPartitions` is
+used and this value is ignored. When neither is set, the target is
+4096 rows.
+
+Larger partitions mean fewer of them, which makes the first part of the
+search (picking the right partitions) faster and the second part
+(searching within a partition) slower.