fix: inherit Lance defaults for IVF RQ quantization (#4585)

`IvfRq()` forced `num_bits=1` through Python's Rust binding, overriding
Lance's five-bit default. The shared native index creation path also
substituted one bit when the Rust or Node builder left the value unset.

Make Python's bit count optional and forward only explicit values. Use
`RQBuildParams::default()` for an unset native bit count, and omit unset
values when dispatching to pylance for accelerator builds. Update the
Python, Rust, Node, and vector-index documentation.

New indexes with an unset bit count now inherit Lance's default,
currently five bits, across Python, Rust, and Node. Five-bit vector
codes use roughly five times the storage of one-bit codes, plus other
index overhead. This also changes reader compatibility: indexes created
with `num_bits >= 2` use a newer on-disk layout that some older LanceDB
versions cannot read. Upgrade readers to versions that support multi-bit
IVF_RQ, or explicitly set `num_bits=1` (for example, `IvfRq(num_bits=1)`
in Python) when sharing indexes with readers limited to one-bit IVF_RQ.

Validation:

- Reproduced six regression failures before the fix: one-bit default
metadata, rejected `None`, and incorrect remote/accelerator forwarding.
- After merging `main` through `b762ec22`, rebuilt both native bindings
against Lance 15.0.0-beta.1. Python validation passed with 106 tests and
15 optional Ollama skips. Coverage includes the full index suite,
omitted and explicit bit counts, reading Lance-created IVF_RQ indexes,
accelerator forwarding, CUDA-unavailability handling, timeout
validation, remote requests, and table/namespace listing behavior.
- Rust formatting and Python package Ruff lint/format checks pass. Node
`pnpm lint`, `pnpm build` (including TypeScript checks), `pnpm run
docs`, the full connection suite, and the existing IVF_RQ regression
pass. Generated documentation adds no further changes.

The merge preserves upstream's explicit RaBitQ heading anchor alongside
the `IvfRq()` example and five-bit default/reader-compatibility
guidance.

Repository-wide `ruff check .` still reports 49 existing errors in
unchanged documentation examples under `docs/web-tests`; root formatting
also finds 11 unchanged documentation example files. The Python package
checks pass.

Fixes #4582

<!-- lance-gatekeeper-fix:v1 agent=6ef9406f0142b1ab66d53d9d56098223
generation=1 -->

---------

Co-authored-by: Gatefixer <313497061+lancedb-gatefixer[bot]@users.noreply.github.com>
Co-authored-by: Xuanwo <github@xuanwo.io>
This commit is contained in:
lancedb-gatefixer[bot]
2026-10-26 18:57:25 +08:00
committed by GitHub
co-authored by Xuanwo
parent 22fc2a2087
commit 66357275bb
11 changed files with 82 additions and 19 deletions
+2 -1
View File
@@ -58,7 +58,8 @@ Number of bits per dimension for residual quantization.
This value controls how much each residual component is compressed. The more
bits, the more accurate the index will be but the slower search. Typical values
are small integers; the default is 1 bit per dimension.
are small integers. If unset, uses Lance's default (currently 5 bits per
dimension). Set to 1 for maximum compression.
***
+11 -6
View File
@@ -58,7 +58,7 @@ Lance with `index_type`.
| If your priority is... | Use this index | Why | Approx. compression ratio | Python config class |
| :--- | :--- | :--- | :--- | :--- |
| Maximum compression | `IVF_RQ` | IVF indexing with RaBitQ quantization | Around `1/32` of raw size | `IvfRq` |
| Maximum compression | `IVF_RQ` | IVF indexing with RaBitQ quantization | Around `1/32` of raw size with `num_bits=1` | `IvfRq` |
| Higher accuracy at small dimensions (`dimension <= 256`) | `IVF_PQ` | IVF indexing with product quantization | Usually `1/64` to `1/16` of raw size (depends on `num_sub_vectors`) | `IvfPq` |
| Best recall/latency trade-off | `IVF_HNSW_SQ` | IVF-HNSW indexing with scalar | Typically a little larger than `1/4` of raw size | `IvfHnswSq` |
| Highest recall / no quantization | `IVF_HNSW_FLAT` | IVF-HNSW indexing with no quantization; embedded library only | Around raw vector size plus HNSW graph overhead | `IvfHnswFlat` |
@@ -377,7 +377,7 @@ which has been compressed to $4$ chunks $\times 8$-bit integers $= 32$ bits of q
RaBitQ is an advanced quantization technique that outperforms PQ in several ways.
It needs no codebook to train, estimates distances very quickly at query-time,
and crucially, quantizes each vector in (with some small overhead) just **one bit per dimension!**
and can quantize each vector in (with some small overhead) as little as **one bit per dimension!**
In practice, RaBitQ compresses a $1024$-dimensional `float32` vector into just a few thousand bits, while maintaining good recall.
The inner workings of RaBitQ quantization are rather mathematically dense. It generates a quantization codebook by
@@ -385,14 +385,19 @@ applying a uniformly random, approximately distance-preserving orthogonal transf
where $d$ is the dimensionality of the dataset. We defer the details, and an elegant theoretical error bound, to [the original paper.](https://arxiv.org/pdf/2405.12497)
##### Using RaBitQ {#using-rabitq}
Use RaBitQ quantization by selecting quantized index types ending in the suffix `RQ`. For example, call `create_index` with `index_type="IVF_RQ"`.
Use RaBitQ quantization by selecting quantized index types ending in the suffix `RQ`. For example, import `IvfRq` from `lancedb.index` and call `table.create_index("vector", config=IvfRq())`.
Note that when using `IVF_RQ`, the dimension of the dataset must be a multiple of `8`.
`num_bits` determines how many bits are used to quantize each dimension.
`1` is the standard RaBitQ setting. Increase to `2`, `4`, or `8` bits to achieve better recall for additional storage and query-time compute.
By default, Python, Rust, and Node leave this unset and inherit Lance's default, currently `5` bits.
Higher values improve recall at the cost of additional storage and query-time compute.
To retain the previous one-bit default and maximum compression, explicitly set `num_bits=1` (for example, `IvfRq(num_bits=1)` in Python).
Five-bit quantized vector codes use roughly five times the storage of one-bit codes; total index size also includes other overhead.
<Warning title="Reading multi-bit indexes across versions">
RaBitQ-quantized indexes computed with `num_bits >= 2` use a newer on-disk layout, and cannot be read by some older LanceDB versions.
RaBitQ-quantized indexes computed with `num_bits >= 2` use a newer on-disk layout and cannot be read by some older LanceDB versions.
The new five-bit default therefore affects reader compatibility as well as index size.
Upgrade readers to versions that support multi-bit IVF_RQ before creating indexes with the default settings, or explicitly set `num_bits=1` when sharing indexes with readers that support only one-bit IVF_RQ.
</Warning>
See this [blog post](https://lancedb.com/blog/feature-rabitq-quantization/) for further discussion and benchmarking of LanceDB's RaBitQ implementation.
@@ -410,7 +415,7 @@ among all integral multiples $\{j \times 10^5: -128 \leq j \leq 127 \}$.
| Parameter | Description |
| :--- | :--- |
| `num_bits` | Bits per dimension for quantization. Only applies to `IVF_PQ`/`IVF_HNSW_PQ` (default `8`) and `IVF_RQ` (default `1`, RaBitQ); not used by `IVF_FLAT`/`IVF_SQ`/`IVF_HNSW_FLAT`/`IVF_HNSW_SQ`. Higher values improve accuracy at the cost of additional storage. |
| `num_bits` | Bits per dimension for quantization. Only applies to `IVF_PQ`/`IVF_HNSW_PQ` (default `8`) and `IVF_RQ` (`IvfRq()` inherits Lance's default, currently `5`, RaBitQ); not used by `IVF_FLAT`/`IVF_SQ`/`IVF_HNSW_FLAT`/`IVF_HNSW_SQ`. Higher values improve accuracy at the cost of additional storage. |
`max_iterations` and `sample_rate` also affect quantizer training, but since they apply to every IVF/HNSW index type (not just quantized ones), they're documented as general [Build-time Parameters](/indexing/vector-index#build-time-parameters) instead. All three are passed as keyword arguments to `create_index`, alongside `index_type`: