feat: expose approx_mode on vector queries

`ApproxMode` landed in the Rust core in #3549, but the Python and Node.js bindings were never plumbed, so `approx_mode` is unreachable from either SDK. Both were built mirroring the existing `ef` plumbing.

Co-authored-by: Leo <leo@insight.co>
This commit is contained in:
Leo
2026-11-03 12:45:52 +08:00
committed by GitHub
co-authored by Leo
parent f9389f113e
commit f2bcb156ad
13 changed files with 326 additions and 12 deletions
+32
View File
@@ -101,6 +101,38 @@ AnalyzeExec verbose=true, metrics=[]
***
### approxMode()
```ts
approxMode(approxMode): VectorQuery
```
Set the speed / accuracy tradeoff to use for approximate search
This currently affects RQ-quantized indexes, such as IVF_RQ, and
prefiltered search on HNSW indexes, where "fast" enables the ACORN
traversal. Other index types ignore this setting.
"fast" favors lower latency and may reduce recall, "normal" (the default)
uses the index's default balance, and "accurate" favors higher recall and
may increase latency.
#### Parameters
* **approxMode**: `"fast"` \| `"normal"` \| `"accurate"`
#### Returns
[`VectorQuery`](VectorQuery.md)
#### Example
```ts
table.vectorSearch([0.1, 0.2]).approxMode("fast").limit(10);
```
***
### bypassVectorIndex()
```ts
+4 -4
View File
@@ -206,7 +206,7 @@ When a vector index is used, `_distance` is not always the true distance between
### Exact vs Approximate Distances
When doing vector search, the meaning of "distance" depends on whether you are using an index and whether `refine_factor` is specified as part of your query.
`nprobes` controls how many partitions are searched to find candidates, `approx_mode` controls the query-time speed/recall trade-off for RQ-quantized indexes, and `refine_factor` controls how many candidates are rescored on full vectors for better distance fidelity and reranking quality.
`nprobes` controls how many partitions are searched to find candidates, `approx_mode` controls the query-time speed/recall trade-off for RQ-quantized and filtered HNSW indexes, and `refine_factor` controls how many candidates are rescored on full vectors for better distance fidelity and reranking quality.
The table below summarizes the behavior of `_distance` in search results based on your query configuration:
@@ -236,17 +236,17 @@ following sections cover query-time settings.
### Tuning `approx_mode` {#tuning-approx-mode}
Use `approx_mode` when you want to adjust the speed/recall trade-off for approximate vector search at query time. This setting currently applies only to RQ-quantized indexes, such as `IVF_RQ`; other index types ignore it.
Use `approx_mode` when you want to adjust the speed/recall trade-off for approximate vector search at query time. It applies to RQ-quantized indexes, such as `IVF_RQ`, and to filtered searches on HNSW indexes. Other index types, and queries that bypass the index, ignore it.
The supported values are:
| Value | Behavior |
| :--- | :--- |
| `fast` | Prefer lower query latency, which can reduce recall. |
| `normal` | Use the default balance between query latency and recall. This only has an effect for RQ indexes built with `num_bits > 1`. |
| `normal` | Use the default balance between query latency and recall. For RQ indexes built with `num_bits=1`, this uses the same one-bit scoring path as `fast`. |
| `accurate` | Prefer higher recall, which can increase query latency. |
Only the Rust SDK sets `approx_mode`; the Python and TypeScript SDKs do not expose it. You can change `approx_mode` per query without rebuilding the index. For RQ indexes built with `num_bits=1`, `normal` uses the same one-bit scoring path as `fast`. If you also set `refine_factor`, LanceDB first uses `approx_mode` while finding candidates, then reranks the selected candidates on full vectors.
`approx_mode` defaults to `normal` and can change per query without rebuilding the index. The Python and Rust APIs call it `approx_mode`, and the TypeScript API calls it `approxMode`. On HNSW, `fast` enables ACORN traversal only when a filter leaves a non-trivial subset of rows; unfiltered or all-row searches use the regular traversal, and very selective filters may use an exact scan instead. If you also set `refine_factor`, LanceDB first uses `approx_mode` while finding candidates, then reranks the selected candidates on full vectors.
### Tuning `nprobes` {#tuning-nprobes}
+17
View File
@@ -1626,6 +1626,23 @@ describe("When creating an index", () => {
// test ef
rst = await tbl.query().limit(2).nearestTo(queryVec).ef(100).toArrow();
expect(rst.numRows).toBe(2);
// test approxMode
for (const mode of ["fast", "normal", "accurate"] as const) {
rst = await tbl
.query()
.limit(2)
.nearestTo(queryVec)
.approxMode(mode)
.toArrow();
expect(rst.numRows).toBe(2);
}
expect(() =>
tbl
.query()
.nearestTo(queryVec)
.approxMode("turbo" as "fast"),
).toThrow("approx_mode must be one of");
});
it("should be able to query unindexed data", async () => {
+21
View File
@@ -926,6 +926,27 @@ export class VectorQuery extends StandardQueryBase<NativeVectorQuery> {
return this;
}
/**
* Set the speed / accuracy tradeoff to use for approximate search
*
* This currently affects RQ-quantized indexes, such as IVF_RQ, and
* prefiltered search on HNSW indexes, where "fast" enables the ACORN
* traversal. Other index types ignore this setting.
*
* "fast" favors lower latency and may reduce recall, "normal" (the default)
* uses the index's default balance, and "accurate" favors higher recall and
* may increase latency.
*
* @example
* ```ts
* table.vectorSearch([0.1, 0.2]).approxMode("fast").limit(10);
* ```
*/
approxMode(approxMode: "fast" | "normal" | "accurate"): VectorQuery {
super.doCall((inner) => inner.approxMode(approxMode));
return this;
}
/**
* Set the vector column to query
*
+8
View File
@@ -15,6 +15,7 @@ use arrow_array::{
};
use arrow_buffer::ScalarBuffer;
use half::f16;
use lancedb::ApproxMode;
use lancedb::index::scalar::{
BooleanQuery, BoostQuery, FtsQuery, FullTextSearchQuery, MatchQuery, MultiMatchQuery, Occur,
Operator, PhraseQuery,
@@ -352,6 +353,13 @@ impl VectorQuery {
self.inner = self.inner.clone().ef(ef as usize);
}
#[napi]
pub fn approx_mode(&mut self, approx_mode: String) -> napi::Result<()> {
let approx_mode = ApproxMode::try_from(approx_mode.as_str()).default_error()?;
self.inner = self.inner.clone().approx_mode(approx_mode);
Ok(())
}
#[napi]
pub fn bypass_vector_index(&mut self) {
self.inner = self.inner.clone().bypass_vector_index()
+3
View File
@@ -825,6 +825,7 @@ class VectorQuery:
def search_effort(self, search_effort: Optional[float]): ...
def minimum_nprobes(self, minimum_nprobes: int): ...
def maximum_nprobes(self, maximum_nprobes: int): ...
def approx_mode(self, approx_mode: str): ...
def bypass_vector_index(self): ...
def nearest_to_text(self, query: dict) -> HybridQuery: ...
def order_by(self, ordering: Optional[List[ColumnOrdering]]): ...
@@ -851,6 +852,7 @@ class HybridQuery:
def search_effort(self, search_effort: Optional[float]): ...
def minimum_nprobes(self, minimum_nprobes: int): ...
def maximum_nprobes(self, maximum_nprobes: int): ...
def approx_mode(self, approx_mode: str): ...
def bypass_vector_index(self): ...
def order_by(self, ordering: Optional[List[ColumnOrdering]]): ...
def to_vector_query(self) -> VectorQuery: ...
@@ -882,6 +884,7 @@ class PyQueryRequest:
lower_bound: Optional[float]
upper_bound: Optional[float]
ef: Optional[int]
approx_mode: Optional[str]
refine_factor: Optional[int]
distance_type: Optional[str]
bypass_vector_index: Optional[bool]
+107
View File
@@ -912,6 +912,12 @@ class Query(pydantic.BaseModel):
ef : Optional[int]
The size of the nearest neighbor list maintained during HNSW search
This will only be set on vector search.
approx_mode : Optional[str]
The speed / accuracy tradeoff to use for approximate vector search
One of "fast", "normal" or "accurate".
This will only be set on vector search.
full_text_query : Optional[Union[str, dict]]
The full text search query
@@ -1014,6 +1020,9 @@ class Query(pydantic.BaseModel):
# size of the nearest neighbor list maintained during HNSW search
ef: Optional[int] = None
# speed / accuracy tradeoff for approximate vector search
approx_mode: Optional[str] = None
# Bypass the vector index and use a brute force search
bypass_vector_index: Optional[bool] = None
@@ -1044,6 +1053,7 @@ class Query(pydantic.BaseModel):
query.lower_bound = req.lower_bound
query.upper_bound = req.upper_bound
query.ef = req.ef
query.approx_mode = req.approx_mode
query.refine_factor = req.refine_factor
query.bypass_vector_index = req.bypass_vector_index
query.postfilter = req.postfilter
@@ -1212,6 +1222,7 @@ class LanceQueryBuilder(ABC):
self._vector = None
self._text = None
self._ef = None
self._approx_mode = None
self._bypass_vector_index = None
self._order_by = None
@@ -2136,6 +2147,35 @@ class LanceVectorQueryBuilder(LanceQueryBuilder):
self._ef = ef
return self
def approx_mode(
self, approx_mode: Literal["fast", "normal", "accurate"]
) -> LanceVectorQueryBuilder:
"""Set the speed / accuracy tradeoff to use for approximate search.
This currently affects RQ-quantized indexes, such as IVF_RQ, and
prefiltered search on HNSW indexes, where "fast" enables the ACORN
traversal. Other index types ignore this setting.
Examples
--------
>>> import lancedb
>>> db = lancedb.connect("memory://")
>>> table = db.create_table("approx_vectors", [{"vector": [0.1, 0.2]}])
>>> query = table.search([0.1, 0.2]).approx_mode("fast")
Parameters
----------
approx_mode: "fast" or "normal" or "accurate"
The tradeoff to use. By default "normal" is used.
Returns
-------
LanceVectorQueryBuilder
The LanceQueryBuilder object.
"""
self._approx_mode = approx_mode
return self
def refine_factor(self, refine_factor: int) -> LanceVectorQueryBuilder:
"""Set the refine factor to use, increasing the number of vectors sampled.
@@ -2230,6 +2270,7 @@ class LanceVectorQueryBuilder(LanceQueryBuilder):
offset=self._offset,
fast_search=self._fast_search,
ef=self._ef,
approx_mode=self._approx_mode,
bypass_vector_index=self._bypass_vector_index,
order_by=self._order_by,
)
@@ -3060,6 +3101,43 @@ class LanceHybridQueryBuilder(LanceQueryBuilder):
self._ef = ef
return self
def approx_mode(
self, approx_mode: Literal["fast", "normal", "accurate"]
) -> LanceHybridQueryBuilder:
"""
Set the speed / accuracy tradeoff to use for approximate search.
This currently affects RQ-quantized indexes, such as IVF_RQ, and
prefiltered search on HNSW indexes, where "fast" enables the ACORN
traversal. Other index types ignore this setting.
Examples
--------
>>> import lancedb
>>> db = lancedb.connect("memory://")
>>> table = db.create_table(
... "approx_hybrid", [{"vector": [0.1, 0.2], "text": "dog"}]
... )
>>> query = (
... table.search(query_type="hybrid")
... .vector([0.1, 0.2])
... .text("dog")
... .approx_mode("accurate")
... )
Parameters
----------
approx_mode: "fast" or "normal" or "accurate"
The tradeoff to use. By default "normal" is used.
Returns
-------
LanceHybridQueryBuilder
The LanceHybridQueryBuilder object.
"""
self._approx_mode = approx_mode
return self
def metric(self, metric: Literal["l2", "cosine", "dot"]) -> LanceHybridQueryBuilder:
"""Set the distance metric to use.
@@ -3304,6 +3382,8 @@ class LanceHybridQueryBuilder(LanceQueryBuilder):
self._vector_query.refine_factor(self._refine_factor)
if self._ef:
self._vector_query.ef(self._ef)
if self._approx_mode:
self._vector_query.approx_mode(self._approx_mode)
if self._bypass_vector_index:
self._vector_query.bypass_vector_index()
if self._lower_bound is not None or self._upper_bound is not None:
@@ -4430,6 +4510,33 @@ class AsyncVectorQueryBase:
self._inner.ef(ef)
return self
def approx_mode(self, approx_mode: Literal["fast", "normal", "accurate"]) -> Self:
"""
Set the speed / accuracy tradeoff to use for approximate search
This currently affects RQ-quantized indexes, such as IVF_RQ, and prefiltered
search on HNSW indexes, where "fast" enables the ACORN traversal. Other index
types ignore this setting.
"fast" favors lower latency and may reduce recall, "normal" (the default) uses
the index's default balance, and "accurate" favors higher recall and may
increase latency.
Examples
--------
>>> import asyncio
>>> import lancedb
>>> async def build_query():
... db = await lancedb.connect_async("memory://")
... table = await db.create_table(
... "approx_async", [{"vector": [0.1, 0.2]}]
... )
... return table.vector_search([0.1, 0.2]).approx_mode("accurate")
>>> query = asyncio.run(build_query())
"""
self._inner.approx_mode(approx_mode)
return self
def refine_factor(self, refine_factor: int) -> Self:
"""
A multiplier to control how many additional rows are taken during the refine
+13 -4
View File
@@ -128,6 +128,15 @@ def _should_push_down_query_table(
return namespace_client is not None and "QueryTable" in pushdown_operations
def _requires_local_namespace_execution(query: Query) -> bool:
# The namespace QueryTable request does not support these query options.
return (
query.take_offsets is not None
or query.search_effort is not None
or query.approx_mode is not None
)
_CUDA_UNAVAILABLE_MESSAGE = (
"CUDA acceleration is unavailable in this Python environment. Install a "
"CUDA-enabled PyTorch build and verify that the CUDA device is accessible, or "
@@ -5023,8 +5032,7 @@ class LanceTable(Table):
)
and not self._route_pushdown_to_rust
and self.current_branch() is None
and query.take_offsets is None
and query.search_effort is None
and not _requires_local_namespace_execution(query)
):
from lancedb.namespace import _execute_server_side_query
@@ -7199,6 +7207,8 @@ class AsyncTable:
async_query = async_query.column(query.vector_column)
if query.ef:
async_query = async_query.ef(query.ef)
if query.approx_mode:
async_query = async_query.approx_mode(query.approx_mode)
if query.bypass_vector_index:
async_query = async_query.bypass_vector_index()
@@ -7224,8 +7234,7 @@ class AsyncTable:
self._namespace_client, self._pushdown_operations
)
and not self._route_pushdown_to_rust
and query.take_offsets is None
and query.search_effort is None
and not _requires_local_namespace_execution(query)
):
from lancedb.namespace import _execute_server_side_query
+47
View File
@@ -173,6 +173,9 @@ class _FailingSyncInner:
async def to_arrow(self):
raise RuntimeError("direct table to_arrow should not be used")
async def _execute_query(self, query, **kwargs):
raise RuntimeError("local execution reached")
class _FailingAsyncInner:
def name(self):
@@ -1264,6 +1267,25 @@ class TestPushdownOperations:
]
assert all(r.k <= 2**31 - 1 for r in namespace_client.requests)
@pytest.mark.parametrize(
"query_kwargs",
[
{"approx_mode": "fast"},
{"search_effort": 0.75},
{"approx_mode": "fast", "search_effort": 0.75},
],
ids=["approx_mode", "search_effort", "both"],
)
def test_lance_table_local_only_options_are_not_pushed_down(self, query_kwargs):
namespace_client = _NamespaceClient()
table = _namespace_lance_table(namespace_client)
# The QueryTable request cannot carry these options, so the query has to
# run locally rather than silently ignore them.
with pytest.raises(RuntimeError, match="local execution reached"):
table._execute_query(Query(vector=[1.0, 2.0], **query_kwargs))
assert namespace_client.requests == []
def test_probe_fields_pass_through_independently(self):
request = _query_to_namespace_request(
["geneva", "hist"],
@@ -1348,6 +1370,31 @@ class TestAsyncPushdownOperations:
]
assert all(r.k <= 2**31 - 1 for r in namespace_client.requests)
@pytest.mark.parametrize(
"query_kwargs",
[
{"approx_mode": "fast"},
{"search_effort": 0.75},
{"approx_mode": "fast", "search_effort": 0.75},
],
ids=["approx_mode", "search_effort", "both"],
)
async def test_async_table_local_only_options_are_not_pushed_down(
self, query_kwargs
):
namespace_client = _NamespaceClient()
table = AsyncTable(
_FailingAsyncInner(),
namespace_path=["geneva"],
namespace_client=namespace_client,
pushdown_operations={"QueryTable"},
)
with pytest.raises(AssertionError, match="direct async query"):
await table._execute_query(Query(vector=[1.0, 2.0], **query_kwargs))
assert namespace_client.requests == []
def test_local_table_to_arrow_and_to_pandas_are_unchanged(tmp_path):
db = lancedb.connect(str(tmp_path / "db"))
+52
View File
@@ -1308,6 +1308,28 @@ async def test_invalid_nprobes_async(table_async: AsyncTable):
)
def test_approx_mode_works_sync(table):
for mode in ["fast", "normal", "accurate"]:
LanceVectorQueryBuilder(table, [0, 0], "vector").approx_mode(mode).to_list()
def test_invalid_approx_mode_sync(table):
with pytest.raises(ValueError, match="approx_mode must be one of"):
LanceVectorQueryBuilder(table, [0, 0], "vector").approx_mode("turbo").to_list()
@pytest.mark.asyncio
async def test_approx_mode_works_async(table_async: AsyncTable):
for mode in ["fast", "normal", "accurate"]:
await table_async.vector_search([0, 0]).approx_mode(mode).to_list()
@pytest.mark.asyncio
async def test_invalid_approx_mode_async(table_async: AsyncTable):
with pytest.raises(ValueError, match="approx_mode must be one of"):
table_async.vector_search([0, 0]).approx_mode("turbo")
def test_query_builder_with_prefilter(table):
df = (
LanceVectorQueryBuilder(table, [0, 0], "vector")
@@ -1962,6 +1984,9 @@ def test_query_serialization_sync(table: lancedb.table.Table):
q = table.search([5.0, 6.0]).ef(7).to_query_object()
check_set_props(q, ef=7, vector_column="vector", vector=[5.0, 6.0])
q = table.search([5.0, 6.0]).approx_mode("fast").to_query_object()
check_set_props(q, approx_mode="fast", vector_column="vector", vector=[5.0, 6.0])
q = table.search([5.0, 6.0]).bypass_vector_index().to_query_object()
check_set_props(
q, bypass_vector_index=True, vector_column="vector", vector=[5.0, 6.0]
@@ -2190,6 +2215,19 @@ async def test_query_serialization_async(table_async: AsyncTable):
limit=10,
)
q = (await table_async.search([5.0, 6.0])).approx_mode("fast").to_query_object()
check_set_props(
q,
approx_mode="fast",
vector=sample_vector,
postfilter=False,
minimum_nprobes=None,
maximum_nprobes=None,
with_row_id=False,
bypass_vector_index=False,
limit=10,
)
q = (await table_async.search([5.0, 6.0])).bypass_vector_index().to_query_object()
check_set_props(
q,
@@ -2228,6 +2266,20 @@ async def test_query_serialization_async(table_async: AsyncTable):
)
@pytest.mark.asyncio
async def test_hybrid_query_serialization_async(table_async: AsyncTable):
q = (
table_async.query()
.nearest_to([5.0, 6.0])
.nearest_to_text(MatchQuery("dog", "text"))
.ef(7)
.approx_mode("fast")
.to_query_object()
)
assert q.ef == 7
assert q.approx_mode == "fast"
def test_query_schema(tmp_path):
db = lancedb.connect(tmp_path)
tbl = db.create_table(
+15
View File
@@ -16,6 +16,7 @@ use arrow::array::make_array;
use arrow::pyarrow::FromPyArrow;
use arrow::pyarrow::IntoPyArrow;
use arrow::pyarrow::ToPyArrow;
use lancedb::ApproxMode;
use lancedb::index::scalar::{
BooleanQuery, BoostQuery, CombinedFieldsQuery, DocumentGranularity, FtsQuery,
FullTextSearchQuery, MatchQuery, MultiMatchQuery, Occur, Operator, PhraseQuery,
@@ -370,6 +371,7 @@ pub struct PyQueryRequest {
pub lower_bound: Option<f32>,
pub upper_bound: Option<f32>,
pub ef: Option<usize>,
pub approx_mode: Option<String>,
pub refine_factor: Option<u32>,
pub distance_type: Option<String>,
pub bypass_vector_index: Option<bool>,
@@ -403,6 +405,7 @@ impl From<AnyQuery> for PyQueryRequest {
lower_bound: None,
upper_bound: None,
ef: None,
approx_mode: None,
refine_factor: None,
distance_type: None,
bypass_vector_index: None,
@@ -432,6 +435,7 @@ impl From<AnyQuery> for PyQueryRequest {
lower_bound: vector_query.lower_bound,
upper_bound: vector_query.upper_bound,
ef: vector_query.ef,
approx_mode: vector_query.approx_mode.map(|m| m.to_string()),
refine_factor: vector_query.refine_factor,
distance_type: vector_query.distance_type.map(|d| d.to_string()),
bypass_vector_index: Some(!vector_query.use_index),
@@ -1069,6 +1073,12 @@ impl VectorQuery {
self.inner = self.inner.clone().ef(ef as usize);
}
pub fn approx_mode(&mut self, approx_mode: String) -> PyResult<()> {
let approx_mode = ApproxMode::try_from(approx_mode.as_str()).infer_error()?;
self.inner = self.inner.clone().approx_mode(approx_mode);
Ok(())
}
pub fn bypass_vector_index(&mut self) {
self.inner = self.inner.clone().bypass_vector_index()
}
@@ -1246,6 +1256,10 @@ impl HybridQuery {
self.inner_vec.ef(ef);
}
pub fn approx_mode(&mut self, approx_mode: String) -> PyResult<()> {
self.inner_vec.approx_mode(approx_mode)
}
pub fn bypass_vector_index(&mut self) {
self.inner_vec.bypass_vector_index();
}
@@ -1291,6 +1305,7 @@ impl HybridQuery {
req.maximum_nprobes = vec_req.maximum_nprobes;
req.search_effort = vec_req.search_effort;
req.ef = vec_req.ef;
req.approx_mode = vec_req.approx_mode;
req.refine_factor = vec_req.refine_factor;
req.lower_bound = vec_req.lower_bound;
req.upper_bound = vec_req.upper_bound;
+3 -2
View File
@@ -333,8 +333,9 @@ impl Display for DistanceType {
/// Controls the speed / accuracy tradeoff for approximate vector search.
///
/// This currently only affects RQ-quantized vector indexes, such as IVF_RQ.
/// Other index types ignore this setting.
/// This currently affects RQ-quantized vector indexes (such as IVF_RQ) and
/// prefiltered search on HNSW sub-indexes, where `Fast` enables the ACORN
/// traversal. Other index types ignore this setting.
#[derive(Debug, Copy, Clone, PartialEq, Eq, Serialize, Deserialize, Default)]
#[non_exhaustive]
#[serde(rename_all = "lowercase")]
+4 -2
View File
@@ -1476,8 +1476,10 @@ impl VectorQuery {
/// Set the speed / accuracy tradeoff for approximate vector search.
///
/// This setting is currently only used by RQ-quantized indexes, such as
/// IVF_RQ. Other index types ignore this setting.
/// This setting is currently used by RQ-quantized indexes (such as IVF_RQ)
/// and by prefiltered search on HNSW sub-indexes, where
/// [`ApproxMode::Fast`] enables the ACORN traversal. Other index types
/// ignore this setting.
pub fn approx_mode(mut self, approx_mode: ApproxMode) -> Self {
self.request.approx_mode = Some(approx_mode);
self