## What's broken
`Table.update(values={...})` raises `NotImplementedError: SQL conversion
is not implemented for this type` when a value is a numpy scalar such as
`np.int64`, `np.int32`, `np.float32`, or `np.bool_`. These arise
naturally from indexing an ndarray or a pandas int/bool column.
`np.float64` happens to work (it subclasses `float`), which makes the
failure inconsistent and surprising.
```python
df = pd.DataFrame({"id": np.array([10, 20], dtype="int32")})
t.update(where="id = 1", values={"id": df["id"].iloc[0]}) # np.int32
# -> NotImplementedError: SQL conversion is not implemented for this type
```
## Why it happens
`value_to_sql` is a `singledispatch` with handlers only for native
Python types and `np.ndarray`; numpy `integer`/`floating`/`bool_`
scalars aren't Python subclasses, so they fall through to the
`NotImplementedError` base.
## Fix
Register handlers for `np.bool_`, `np.integer`, and `np.floating` that
delegate to the existing native handlers.
## Test
`value_to_sql` on `np.int32/int64/float32/float64/bool_` all convert;
`np.int32` raised before.
Co-authored-by: Ishaan Samantray <ishaansamantray@Ishaans-MacBook-Pro.local>
## What's broken
Calling `RRFReranker().rerank_multivector([])` crashes with `IndexError:
list index out of range` because the method accesses `vector_results[0]`
for the type-homogeneity check before verifying the list is non-empty.
The `all()` call passes vacuously on an empty iterable so the crash hits
the next lines.
```python
from lancedb.rerankers import RRFReranker
RRFReranker().rerank_multivector([])
# IndexError: list index out of range
```
## Why it happens
The type check uses `vector_results[0]` as the reference type but never
guards against an empty list. `all(...)` short-circuits to `True` when
the iterable is empty, so the bad index access on the lines that follow
is never reached by the existing guard logic.
## Fix
Add an explicit empty-list check before any indexing.
## What's broken
`MRRReranker.rerank_multivector([])` raises `IndexError: list index out
of range`. The crash happens on line 128 (the `all()` type-homogeneity
check passes vacuously on an empty iterable) and on line 134 which
accesses `vector_results[0]` unconditionally, with no prior guard for an
empty list.
## Why it happens
`all()` over an empty iterable returns `True`, so the type check
silently passes and execution falls through to `vector_results[0]` which
crashes.
## Fix
Added a two-line guard at the top of `rerank_multivector` that raises a
clear `ValueError("vector_results must not be empty")` before any
indexing occurs.
## Test
Added `test_mrr_reranker_empty_input` in `test_rerankers.py` which calls
`rerank_multivector([])` and asserts that a `ValueError` with the
message "must not be empty" is raised.
Fixes#3468
Co-authored-by: Aegis Dev <aegis@devteamaegis.com>
## Problem
`LinearCombinationReranker.merge_results` has two related bugs that make
it return **inverted relevance rankings** — the least relevant document
ranks first (closes#3154).
### Bug 1 — `_combine_score` subtracts from 1, inverting the final
ranking
```python
def _combine_score(self, vector_score, fts_score):
return 1 - (self.weight * vector_score + (1 - self.weight) * fts_score)
```
Both `vector_score` (already converted via `_invert_score`) and
`fts_score` (BM25 relevance) are in **higher-is-better** space. Wrapping
the weighted average in `1 - (...)` flips the direction: a perfectly
matching document (`vector_score=1, fts_score=1`) gets `_relevance_score
= 0.0`, while a non-matching document gets a high score.
### Bug 2 — Documents missing an FTS score are rewarded, not penalised
```python
fts_score = result.get("_score", fill) # fill=1.0 by default
```
When a document has no FTS match, `fts_score = fill = 1.0`. In
`_combine_score` (with the bug-1 formula), this large value becomes a
**negative penalty** via `1 - (... + 0.3 * 1.0)`, counterintuitively
*boosting* the document's score. By contrast, missing vector results
correctly receive `_invert_score(fill) = 0.0` (penalised).
## Fix
**Bug 1** — remove the `1 -` inversion from `_combine_score`:
```python
def _combine_score(self, vector_score, fts_score):
return self.weight * vector_score + (1 - self.weight) * fts_score
```
**Bug 2** — use `1 - fill` for missing FTS scores so both penalties are
symmetric (mirror of what `_invert_score(fill)` already does for missing
vector scores):
```python
fts_score = result.get("_score", 1 - fill) # was: fill
```
With `fill=1.0` (default): `1 - 1.0 = 0.0` — missing-FTS entries
contribute `0` to the FTS term, identical to how missing-vector entries
contribute `0` to the vector term.
## Verification
Concrete example from the issue. With `weight=0.7`, `fill=1.0`:
| Document | `_distance` | `_score` | Old `_relevance_score` | New
`_relevance_score` |
|----------|-------------|----------|------------------------|------------------------|
| `apple orange` | 0.0 (best) | 2.41 (only FTS) | 0.30 (**wrong: ranked
2nd**) | 1.42 (**correct: ranked 1st**) |
| `banana grape` | 0.9999 (worst) | — | 0.70 (**wrong: ranked 1st**) |
0.00 (**correct: ranked last**) |
## Tests
Two regression tests added to `python/python/tests/test_rerankers.py`:
- `test_linear_combination_best_match_ranks_first` — the document with
the smallest distance **and** an FTS match must have the highest
`_relevance_score`.
- `test_linear_combination_missing_fts_is_penalised` — a document with
any FTS score must beat an otherwise-equal document with no FTS match.
---------
Co-authored-by: Will Jones <willjones127@gmail.com>
devteamaegisandIshaan Samantray
