fix(python): rank each result list separately in RRF multivector rerank (#4325)

## What's wrong

`RRFReranker.rerank_multivector` concatenates every vector result list
and passes the concatenation to `rerank_hybrid` as a single list.
`rerank_hybrid` then scores each row by its position in that
concatenation, so a row's rank depends on how many rows came before it
in *earlier* lists.

With two searches of 3 rows each (`K=60`):

| row | rank in its own list | score today | RRF score |
|---|---|---|---|
| 1 (top of list 1) | 1 | 1/61 | 1/61 |
| 3 (third of list 1) | 3 | 1/63 | 1/63 |
| 4 (top of list 2) | 1 | **1/64** | 1/61 |
| 5 (second of list 2) | 2 | **1/65** | 1/62 |

So the best hit of the second search ranks below the worst hit of the
first, and the order of the lists passed in decides the fused ranking.
`MRRReranker.rerank_multivector` already scores each list on its own
("MRR semantics require treating each vector result as a separate
ranking system"). RRF is defined the same way.

## Fix

Score each result list with its own ranks (`1 / (rank + K)`, summed
across lists for rows found by more than one), then dedupe the
concatenation by `_rowid`, attach `_relevance_score`, and sort. The
`return_score` handling is unchanged.

## Tests

`test_rrf_multivector_ranks_each_result_list` in
`python/python/tests/test_rerankers.py`:

- On `main`, it fails with `assert 0.015625 == 0.01639344262295082` (the
top hit of list 2 got `1/64` instead of `1/61`).
- With the fix, it passes, and so do the other tests in
`test_rerankers.py` (16 passed, 12 skipped for missing API keys or
models).
- `ruff format --check` and `ruff check` are clean.

The fix was found and written by an AI agent (Breken) and checked
against the tests above.

Co-authored-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
This commit is contained in:
breken
2026-09-28 20:19:51 -05:00
committed by GitHub
co-authored by breken-ai
parent ef11e44adc
commit c119f0c8e5
2 changed files with 52 additions and 4 deletions
+32
View File
@@ -395,6 +395,38 @@ def test_mrr_multivector_rewards_consensus():
assert result["_rowid"].to_pylist()[0] == 2
def test_rrf_multivector_ranks_each_result_list():
# RRF scores a document by its rank *within each* result list. The top hit
# of the second list must score the same as the top hit of the first list,
# not as if it were ranked after every row of the first list.
reranker = RRFReranker(K=60)
def ranking(row_ids):
return pa.table({"_rowid": pa.array(row_ids, type=pa.int64())})
rs1 = ranking([1, 2, 3])
rs2 = ranking([4, 5, 6])
result = reranker.rerank_multivector([rs1, rs2])
scores = dict(
zip(result["_rowid"].to_pylist(), result["_relevance_score"].to_pylist())
)
assert scores[1] == pytest.approx(1 / 61)
assert scores[4] == pytest.approx(1 / 61)
assert scores[5] == pytest.approx(1 / 62)
# the second-ranked hit of the second list beats the third hit of the first
assert scores[5] > scores[3]
# A document found by both lists sums its reciprocal ranks from each list.
result = reranker.rerank_multivector([ranking([1, 2]), ranking([2, 3])])
scores = dict(
zip(result["_rowid"].to_pylist(), result["_relevance_score"].to_pylist())
)
assert scores[2] == pytest.approx(1 / 62 + 1 / 61)
assert result["_rowid"].to_pylist() == [2, 1, 3]
def test_rrf_reranker_distance():
data = pa.table(
{