Mohammad HijjawiandXuanwo a0946282ca fix(python): return original scores from hybrid search with normalize="rank" (#4343) ...

## Summary

With `rerank(..., normalize="rank")`, a hybrid query returns ranks (1.0,
2.0, 3.0, ...) in the `_distance` and `_score` columns instead of the
real vector distances and BM25 scores.

`_combine_hybrid_results` converted both result sets to ranks first, and
only then captured `original_distances` / `original_scores`. So the step
that restores the original values after reranking put the ranks back.
With `normalize="score"`, the real values are returned as expected.

Repro (RRFReranker(return_score="all"), vector distances 0 / 25 / 100):

| normalize | `_distance` returned | `_score` returned |
| --- | --- | --- |
| `"score"` | 0.0, 25.0, 100.0 | 0.47, 0.67, None |
| `"rank"` (before) | 1.0, 2.0, 3.0 | 1.0, 2.0, None |
| `"rank"` (after) | 0.0, 25.0, 100.0 | 0.47, 0.67, None |

## Changes

- Capture the original `_distance` / `_score` columns before converting
them to ranks, then normalize the ranked column. The reranker input is
unchanged; only the values restored afterwards are fixed.
`AsyncHybridQuery` uses the same function, so both sync and async are
covered.

## Test plan

- [x] New `test_hybrid_return_all_restores_original_scores` in
`test_rerankers.py`, parametrized over `score` / `rank`: the `rank` case
fails before the fix and passes after
- [x] `pytest python/tests/test_rerankers.py
python/tests/test_hybrid_query.py`
- [x] `ruff format --check` / `ruff check`

I have no local Rust toolchain, so I ran the tests against the released
0.39.0 native module with the changed Python files overlaid. The changed
functions are identical between 0.39.0 and main.

Co-authored-by: Xuanwo <github@xuanwo.io>

2026-10-23 02:42:06 +08:00

..

lancedb

fix(python): return original scores from hybrid search with normalize="rank" (#4343)

2026-10-23 02:42:06 +08:00

tests

fix(python): return original scores from hybrid search with normalize="rank" (#4343)

2026-10-23 02:42:06 +08:00

type_tests

fix(python): expose inline types to downstream checkers (#3817)

2026-08-07 17:31:42 +08:00