mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-23 20:22:37 +00:00
Christopher KrahandXuanwo
b6d15558ec
`retry_with_exponential_backoff` retries every exception except an auth
error (`python/python/lancedb/embeddings/utils.py:298-318`). For Cohere,
Gemini, Jina and VoyageAI a missing API key raises from
`api_key_not_found_help`, and a missing provider package raises
`ImportError` everywhere. Neither can succeed on a retry, yet both sleep
through all seven rounds and then surface wrapped in `Exception("Maximum
number of retries...")`.
Before, on `142d032`, a VoyageAI function with no key set (script below,
`time.sleep` recorded, `random.seed(0)`):
```
$ python lance_retry_deterministic.py python/python
raised: Exception('Maximum number of retries (7) exceeded.', ValueError('Please set the VOYAGEAI_API_KEY environment v
attempts: 8 sleeps: [4, 13, 37, 93, 280, 788, 2811]
slept 4026 s before a missing-API-key error surfaced
```
After:
```
$ python lance_retry_deterministic.py python/python
raised: MissingApiKeyError('Please set the VOYAGEAI_API_KEY environment variable.')
attempts: 1 sleeps: []
```
<details><summary>lance_retry_deterministic.py</summary>
```python
# loads the pure-Python embeddings package from the tree, native extension stubbed
import os, random, sys, time, types
TREE = sys.argv[1]
pkg = types.ModuleType("lancedb"); pkg.__path__ = [TREE + "/lancedb"]; sys.modules["lancedb"] = pkg
native = types.ModuleType("lancedb._lancedb"); native.__getattr__ = lambda name: None
sys.modules["lancedb._lancedb"] = native
import pyarrow as pa
from lancedb.embeddings import get_registry
random.seed(0)
os.environ.pop("VOYAGE_API_KEY", None)
slept = []
time.sleep = slept.append
func = get_registry().get("voyageai").create(name="voyage-3")
try:
func.compute_source_embeddings_with_retry(pa.array(["hello"]))
except Exception as e:
print("raised:", repr(e)[:110])
print("attempts:", len(slept) + 1, "sleeps:", [round(s) for s in slept])
print("slept %.0f s before a missing-API-key error surfaced" % sum(slept))
```
</details>
The message names the wrong variable: `voyageai.py:501` reads
`VOYAGE_API_KEY`. I left that for a separate fix.
`api_key_not_found_help` now raises `MissingApiKeyError`, a `ValueError`
subclass, so existing `except ValueError` callers still catch it. That
error and `ImportError` join the 401 exemption from #2995. A plain
`ValueError` is still retried, because `bedrock.py:171` wraps boto3
throttling in one. No issue exists for this. OpenAI is not covered: with
no key, `openai.OpenAI()` raises `OpenAIError`, which has no status
code, so it still retries. I can follow up on that.
### Verification
```
# lance_stub: an empty stand-in for lancedb._lancedb, loaded from the source tree
$ pytest -p lance_stub --noconftest python/tests/test_embeddings.py -k "retry or 401 or deterministic" -q
5 passed, 23 deselected in 0.59s
# utils.py from 142d032, tests kept:
2 failed, 3 passed, 23 deselected in 0.74s
$ pytest -p lance_stub --noconftest python/tests/test_voyageai_embeddings.py -q
18 passed in 0.18s
# not run: the full suite or pyright; the native extension did not build here
```
Baseline on `142d032` has 2 of these tests. The new ones cover a missing
key, a missing package, and a wrapped throttling error that must still
retry. ruff 0.16.10 check and format are clean.
@wjones127 you merged the 401 change. Is a `ValueError` subclass the way
you want to mark this?
Co-authored-by: Xuanwo <github@xuanwo.io>