Avanish GargandXuanwo 2d54630a3c fix(python): stop retrying permanent embedding errors and cap the backoff delay (#4542) ...

Permanent embedding errors (HTTP 400/404/422 and other 4xx) were retried
the full `max_retries` times with an
uncapped, multiplicatively jittered backoff, so a typo'd model name
looked like a hang of about an hour
(median 50 min of sleeping with a 404; see the issue for the repro and
numbers).

Changes (`python/python/lancedb/embeddings/utils.py`,
`embeddings/base.py`):
- An HTTP status in 4xx other than 408/409/429 is raised immediately.
The status is read from `status_code`,
`http_status`, `response.status_code`, an integer `code`
(google-api-core, urllib) or a botocore
`response["ResponseMetadata"]["HTTPStatusCode"]`. AWS throttling
reported as 400 stays retryable (matched by
  error code). `AuthenticationError` by name is unchanged.
- Still retried: 408, 409, 429, >=500, and anything without a
recognisable status (connection errors, timeouts,
custom exceptions), so existing custom embedding functions behave as
before.
- New keyword `max_delay=60` (last in the signature, default keeps call
sites working). `delay = min(delay * base,
max_delay)` with equal jitter `delay/2 + random() * delay/2`. With
defaults the waits are [1,2], [2,4], [4,8],
[8,16], [16,32], [30,60], [30,60] s, worst case about 182 s instead of
up to 2.5 h.
- The "Maximum number of retries" exception now has `from e`. The
`max_retries` comment in `base.py` describes the
  rules.

Not changed: `embeddings/openai.py` (it swallows `BadRequestError`; open
#4154/#4005/#3152 cover it), Retry-After
handling (the code never used it), and the "C Data interface error"
wrapping from the native layer (the original
message is now included in it).

Behaviour changes to be aware of: every 4xx except 408/409/429 fails
fast, including 401/403 coming through
`response.status_code` (requests/httpx), which used to be retried; a
provider that signals transient failures with
a 4xx and no AWS-style code is no longer retried; the first retry waits
1-2 s instead of 2-4 s.

Tests (`python/python/tests/test_embeddings.py`): 48 new cases (23 test
functions): permanent errors are not
retried, transient errors are, botocore-style errors, the delay is
capped, exact exponential growth without jitter,
jitter bounds, no float overflow for a large `max_retries`, the original
error is attached, and end to end through
`table.add` and `table.search` with a stub function (one call and no
sleep). On `main` 23 of them fail with the
change stashed; the rest are guards that must not change.
`test_embeddings.py`: 61 passed, 4 skipped.
`ruff format`/`ruff check` are clean on the touched files.

Verification caveat: `main`'s Python layer needs a newer native module
than the 0.40.0 wheel, so the tests were
run with the wheel's native module preloaded through a small shim (the
same shim ran the baseline on `main`); a
wider run of embeddings/pydantic/query/table tests has the same 36
failures before and after (that mismatch).
Not run: real provider APIs (a fake OpenAI-style server was used), Node
or Rust code.

Fixes #4541

Co-authored-by: Xuanwo <github@xuanwo.io>

2026-10-23 03:22:24 +08:00

..

lancedb

fix(python): stop retrying permanent embedding errors and cap the backoff delay (#4542)

2026-10-23 03:22:24 +08:00

tests

fix(python): stop retrying permanent embedding errors and cap the backoff delay (#4542)

2026-10-23 03:22:24 +08:00

type_tests

fix(python): expose inline types to downstream checkers (#3817)

2026-08-07 17:31:42 +08:00