mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-28 22:46:14 +00:00
docs: describe refresh and retries the same way on every page
Five pages still said a refresh never revisits a filled row and that a changed input needs the column dropped and declared again. Since lancedb#4161 a refresh also recomputes rows whose inputs changed, so the quickstart, the feature-engineering overview, Guarantees, and the SQL REFRESH and UPDATE pages now say that. Retry claims are qualified the same way. A local refresh commits the column once, and a deployment's job commits finished batches. A retry fills what is still empty or stale, but can recompute values that were already written (a refresh whose input record was not saved recomputes next time), so no page promises that a retry costs only the unfinished rows.
This commit is contained in:
@@ -31,16 +31,16 @@ separation is the point:
|
||||
- Filling is a separate call you control, can retry, and -- against a deployment
|
||||
-- can hand to the server as a [job](/basics/jobs) rather than waiting on.
|
||||
|
||||
**Filling never revisits work it has already done.** A refresh gives values to
|
||||
rows that lack them and leaves the rest alone, so calling it twice is not twice
|
||||
the work, and appending rows means the next refresh handles only the new ones.
|
||||
**Filling does not redo current work.** A refresh gives values to rows that
|
||||
lack them and recomputes rows whose inputs changed since they were computed. It
|
||||
leaves the rest alone, so calling it twice is not twice the work, and appending
|
||||
rows means the next refresh handles only the new ones.
|
||||
|
||||
The corollary is worth stating plainly: **a filled value is not recomputed when
|
||||
its inputs change.** If a column derives from `price` and a row's `price` is
|
||||
later updated, the derived value stays as it was computed. Recomputing means
|
||||
dropping the column and declaring it again. That is a deliberate trade -- it is
|
||||
what makes refreshes idempotent and cheap -- but it will surprise you if you
|
||||
expect a spreadsheet.
|
||||
The corollary is worth stating plainly: **a filled value catches up with its
|
||||
inputs only when you refresh.** If a column derives from `price` and a row's
|
||||
`price` is later updated, the derived value keeps its old value until the next
|
||||
refresh recomputes it. That is a deliberate trade -- it is what makes refreshes
|
||||
idempotent and cheap -- but it will surprise you if you expect a spreadsheet.
|
||||
|
||||
## Which one to reach for {#which-one}
|
||||
|
||||
@@ -85,16 +85,18 @@ That relocation is the main thing to understand about Enterprise feature
|
||||
engineering. The API is the same; what changes is that the code runs somewhere
|
||||
else, which is also why a function must declare the environment it needs.
|
||||
|
||||
## Failure is resumable {#failures}
|
||||
## Recovering from a failure {#failures}
|
||||
|
||||
A function that raises fails the job. There is no per-row retry or skip policy
|
||||
in the function definition, so a function calling a flaky external service
|
||||
should handle that itself, inside the function body.
|
||||
|
||||
The alternative is to let the job fail and run the refresh again. Since a
|
||||
refresh never revisits a filled row, re-running after a partial failure costs
|
||||
only the rows that did not complete -- **a failed backfill is resumed, not
|
||||
restarted.**
|
||||
The alternative is to let the job fail, fix the cause, and run the refresh
|
||||
again. The new refresh fills what is still empty or stale. How much of the
|
||||
failed run was kept depends on where it ran (see
|
||||
[Refresh a column](/feature-engineering/refresh-column#failures)), and a retry
|
||||
can recompute values that were already written, so do not count on it costing
|
||||
only the rows that did not complete.
|
||||
|
||||
## Where to go next {#next}
|
||||
|
||||
|
||||
@@ -115,8 +115,11 @@ was. A refresh job on a deployment commits finished batches as it goes, so some
|
||||
values can remain after a failure.
|
||||
|
||||
Either way, recover by fixing the cause and refreshing again. The new refresh
|
||||
fills only rows that are still empty or whose inputs changed, so values that
|
||||
were kept are not computed twice.
|
||||
fills rows that are still empty or whose inputs changed, and normally skips
|
||||
values that were kept. That is not guaranteed: a refresh can recompute values
|
||||
that were already written -- for example when the record of the inputs they
|
||||
were computed from was not saved -- so do not count on a retry costing only the
|
||||
rows that did not complete.
|
||||
|
||||
For a column filled by a [function](/management/function), the usual cause is
|
||||
the worker environment rather than the data. See
|
||||
|
||||
Reference in New Issue
Block a user