docs: restructure Documentation around one axis, promote the SQL reference

Every group now answers exactly one question. Get started and Concepts cover
what the system is; the verb groups cover doing something with it; Operate,
Secure and Deploy cover running it; SQL reference is lookup. Maintenance becomes
Operate, Governance becomes Secure, and the rest take verbs to match.

Three new pages in Concepts and Get started:

Guarantees states what a write promises when it returns, which version a reader
sees, what survives a failure, and -- the half usually missing -- what is not
guaranteed: no multi-table transactions, no read-your-writes across processes
without an interval, unenforced primary keys that do not enforce, no background
compaction.

Limits collects bounds that were stated in passing across a dozen pages, with
the numbers verified against the code rather than inferred: name lengths, job
event caps, SHOW ERRORS bounds, the sampling range that is refused rather than
clamped.

Architecture illustrates embedded against distributed as the one deployment
decision, and is written to cover both rather than only Enterprise.

Indexing splits by question. Choosing an index is a Query decision and moves
there; building, repairing and GPU-building it stay in Operate.

SQL becomes a top-level reference of fifteen pages instead of a sub-group of
Query. Auditing sophon-sql turned up a large undocumented surface: ALTER TABLE
ADD COLUMN in its plain, computed and function forms with FILTER, IGNORE NULLS
and OVER modifiers; SET UNENFORCED PRIMARY KEY; SET and UNSET WAL with sharding;
CREATE FUNCTION with its dependency options; REFRESH COLUMN and REFRESH
MATERIALIZED VIEW; and the SHOW family. None of it was written down.
This commit is contained in:
Jack Ye
2026-09-13 23:32:15 -07:00
parent 7b112debca
commit 5b5ed930ac
13 changed files with 1257 additions and 65 deletions
+95
View File
@@ -0,0 +1,95 @@
---
title: "REFRESH"
sidebarTitle: "Refresh"
description: "Filling computed columns and bringing materialized views up to date."
icon: arrows-rotate
keywords: ["refresh column", "refresh materialized view", "backfill", "cascade"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
LanceDB extensions with no DataFusion equivalent. They are what makes
[feature engineering](/feature-engineering) reachable from SQL: a column
declared by [DDL](/sql/ddl#computed-column) starts empty, and a refresh fills it.
## REFRESH COLUMN {#refresh-column}
```sql
REFRESH COLUMN <column> ON [TABLE] <table>
[WHERE <predicate>]
[WITH (<option> = <value> [, ...])]
```
```sql
REFRESH COLUMN doubled ON my_table
REFRESH COLUMN embedding ON documents WHERE created_at > '2026-01-01'
```
A refresh gives values to rows that lack them and **leaves filled rows alone**.
That makes it idempotent, cheap to repeat as data arrives, and resumable after a
failure -- re-running costs only the rows that did not complete.
`WHERE` scopes the refresh, which is how you fill a slice and check it before
committing to the whole table.
### Options {#refresh-column-options}
| Option | Meaning |
| --- | --- |
| `num_workers` | how many workers to use |
| `cpu_millis` | CPU budget per worker |
| `memory_bytes` | memory budget per worker |
| `gpu_count` | GPUs per worker |
| `batch_size` | rows a worker takes at a time |
| `error_policy` | `'skip'` continues past failing rows instead of failing the job |
```sql
REFRESH COLUMN embedding ON documents
WITH (cpu_millis = 8000, memory_bytes = 17179869184, gpu_count = 2)
```
`num_workers` sets a count; the budget options size each worker. They are
alternatives rather than a pair.
<Note>
`error_policy` and `num_workers` are refused for a column filled by a registered
[function](/management/function) with its own resource budget -- use the CPU,
memory and GPU options there.
</Note>
## REFRESH MATERIALIZED VIEW {#refresh-materialized-view}
```sql
REFRESH MATERIALIZED VIEW <name> [WITH (<option> = <value>)]
```
```sql
REFRESH MATERIALIZED VIEW chunks
REFRESH MATERIALIZED VIEW chunks WITH (full = true)
```
By default a refresh is **incremental** -- it processes what the source has
added since the last one. `full = true` recomputes every row, which is what you
want after changing something the view's rows depend on.
<Warning>
`CASCADE` parses but is **not implemented**. A view built on another view is
refreshed by refreshing each in dependency order yourself.
</Warning>
## Watching a refresh {#watching}
Both statements submit work that outlives the call. Track it with
[`SHOW JOBS`](/sql/show#jobs), and diagnose failures with
[`SHOW ERRORS`](/sql/show#errors).
## See also {#see-also}
<CardGroup cols={2}>
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
The same operation from the SDKs, and what a refresh guarantees.
</Card>
<Card title="DDL" icon="pen-ruler" href="/sql/ddl#columns">
Declaring the columns this fills.
</Card>
</CardGroup>