Files
lancedb/docs/web/feature-engineering/materialized-views.mdx
T
Brendan Clement a2803e1a02 docs: scope the materialized-view note to the SDK it was tested on
The note dated the new behavior to "SDK 0.40", but it was verified on the
development SDK 0.40.0b12 built from main, and no stable 0.40 has been
released. The note now names that build and keeps the comparison with
the published 0.39.0, which creates the view empty, has no with_no_data,
and writes both matching rows on the first refresh.
2026-10-05 15:00:20 -07:00

129 lines
4.7 KiB
Plaintext

---
title: "Refresh a materialized view"
sidebarTitle: "Materialized View Refresh"
description: "A lakehouse object: a table defined by a query over another table, kept current incrementally."
icon: table
keywords: ["materialized view", "create_materialized_view", "refresh", "incremental"]
---
import {
PyMaterializedViewCreate as MaterializedViewCreate,
PyMaterializedViewRefresh as MaterializedViewRefresh,
} from '/snippets/materialized_view.mdx';
A materialized view is a lakehouse object: a table whose contents are defined by
a query over another table. The query is recorded in the view's schema;
creating the view computes its rows, and a refresh brings them up to date.
The result is an ordinary table. It can be queried, indexed, and searched, and
it appears in `table_names()` — nothing about reading it is special.
<Note>
In SQL this is [`REFRESH MATERIALIZED VIEW`](/sql/refresh#refresh-materialized-view).
</Note>
## Create and refresh {#create-and-refresh}
<CodeBlock filename="Python" language="Python" icon="python">
{MaterializedViewCreate}
</CodeBlock>
Creating the view computes its rows, so it is ready to read. Pass
`with_no_data=True` to create it empty instead. Later changes to the source
reach the view through a refresh:
<CodeBlock filename="Python" language="Python" icon="python">
{MaterializedViewRefresh}
</CodeBlock>
<Note>
These results were verified with the development SDK 0.40.0b12, built from
lancedb main. With the published 0.39.0, `create_materialized_view` creates the
view empty and has no `with_no_data`, so the first `refresh()` writes every
matching row, two in this example.
</Note>
`select` takes column names, `(alias, expression)` pairs, or a dict of the same.
Omitting it selects every source column, expanded against the schema as it was
at creation time. `where` filters source rows; `limit` caps the view.
## The source needs stable row ids {#stable-row-ids}
<Warning>
The source table must have been created with stable row ids:
```python
db = lancedb.connect(uri, storage_options={"new_table_enable_stable_row_ids": "true"})
```
They are what keeps the view's provenance valid when the source is compacted,
and **they cannot be enabled after a table exists**. A table created without
them cannot later become the source of a view — it has to be recreated.
</Warning>
This is the one decision here that is hard to undo, so it is worth setting the
option on any table you might later want to derive from.
## Refreshes are incremental {#incremental}
`refresh()` reconciles the source's changes into the view — rows added, changed,
or removed since the last refresh — rather than recomputing from scratch. It
falls back to a full rebuild when the changes cannot be reconciled.
Two arguments override that:
- `full=True` forces a rebuild.
- `source_version=N` refreshes against that source version instead of the
latest, which is how you make a view reproduce a known point in time.
Concurrent refreshes of one view do not duplicate rows. If two plan the same
source rows, they conflict on commit and the loser raises rather than writing
them twice — so a retry loop around `refresh()` is safe.
<Note>
**Enterprise job type:** `mat_view_refresh`.
</Note>
## Against a deployment {#availability}
<Badge color="red">Enterprise</Badge>
`create_materialized_view` works on a `db://` connection as well as a local one.
What changes is where the refresh runs -- across the cluster rather than in your
process -- and that SQL can drive it:
```sql
REFRESH MATERIALIZED VIEW chunks
```
`WITH (full = true)` recomputes every row instead of reconciling what changed
in the source since the last refresh. `SHOW MATERIALIZED VIEWS` lists them, which is
cheaper than `list_materialized_views` -- that method finds views by reading
every table's schema, so it costs one open per table.
<Note>
`CASCADE` is accepted by the parser but not yet implemented. A view built on
another view is refreshed by refreshing each in dependency order.
</Note>
## Working with the view {#working-with-the-view}
`view.table` is the underlying table, for querying, indexing, or search.
`view.definition` is the recorded query — the select list, predicate, and
limit — which is how you check what a view actually contains without inferring
it from the rows. Both are properties; on an async view, the definition is read
with `await view.definition()`.
An existing view is reopened with `db.open_materialized_view(name)`.
## What to read next {#next}
<CardGroup cols={2}>
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
Derive a column instead of a whole table.
</Card>
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
Every refresh commits a table version.
</Card>
</CardGroup>