Files
lancedb/docs/web/feature-engineering/materialized-views.mdx
T
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00

125 lines
5.5 KiB
Plaintext

---
title: "Refresh a materialized view"
sidebarTitle: "Materialized View Refresh"
description: "A lakehouse object: a table defined by a query over another table, brought up to date by each refresh."
icon: table
keywords: ["materialized view", "create_materialized_view", "refresh", "full refresh"]
---
import {
PyMaterializedViewCreate as MaterializedViewCreate,
PyMaterializedViewRefresh as MaterializedViewRefresh,
} from '/snippets/materialized_view.mdx';
A materialized view is a lakehouse object: a table whose contents are defined by
a query over another table. The query is recorded in the view's schema;
creating the view computes its rows, and a refresh brings them up to date.
The result is an ordinary table. It can be queried, indexed, and searched, and
it appears in `table_names()` — nothing about reading it is special.
<Note>
In SQL this is [`REFRESH MATERIALIZED VIEW`](/sql/refresh#refresh-materialized-view).
</Note>
## Create and refresh {#create-and-refresh}
<CodeBlock filename="Python" language="Python" icon="python">
{MaterializedViewCreate}
</CodeBlock>
Creating the view computes its rows, so it is ready to read. Pass
`with_no_data=True` to create it empty instead. Later changes to the source
reach the view through a refresh:
<CodeBlock filename="Python" language="Python" icon="python">
{MaterializedViewRefresh}
</CodeBlock>
<Note>
These results were verified with the development SDK 0.40.0b13, built from
lancedb main. The published 0.39.0 differs: the source must be created with
stable row ids (the `new_table_enable_stable_row_ids` storage option),
`create_materialized_view` creates the view empty and has no `with_no_data`,
and `refresh()` still accepts `full`. Its first `refresh()` writes every
matching row, two in this example.
</Note>
`select` takes column names, `(alias, expression)` pairs, or a dict of the same.
Omitting it selects every source column, expanded against the schema as it was
at creation time. `where` filters source rows; `limit` caps the view.
## Local sources no longer need stable row ids {#stable-row-ids}
A local refresh does not track which source rows the view came from, so a local
view's source does not need stable row ids (`new_table_enable_stable_row_ids`):
the example above creates its source with the defaults. Earlier development
builds and the published 0.39.0 required them, and a deployment may still
require them; see [Against a deployment](#availability).
The source is still checked, when the view is created and on every refresh: one
with an LSM write spec, or with retained write-ahead-log rows not yet compacted,
is refused.
## Every local refresh rebuilds the view {#incremental}
Locally, `refresh()` runs the view's query against the source and replaces
everything the view holds with the result, in one commit. It does not reconcile
changes: `rows_written` is the size of the whole result — two after adding Bea,
not one — and a refresh with nothing new to read still rewrites the view and
commits a new table version.
`source_version=N` refreshes against that source version instead of the latest,
which is how a view reproduces a known point in time. There is no `full`
argument; every local refresh is a full one.
## Against a deployment {#availability}
<Badge color="red">Enterprise</Badge>
`create_materialized_view` and `refresh` also work on a `db://` connection.
There the SDK sends them to the deployment as SQL -- `CREATE MATERIALIZED VIEW`
and `REFRESH MATERIALIZED VIEW` -- and the work runs on the cluster rather than
in your process. `source_version` is for local views only; the SDK refuses it
on a remote refresh.
Materialized views are off by default on a deployment and have to be enabled
there. How a deployment refreshes a view -- rebuilding the whole view, as above,
or reconciling what changed in the source -- depends on its LanceDB Enterprise
version and configuration, and so do the options
[`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh#refresh-materialized-view)
accepts and whether the source table needs stable row ids.
Stable row ids are set up on the deployment, not by the SDK: a `db://`
connection does not send `new_table_enable_stable_row_ids`, and no SDK call adds
them to an existing table. Depending on its version and configuration, a
deployment may give new tables stable row ids, may refuse a view created through
SQL -- as the SDK creates one -- over a source without them, or may convert an
existing source when a view is created through its REST API. Check with the
deployment's administrator what it requires before relying on a source.
`SHOW MATERIALIZED VIEWS` lists them, which is cheaper than
`list_materialized_views` -- that method finds views by reading every table's
schema, so it costs one open per table.
## Working with the view {#working-with-the-view}
`view.table` is the underlying table, for querying, indexing, or search.
`view.definition` is the recorded query — the select list, predicate, and
limit — which is how you check what a view actually contains without inferring
it from the rows. Both are properties; on an async view, the definition is read
with `await view.definition()`.
An existing view is reopened with `db.open_materialized_view(name)`.
## What to read next {#next}
<CardGroup cols={2}>
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
Derive a column instead of a whole table.
</Card>
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
Every refresh commits a table version.
</Card>
</CardGroup>