mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-16 16:52:34 +00:00
Brendan Clement
06aa526ae1
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. The curation, SQL and operations pages contain outdated claims and examples that do not reliably work as written. This corrects those pages against the reviewed SDK and local server evidence, including prepared statements, background jobs, cleanup and monitoring. Integration guides are grouped with their examples in #4533. Function and Secret-object documentation stays unpublished until its release. The two Function pages are listed in `.mintignore` and left out of the navigation, and the sections about them on other pages are MDX comments. Stacked on #4533. Each MDX file has one PR owner. Rebase onto main after #4533 is squash-merged. Local assembly, Mint validation and link checks pass, and the assembled site shows no Function or Secret-object content. A local release rehearsal that uncomments the drafts and restores the navigation also passes. Hold publication until this PR and lancedb/sophon#8688 are on `main`.
118 lines
5.0 KiB
Plaintext
118 lines
5.0 KiB
Plaintext
---
|
|
title: "Refresh a materialized view"
|
|
sidebarTitle: "Materialized View Refresh"
|
|
description: "Create a materialized view and refresh it when the source data changes."
|
|
icon: table
|
|
keywords: ["materialized view", "create_materialized_view", "refresh", "full refresh"]
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Both offerings support materialized views. OSS refreshes run in your process; Enterprise runs the work on remote compute. See [Enterprise refreshes](#availability) for server compatibility requirements.
|
|
---
|
|
|
|
import {
|
|
PyMaterializedViewCreate as MaterializedViewCreate,
|
|
PyMaterializedViewRefresh as MaterializedViewRefresh,
|
|
} from '/snippets/materialized_view.mdx';
|
|
|
|
A materialized view stores the result of a query as a table. Use it to keep a
|
|
filtered or transformed dataset that you can query, index and search repeatedly.
|
|
Refresh the view after its source data changes.
|
|
|
|
<Note>
|
|
In SQL this is [`REFRESH MATERIALIZED VIEW`](/sql/refresh#refresh-materialized-view).
|
|
</Note>
|
|
|
|
## Create and refresh {#create-and-refresh}
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{MaterializedViewCreate}
|
|
</CodeBlock>
|
|
|
|
Creating the view computes its rows, so it is ready to read. Pass
|
|
`with_no_data=True` to create it empty instead. Later changes to the source
|
|
reach the view through a refresh:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{MaterializedViewRefresh}
|
|
</CodeBlock>
|
|
|
|
The example works with Python SDK 0.40.0. It creates the source with stable
|
|
row IDs, which that release requires for ordinary materialized-view sources.
|
|
|
|
`select` takes column names, `(alias, expression)` pairs, or a dict of the same.
|
|
Omitting it selects every source column, expanded against the schema as it was
|
|
at creation time. `where` filters source rows; `limit` caps the view.
|
|
|
|
## Prepare the source table {#stable-row-ids}
|
|
|
|
In Python SDK 0.40.0, create the source with
|
|
`storage_options={"new_table_enable_stable_row_ids": "true"}`, as the example
|
|
above does. Stable row IDs let the view track source rows across updates and
|
|
compaction. Set this option when creating the table; it cannot be added later.
|
|
|
|
A source with an LSM write specification or retained, uncompacted write-ahead-log
|
|
rows is not accepted by the local materialized-view implementation.
|
|
|
|
## Refresh behavior {#incremental}
|
|
|
|
With Python SDK 0.40.0, a refresh applies source changes incrementally when the
|
|
query supports it, and rebuilds the view otherwise. In the example, adding Bea
|
|
writes one new view row. Refreshing again without source changes writes none.
|
|
Use `refresh(full=True)` to force a rebuild.
|
|
|
|
`rows_written` counts rows written by that refresh, not all rows in the view.
|
|
Use `view.table.count_rows()` to get the total. Pass `source_version=N` to
|
|
refresh against a particular source version instead of the latest.
|
|
|
|
<Note>
|
|
Refresh behavior has changed between SDK versions. The earlier 0.40.0b13
|
|
development build rebuilt every local view on refresh and did not accept
|
|
`full`; 0.39.0 created views empty. Use the reference for your installed SDK
|
|
when maintaining an existing pipeline.
|
|
</Note>
|
|
|
|
## Enterprise refreshes {#availability}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
Enterprise runs materialized-view creation and refresh on remote compute.
|
|
For SQL, see [`CREATE MATERIALIZED VIEW`](/sql/ddl/materialized-view) and
|
|
[`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh#refresh-materialized-view).
|
|
|
|
Use an SDK version supported by your Enterprise deployment. Materialized-view
|
|
APIs and stored definitions have changed across releases; a local SDK example
|
|
alone does not establish compatibility with an older server. Before upgrading
|
|
an existing pipeline, confirm the supported refresh options and whether its
|
|
views need to be recreated with your administrator.
|
|
|
|
Stable row IDs and feature configuration for Enterprise sources are managed on
|
|
the server. The source-table storage option in the local example does not
|
|
configure the Enterprise deployment, and Python SDK 0.40.0's `create_table` on a
|
|
`db://` connection does not accept `storage_options`: leave it out, and confirm
|
|
with your administrator that the deployment gives new tables stable row IDs.
|
|
|
|
`list_materialized_views` requests the server's view listing. On a local
|
|
database it reads table schemas to find views.
|
|
|
|
## Working with the view {#working-with-the-view}
|
|
|
|
`view.table` is the underlying table, for querying, indexing, or search.
|
|
`view.definition` is the recorded query: the select list, predicate, and
|
|
limit. Use it to inspect the query without inferring it from the rows. Both are properties; on an async view, the definition is read
|
|
with `await view.definition()`.
|
|
|
|
An existing view is reopened with `db.open_materialized_view(name)`.
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
|
|
Derive a column instead of a whole table.
|
|
</Card>
|
|
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
|
|
Inspect the table history and read a particular version.
|
|
</Card>
|
|
</CardGroup>
|