mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 15:26:52 +00:00
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
125 lines
5.4 KiB
Plaintext
125 lines
5.4 KiB
Plaintext
---
|
|
title: "Refresh a materialized view"
|
|
sidebarTitle: "Materialized View Refresh"
|
|
description: "Create a materialized view and refresh it when the source data changes."
|
|
icon: table
|
|
keywords: ["materialized view", "create_materialized_view", "refresh", "full refresh"]
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Creation and refresh in LanceDB Embedded require a source build with the experimental-functions Cargo feature. Enterprise runs the work on the Enterprise service. See [Enterprise refreshes](#availability) for server compatibility requirements.
|
|
---
|
|
|
|
import {
|
|
PyMaterializedViewCreate as MaterializedViewCreate,
|
|
PyMaterializedViewRefresh as MaterializedViewRefresh,
|
|
} from '/snippets/materialized_view.mdx';
|
|
|
|
A materialized view stores the result of a query as a table. Use it to keep a
|
|
filtered or transformed dataset that you can query, index and search repeatedly.
|
|
Refresh the view after its source data changes.
|
|
|
|
<Note>
|
|
In the 0.41 SDK release, materialized-view creation and refresh in LanceDB Embedded require a
|
|
source build with the `experimental-functions` Cargo feature. Published Python
|
|
and TypeScript packages leave it disabled. Opening, reading, listing and dropping
|
|
existing views remain available, as do LanceDB Enterprise operations.
|
|
</Note>
|
|
|
|
<Note>
|
|
In SQL this is [`REFRESH MATERIALIZED VIEW`](/sql/refresh#refresh-materialized-view).
|
|
</Note>
|
|
|
|
## Create and refresh {#create-and-refresh}
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{MaterializedViewCreate}
|
|
</CodeBlock>
|
|
|
|
Creating the view computes its rows, so it is ready to read. Pass
|
|
`with_no_data=True` to create it empty instead. Later changes to the source
|
|
reach the view through a refresh:
|
|
|
|
<CodeBlock filename="Python" language="Python" icon="python">
|
|
{MaterializedViewRefresh}
|
|
</CodeBlock>
|
|
|
|
The example works with Python SDK 0.40.0. It creates the source with stable
|
|
row IDs, which that release requires for ordinary materialized-view sources.
|
|
|
|
`select` takes column names, `(alias, expression)` pairs, or a dict of the same.
|
|
Omitting it selects every source column, expanded against the schema as it was
|
|
at creation time. `where` filters source rows; `limit` caps the view.
|
|
|
|
## Prepare the source table {#stable-row-ids}
|
|
|
|
In Python SDK 0.40.0, create the source with
|
|
`storage_options={"new_table_enable_stable_row_ids": "true"}`, as the example
|
|
above does. Stable row IDs let the view track source rows across updates and
|
|
compaction. Set this option when creating the table; it cannot be added later.
|
|
|
|
A source with the [realtime engine](/tables/realtime-engine) enabled or retained, uncompacted write-ahead-log
|
|
rows is not accepted by the local materialized-view implementation.
|
|
|
|
## Refresh behavior {#incremental}
|
|
|
|
With Python SDK 0.40.0, a refresh applies source changes incrementally when the
|
|
query supports it, and rebuilds the view otherwise. In the example, adding Bea
|
|
writes one new view row. Refreshing again without source changes writes none.
|
|
Use `refresh(full=True)` to force a rebuild.
|
|
|
|
`rows_written` counts rows written by that refresh, not all rows in the view.
|
|
Use `view.table.count_rows()` to get the total. Pass `source_version=N` to
|
|
refresh against a particular source version instead of the latest.
|
|
|
|
<Note>
|
|
Refresh behavior has changed between SDK versions. The earlier 0.40.0b13
|
|
development build rebuilt every local view on refresh and did not accept
|
|
`full`; 0.39.0 created views empty. Use the reference for your installed SDK
|
|
when maintaining an existing pipeline.
|
|
</Note>
|
|
|
|
## Enterprise refreshes {#availability}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
Enterprise runs materialized-view creation and refresh on the Enterprise service.
|
|
For SQL, see [`CREATE MATERIALIZED VIEW`](/sql/ddl/materialized-view) and
|
|
[`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh#refresh-materialized-view).
|
|
|
|
Use an SDK version supported by your Enterprise deployment. Materialized-view
|
|
APIs and stored definitions have changed across releases; a local SDK example
|
|
alone does not establish compatibility with an older server. Before upgrading
|
|
an existing pipeline, confirm the supported refresh options and whether its
|
|
views need to be recreated with your administrator.
|
|
|
|
Stable row IDs and feature configuration for Enterprise sources are managed on
|
|
the server. The source-table storage option in the local example does not
|
|
configure the Enterprise deployment, and Python SDK 0.40.0's `create_table` on a
|
|
`db://` connection does not accept `storage_options`: leave it out, and confirm
|
|
with your administrator that the deployment gives new tables stable row IDs.
|
|
|
|
`list_materialized_views` requests the server's view listing. On a local
|
|
database it reads table schemas to find views.
|
|
|
|
## Working with the view {#working-with-the-view}
|
|
|
|
`view.table` is the underlying table, for querying, indexing, or search.
|
|
`view.definition` is the recorded query: the select list, predicate, and
|
|
limit. Use it to inspect the query without inferring it from the rows. Both are properties; on an async view, the definition is read
|
|
with `await view.definition()`.
|
|
|
|
An existing view is reopened with `db.open_materialized_view(name)`.
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
|
|
Derive a column instead of a whole table.
|
|
</Card>
|
|
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
|
|
Inspect the table history and read a particular version.
|
|
</Card>
|
|
</CardGroup>
|