mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 23:32:27 +00:00
Jack Ye
6e1473278d
Functions and materialized views are objects in the Lance Multimodal Lakehouse, alongside tables and indexes — created, versioned and refreshed like anything else in it. The pages introduced them as client methods, which undersold what they are and left the capability sounding like a helper rather than part of the storage layer. The materialized view page also framed Enterprise availability around which client you are holding. A reader does not care about that; they care whether views exist where their data lives. They do, refreshed across the cluster, and the page now says so and points at the Enterprise documentation rather than implying the feature is missing.
113 lines
4.0 KiB
Plaintext
113 lines
4.0 KiB
Plaintext
---
|
|
title: "Materialized views"
|
|
sidebarTitle: "Materialized views"
|
|
description: "A lakehouse object: a table defined by a query over another table, kept current incrementally."
|
|
icon: table
|
|
keywords: ["materialized view", "create_materialized_view", "refresh", "incremental"]
|
|
---
|
|
|
|
A materialized view is a lakehouse object: a table whose contents are defined by
|
|
a query over another table. The query is recorded in the view's schema; a
|
|
refresh computes the rows.
|
|
|
|
The result is an ordinary table. It can be queried, indexed, and searched, and
|
|
it appears in `table_names()` — nothing about reading it is special.
|
|
|
|
## Create and refresh {#create-and-refresh}
|
|
|
|
```python
|
|
import lancedb
|
|
|
|
db = lancedb.connect(
|
|
"./.lancedb",
|
|
storage_options={"new_table_enable_stable_row_ids": "true"},
|
|
)
|
|
table = db.create_table("people", [{"name": "ada", "age": 36},
|
|
{"name": "kid", "age": 7}])
|
|
|
|
view = db.create_materialized_view(
|
|
"adults",
|
|
"people",
|
|
select=["name", ("shout", "upper(name)")],
|
|
where="age >= 18",
|
|
)
|
|
|
|
result = view.refresh()
|
|
result.rows_written
|
|
# 1
|
|
```
|
|
|
|
`select` takes column names, `(alias, expression)` pairs, or a dict of the same.
|
|
Omitting it selects every source column, expanded against the schema as it was
|
|
at creation time. `where` filters source rows; `limit` caps the view.
|
|
|
|
## The source needs stable row ids {#stable-row-ids}
|
|
|
|
<Warning>
|
|
The source table must have been created with stable row ids:
|
|
|
|
```python
|
|
db = lancedb.connect(uri, storage_options={"new_table_enable_stable_row_ids": "true"})
|
|
```
|
|
|
|
They are what keeps the view's provenance valid when the source is compacted,
|
|
and **they cannot be enabled after a table exists**. A table created without
|
|
them cannot later become the source of a view — it has to be recreated.
|
|
</Warning>
|
|
|
|
This is the one decision here that is hard to undo, so it is worth setting the
|
|
option on any table you might later want to derive from.
|
|
|
|
## Refreshes are incremental {#incremental}
|
|
|
|
`refresh()` reconciles the source's changes into the view — rows added, changed,
|
|
or removed since the last refresh — rather than recomputing from scratch. It
|
|
falls back to a full rebuild when the changes cannot be reconciled.
|
|
|
|
Two arguments override that:
|
|
|
|
- `full=True` forces a rebuild.
|
|
- `source_version=N` refreshes against that source version instead of the
|
|
latest, which is how you make a view reproduce a known point in time.
|
|
|
|
Concurrent refreshes of one view do not duplicate rows. If two plan the same
|
|
source rows, they conflict on commit and the loser raises rather than writing
|
|
them twice — so a retry loop around `refresh()` is safe.
|
|
|
|
## Creating one against a deployment {#availability}
|
|
|
|
<Warning>
|
|
`create_materialized_view` on this client builds a view over a **local**
|
|
database. Called on a remote `db://` connection it raises
|
|
`NotImplementedError`.
|
|
|
|
Views exist on Enterprise deployments too, where a refresh runs across the
|
|
cluster rather than in your process — but they are created through the
|
|
Enterprise feature engineering interface rather than this method. See the
|
|
Enterprise documentation if that is where your data lives.
|
|
</Warning>
|
|
|
|
The direction is the reverse of [functions](/feature-engineering/functions),
|
|
which register only against Cloud and Enterprise and raise locally. Worth
|
|
knowing before concluding either is unavailable.
|
|
|
|
## Working with the view {#working-with-the-view}
|
|
|
|
`view.table()` returns the underlying table for querying, indexing, or search.
|
|
`view.definition()` returns the recorded query — the select list, predicate, and
|
|
limit — which is how you check what a view actually contains without inferring
|
|
it from the rows.
|
|
|
|
An existing view is reopened with `db.open_materialized_view(name)`.
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Computed columns" icon="table-columns" href="/feature-engineering/computed-columns">
|
|
Derive a column instead of a whole table.
|
|
</Card>
|
|
<Card title="Versioning" icon="clock-rotate-left" href="/tables/versioning">
|
|
Every refresh commits a table version.
|
|
</Card>
|
|
</CardGroup>
|