Files
lancedb/docs/web/feature-engineering/materialized-views.mdx
T
Jack YeandClaude Sonnet 5 de95ed3f6d docs: overhaul curation, feature engineering, security and deployment guides (#4748)
A broad documentation pass across the areas LanceDB Enterprise and
Embedded
diverge on most:

**Data Curation** -- GPU Function execution, Slurm scheduling (and what
Slurm is, for readers new to it), blob late materialization as its own
page, joins, aggregation/profiling, sampling, near-duplicate detection,
chunking, distributed shuffle, and IVF-partition clustering.

**Feature Engineering** -- computed columns and materialized views,
restructured to lead with the Function-based path and show the SQL form
side by side with Python throughout; a single Auto Refresh page instead
of
duplicating scheduling details on every refresh page; adaptive
execution;
checkpointing; GPU and Slurm placement for refresh workers; a Training
Cache page for the PyTorch data loader's automatic cache benefit.

**Security & Governance** -- an Audit Log reference (schema, access,
grants), Secrets promoted to an ordinary catalog object with their own
SQL
DDL and a vending/rotation guide, and privilege-table fixes verified
against the authorization source.

**Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry
moved
into a real Deployment area; Automated Cleanup folded into Intelligent
Auto-Maintenance (every other maintenance decision already lived there);
RTE Compaction and Advanced Conflict Resolution documented in depth.

**Landing page** -- the Enterprise feature comparison table reorganized
(several redundant or stale rows removed, several new areas added), and
the
"why Enterprise" bullets replaced with ones tied to a concrete page
(Performance, Scale, Fault tolerance, Multicloud deployment, Simplify
operations) instead of vague promises.

Plus a full pass addressing lancedb-gatekeeper's review: a real
test-safety
bug (a quickstart test could silently overwrite and delete a
deployment's
own `frames` table), a fabricated SQL statement with no backing
implementation, an actual dedup-guarantee bug in `take_row_ids`, and
dozens
of documentation-accuracy fixes verified against source rather than
taken
on faith -- each either fixed, or left with an explicit note on why it
needs a feature or deployment owner's sign-off instead of being guessed
at.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-03 16:17:24 -08:00

187 lines
6.2 KiB
Plaintext

---
title: "Create a materialized view"
sidebarTitle: "Create Materialized View"
description: "Store the result of a query as a table, ready to query, index and search."
icon: table
keywords: ["materialized view", "create_materialized_view"]
availability:
embedded: unavailable
enterprise: available
---
A materialized view stores the result of a query as a table. Use it to keep a
filtered or transformed dataset that you can query, index and search
repeatedly, instead of recomputing the query every time you read it.
<Note>
In SQL this is [`CREATE MATERIALIZED VIEW`](/sql/ddl/materialized-view#create).
</Note>
## Create a view {#create}
A materialized view is defined by a SQL query, so it is created with one.
From Python, run the statement on an [Enterprise connection](/basics/catalog#enterprise)
with [`execute_query`](/basics/sql#sql-from-python); the SQL is the same
either way:
<CodeGroup>
```python Python icon="python"
db.execute_query("""
CREATE MATERIALIZED VIEW frames_captioned AS
SELECT frame_id, video_id, ts, caption
FROM frames
WHERE selected = true
""").read_all()
view = db.open_materialized_view("frames_captioned")
```
```sql SQL icon="database"
CREATE MATERIALIZED VIEW frames_captioned AS
SELECT frame_id, video_id, ts, caption
FROM frames
WHERE selected = true
```
</CodeGroup>
`execute_query` returns once the result stream is readable, not once the
statement has finished, so `.read_all()` waits out the full creation before
the next line opens it.
The query defines the view: here, the `frames` table's `frame_id`,
`video_id`, `ts` and `caption` columns, filtered to the frames selected for
captioning. There is no separate, more limited form for defining one --
joins and [Functions](#functions) in the select list work the same way any
other `SELECT` does, within the query shapes your deployment's LanceDB
Enterprise version and configuration accept; see the
[SQL reference](/sql/ddl/materialized-view#create).
Creating the view computes its rows, so it is ready to read immediately.
[`db.open_materialized_view`](/management/materialized-view#manage) gets a
handle back for reading, refreshing or inspecting it from Python; the view
itself is also an ordinary table, queryable by name like any other.
Your administrator manages stable row IDs and other source-table
requirements for materialized views on the server; confirm support for your
source table's shape before creating a view over it.
## Selecting a Function's output {#functions}
<Badge color="red">Enterprise</Badge>
A `SELECT` expression is SQL text, so it can call a registered
[Function](/curation/create-function) the same way a computed column's
expression can. Continuing the
[`caption` Function](/curation/create-function#chain) registered in Create a
Function, against a `frames` table that already has a `jpeg` column:
<CodeGroup>
```python Python icon="python"
db.execute_query("""
CREATE MATERIALIZED VIEW frames_captioned AS
SELECT frame_id, jpeg, caption(jpeg) AS caption
FROM frames
WHERE selected = true
""").read_all()
```
```sql SQL icon="database"
CREATE MATERIALIZED VIEW frames_captioned AS
SELECT frame_id, jpeg, caption(jpeg) AS caption
FROM frames
WHERE selected = true
```
</CodeGroup>
A query that joins tables works the same way, with no separate syntax for
it. Continuing the join and chained Functions from
[Create a Function](/curation/create-function#chain), producing `jpeg` from
a separate `videos` table and captioning it in the same statement:
<CodeGroup>
```python Python icon="python"
db.execute_query("""
CREATE MATERIALIZED VIEW frames_captioned
WITH (auto_refresh_enabled = false, auto_refresh_interval_seconds = 5)
AS
SELECT
f.frame_id,
extract_frame(v.video, f.ts) AS jpeg,
caption(extract_frame(v.video, f.ts)) AS caption
FROM videos v
JOIN frames f ON v.id = f.video_id
WHERE f.selected = true
""").read_all()
```
```sql SQL icon="database"
CREATE MATERIALIZED VIEW frames_captioned
WITH (auto_refresh_enabled = false, auto_refresh_interval_seconds = 5)
AS
SELECT
f.frame_id,
extract_frame(v.video, f.ts) AS jpeg,
caption(extract_frame(v.video, f.ts)) AS caption
FROM videos v
JOIN frames f ON v.id = f.video_id
WHERE f.selected = true
```
</CodeGroup>
See [Joins](/curation/join) for the join itself, and
[Auto Refresh](/enterprise/auto-refresh) for what `WITH` sets here.
## Automatic refresh at creation {#auto-refresh}
Set `auto_refresh_enabled` and `auto_refresh_interval_seconds` in the `WITH`
clause to schedule refreshes instead of calling
[`refresh`](/feature-engineering/refresh-materialized-view) yourself:
<CodeGroup>
```python Python icon="python"
db.execute_query("""
CREATE MATERIALIZED VIEW frames_captioned
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300)
AS
SELECT frame_id, video_id, ts, caption
FROM frames
WHERE selected = true
""").read_all()
```
```sql SQL icon="database"
CREATE MATERIALIZED VIEW frames_captioned
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300)
AS
SELECT frame_id, video_id, ts, caption
FROM frames
WHERE selected = true
```
</CodeGroup>
Omit `auto_refresh_enabled` to let the server decide whether to enable it,
and omit `auto_refresh_interval_seconds` to let the server decide the
pacing. See [Auto Refresh](/enterprise/auto-refresh) for how scheduling
behaves once the view exists.
## Working with the view {#working-with-the-view}
`view.table` is the underlying table, for querying, indexing, or search.
`view.definition` has a single field, `query` -- the recorded query, in the
canonical spelling the server stores. Use it to inspect the query without
inferring it from the rows. Both are properties; on an async view, the
definition is read with `await view.definition()`.
Creation, refresh and drop also have async, job-returning SDK methods; see
[Materialized view](/management/materialized-view#manage).
## Related guides {#next}
<CardGroup cols={2}>
<Card title="Refresh a materialized view" icon="arrows-rotate" href="/feature-engineering/refresh-materialized-view">
Bring the view up to date with its source.
</Card>
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
Inspect the table history and read a particular version.
</Card>
</CardGroup>