mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
126 lines
4.8 KiB
Plaintext
126 lines
4.8 KiB
Plaintext
---
|
|
title: "Materialized view"
|
|
sidebarTitle: "Materialized View"
|
|
description: "Create, open, list and drop a table whose contents are defined by a query."
|
|
icon: table
|
|
keywords: ["materialized view", "create_materialized_view", "catalog", "ddl"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
A materialized view is a table whose contents are defined by a query over
|
|
another table. The query is recorded in the view's schema, and a refresh
|
|
brings its stored rows up to date with the source.
|
|
|
|
The result is an ordinary table: queryable, indexable, searchable, and present
|
|
in `table_names()`. Nothing about reading it is special.
|
|
|
|
This page covers creating and managing one. Keeping it current is
|
|
[Refresh a materialized view](/feature-engineering/refresh-materialized-view).
|
|
|
|
<Note>
|
|
In SQL, see [`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh-materialized-view) and [`SHOW MATERIALIZED VIEWS`](/sql/ddl/materialized-view#show).
|
|
</Note>
|
|
|
|
## Create a materialized view {#create}
|
|
|
|
A materialized view is defined by a SQL query:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.execute_query("""
|
|
CREATE MATERIALIZED VIEW frames_captioned AS
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
""").read_all()
|
|
view = db.open_materialized_view("frames_captioned")
|
|
```
|
|
|
|
```sql SQL icon="database"
|
|
CREATE MATERIALIZED VIEW frames_captioned AS
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
```
|
|
</CodeGroup>
|
|
|
|
See [Create a materialized view](/feature-engineering/materialized-views#create)
|
|
for the full walkthrough, including joins and Functions in the query.
|
|
|
|
## Open, list and drop views {#manage}
|
|
|
|
| | Python | TypeScript | SQL |
|
|
| --- | --- | --- | --- |
|
|
| create | `create_materialized_view` | `createMaterializedView` | `CREATE MATERIALIZED VIEW` |
|
|
| open | `open_materialized_view` | `openMaterializedView` | -- |
|
|
| list | `list_materialized_views` | `listMaterializedViews` | `SHOW MATERIALIZED VIEWS` |
|
|
| refresh | `refresh` | `refresh` | `REFRESH MATERIALIZED VIEW` |
|
|
| drop | `drop_materialized_view` | `dropMaterializedView` | `DROP MATERIALIZED VIEW` |
|
|
|
|
<Badge color="red">Enterprise</Badge> Creating, refreshing and dropping a
|
|
materialized view each have an async, job-returning counterpart --
|
|
`create_materialized_view_async`, `view.refresh_async()`,
|
|
`drop_materialized_view_async` -- the same pattern as
|
|
[`refresh_column_async`](/feature-engineering/refresh-column#async): each
|
|
returns a [job](/basics/jobs) you `wait()` on instead of blocking, with the
|
|
same arguments as its synchronous counterpart, including
|
|
`auto_refresh_enabled` and `auto_refresh_interval_seconds`; see
|
|
[Auto Refresh](/enterprise/auto-refresh).
|
|
|
|
<Note>
|
|
On a local database, `list_materialized_views` reads table schemas to find
|
|
views. On Enterprise, it asks the server for the view listing, as
|
|
`SHOW MATERIALIZED VIEWS` does.
|
|
</Note>
|
|
|
|
A view is dropped with `drop_materialized_view`, a dedicated operation, even
|
|
though a materialized view **is** a table underneath. That also means
|
|
dropping one removes stored rows, unlike dropping a
|
|
[view](/management/view), which removes only a saved query.
|
|
|
|
## Read the view and its definition {#working}
|
|
|
|
`view.table` is the underlying table, for querying, indexing, or search.
|
|
`view.definition` has a single field, `query` -- the defining query, in the
|
|
canonical spelling the server stores. Both are properties; on an async view, the definition is read with
|
|
`await view.definition()`.
|
|
|
|
## Enterprise compatibility {#deployment}
|
|
|
|
Enterprise supports creating and refreshing materialized views on remote
|
|
compute. Use the SDK version supported by your deployment, or the
|
|
[SQL interface](/sql/ddl/materialized-view).
|
|
|
|
## Privileges {#privileges}
|
|
|
|
A materialized view **is** a table (see [Dropping one](/sql/ddl/materialized-view#drop)),
|
|
granted through the REST ACL API or SQL `GRANT`/`REVOKE` with the same
|
|
privilege names as a table, but two of them mean something different here:
|
|
the server refuses a direct `INSERT`, `UPDATE` or `DELETE` on a materialized
|
|
view's rows -- only a refresh writes them. `UPDATE` is instead what
|
|
authorizes calling refresh, and `OWNERSHIP` is what authorizes `ALTER`
|
|
(auto-refresh settings):
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
GRANT SELECT, UPDATE ON MATERIALIZED VIEW daily_rollup TO ROLE analysts
|
|
```
|
|
</CodeGroup>
|
|
|
|
That grants `analysts` read access and the ability to refresh `daily_rollup`,
|
|
not row-level write access. See [Privileges](/governance/privileges#sql) for
|
|
the full statement syntax and every target it reaches.
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a materialized view" icon="arrows-rotate" href="/feature-engineering/refresh-materialized-view">
|
|
How a refresh works, and what it writes.
|
|
</Card>
|
|
<Card title="Views" icon="eye" href="/management/view">
|
|
The non-materialized counterpart, defined in SQL.
|
|
</Card>
|
|
</CardGroup>
|