mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 22:26:53 +00:00
Python materialized-view creation now accepts a complete defining SQL
query across sync, async, remote, and namespace connections, including
creation jobs:
```python
view = db.create_materialized_view(
"events_by_kind",
query="SELECT kind, count(*) AS n FROM events GROUP BY kind",
)
```
`query` is mutually exclusive with `source`, `select`, `where`, and
`limit`. Existing source-based calls remain supported.
Adds Rust `Connection::create_materialized_view_from_query`: embedded
connections plan through `MaterializedViewDefinition::from_sql`, while
remote connections submit the full query for server-side parsing.
Includes regression coverage for grouping, UNNEST, query forwarding,
creation jobs, and argument validation.
189 lines
6.3 KiB
Plaintext
189 lines
6.3 KiB
Plaintext
---
|
|
title: "Create a materialized view"
|
|
sidebarTitle: "Create Materialized View"
|
|
description: "Store the result of a query as a table, ready to query, index and search."
|
|
icon: table
|
|
keywords: ["materialized view", "create_materialized_view"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
A materialized view stores the result of a query as a table. Use it to keep a
|
|
filtered or transformed dataset that you can query, index and search
|
|
repeatedly, instead of recomputing the query every time you read it.
|
|
|
|
<Note>
|
|
In SQL this is [`CREATE MATERIALIZED VIEW`](/sql/ddl/materialized-view#create).
|
|
</Note>
|
|
|
|
## Create a view {#create}
|
|
|
|
A materialized view is defined by a SQL query, so it is created with one.
|
|
From Python, pass the defining query to `create_materialized_view` on an
|
|
[Enterprise connection](/basics/catalog#enterprise):
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
view = db.create_materialized_view(
|
|
"frames_captioned",
|
|
query="""
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
""",
|
|
)
|
|
```
|
|
|
|
```sql SQL icon="database"
|
|
CREATE MATERIALIZED VIEW frames_captioned AS
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
```
|
|
</CodeGroup>
|
|
|
|
`create_materialized_view` waits for creation and initial population, then
|
|
returns the view handle. Use `create_materialized_view_async` to submit a job
|
|
and wait for it separately. Both methods accept `query=` on synchronous and
|
|
asynchronous connections. `query` cannot be combined with `source`, `select`,
|
|
`where`, or `limit`; put those clauses in the query itself.
|
|
|
|
The query defines the view: here, the `frames` table's `frame_id`,
|
|
`video_id`, `ts` and `caption` columns, filtered to the frames selected for
|
|
captioning. There is no separate, more limited form for defining one --
|
|
joins and [Functions](#functions) in the select list work the same way any
|
|
other `SELECT` does, within the query shapes your deployment's LanceDB
|
|
Enterprise version and configuration accept; see the
|
|
[SQL reference](/sql/ddl/materialized-view#create).
|
|
|
|
Creating the view computes its rows, so it is ready to read immediately.
|
|
[`db.open_materialized_view`](/management/materialized-view#manage) gets a
|
|
handle back for reading, refreshing or inspecting it from Python; the view
|
|
itself is also an ordinary table, queryable by name like any other.
|
|
|
|
Your administrator manages stable row IDs and other source-table
|
|
requirements for materialized views on the server; confirm support for your
|
|
source table's shape before creating a view over it.
|
|
|
|
## Selecting a Function's output {#functions}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
A `SELECT` expression is SQL text, so it can call a registered
|
|
[Function](/curation/create-function) the same way a computed column's
|
|
expression can. Continuing the
|
|
[`caption` Function](/curation/create-function#chain) registered in Create a
|
|
Function, against a `frames` table that already has a `jpeg` column:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.execute_query("""
|
|
CREATE MATERIALIZED VIEW frames_captioned AS
|
|
SELECT frame_id, jpeg, caption(jpeg) AS caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
""").read_all()
|
|
```
|
|
|
|
```sql SQL icon="database"
|
|
CREATE MATERIALIZED VIEW frames_captioned AS
|
|
SELECT frame_id, jpeg, caption(jpeg) AS caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
```
|
|
</CodeGroup>
|
|
|
|
A query that joins tables works the same way, with no separate syntax for
|
|
it. Continuing the join and chained Functions from
|
|
[Create a Function](/curation/create-function#chain), producing `jpeg` from
|
|
a separate `videos` table and captioning it in the same statement:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.execute_query("""
|
|
CREATE MATERIALIZED VIEW frames_captioned
|
|
WITH (auto_refresh_enabled = false, auto_refresh_interval_seconds = 5)
|
|
AS
|
|
SELECT
|
|
f.frame_id,
|
|
extract_frame(v.video, f.ts) AS jpeg,
|
|
caption(extract_frame(v.video, f.ts)) AS caption
|
|
FROM videos v
|
|
JOIN frames f ON v.id = f.video_id
|
|
WHERE f.selected = true
|
|
""").read_all()
|
|
```
|
|
|
|
```sql SQL icon="database"
|
|
CREATE MATERIALIZED VIEW frames_captioned
|
|
WITH (auto_refresh_enabled = false, auto_refresh_interval_seconds = 5)
|
|
AS
|
|
SELECT
|
|
f.frame_id,
|
|
extract_frame(v.video, f.ts) AS jpeg,
|
|
caption(extract_frame(v.video, f.ts)) AS caption
|
|
FROM videos v
|
|
JOIN frames f ON v.id = f.video_id
|
|
WHERE f.selected = true
|
|
```
|
|
</CodeGroup>
|
|
|
|
See [Joins](/curation/join) for the join itself, and
|
|
[Auto Refresh](/enterprise/auto-refresh) for what `WITH` sets here.
|
|
|
|
## Automatic refresh at creation {#auto-refresh}
|
|
|
|
Set `auto_refresh_enabled` and `auto_refresh_interval_seconds` in the `WITH`
|
|
clause to schedule refreshes instead of calling
|
|
[`refresh`](/feature-engineering/refresh-materialized-view) yourself:
|
|
|
|
<CodeGroup>
|
|
```python Python icon="python"
|
|
db.execute_query("""
|
|
CREATE MATERIALIZED VIEW frames_captioned
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300)
|
|
AS
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
""").read_all()
|
|
```
|
|
|
|
```sql SQL icon="database"
|
|
CREATE MATERIALIZED VIEW frames_captioned
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300)
|
|
AS
|
|
SELECT frame_id, video_id, ts, caption
|
|
FROM frames
|
|
WHERE selected = true
|
|
```
|
|
</CodeGroup>
|
|
|
|
Omit `auto_refresh_enabled` to let the server decide whether to enable it,
|
|
and omit `auto_refresh_interval_seconds` to let the server decide the
|
|
pacing. See [Auto Refresh](/enterprise/auto-refresh) for how scheduling
|
|
behaves once the view exists.
|
|
|
|
## Working with the view {#working-with-the-view}
|
|
|
|
`view.table` is the underlying table, for querying, indexing, or search.
|
|
`view.definition` has a single field, `query` -- the recorded query, in the
|
|
canonical spelling the server stores. Use it to inspect the query without
|
|
inferring it from the rows. Both are properties; on an async view, the
|
|
definition is read with `await view.definition()`.
|
|
|
|
Creation, refresh and drop also have async, job-returning SDK methods; see
|
|
[Materialized view](/management/materialized-view#manage).
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a materialized view" icon="arrows-rotate" href="/feature-engineering/refresh-materialized-view">
|
|
Bring the view up to date with its source.
|
|
</Card>
|
|
<Card title="Versioning" icon="clock-rotate-left" href="/management/table#versioning">
|
|
Inspect the table history and read a particular version.
|
|
</Card>
|
|
</CardGroup>
|