mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-14 07:42:28 +00:00
docs: qualify what deployments do with materialized views
The SDK now sends CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW to a db:// deployment as SQL, and refuses source_version on a remote refresh. Current Sophon source accepts both statements, with CASCADE and WITH (full = true), but only on deployments that enable materialized views, and whether a refresh rebuilds the view or reconciles changes depends on configuration. The pages said there is no CREATE MATERIALIZED VIEW statement, that refreshes are incremental by default and that CASCADE is not implemented. The refresh, management and SQL reference pages now describe what the SDK sends, say that materialized views are experimental and must be enabled on a deployment, and leave refresh behaviour, CASCADE and the job type to the deployment's version and configuration. The SQL and management summaries list CREATE MATERIALIZED VIEW. No deployment was exercised.
This commit is contained in:
Brendan Clement
@@ -73,23 +73,22 @@ argument; every refresh is a full one.
|
||||
|
||||
<Badge color="red">Enterprise</Badge>
|
||||
|
||||
`create_materialized_view` works on a `db://` connection as well as a local one.
|
||||
What changes is where the refresh runs -- across the cluster rather than in your
|
||||
process -- and that SQL can drive it:
|
||||
`create_materialized_view` and `refresh` also work on a `db://` connection.
|
||||
There the SDK sends them to the deployment as SQL -- `CREATE MATERIALIZED VIEW`
|
||||
and `REFRESH MATERIALIZED VIEW` -- and the work runs on the cluster rather than
|
||||
in your process. `source_version` is for local views only; the SDK refuses it
|
||||
on a remote refresh.
|
||||
|
||||
```sql
|
||||
REFRESH MATERIALIZED VIEW chunks
|
||||
```
|
||||
Materialized views are experimental on a deployment, which has to have them
|
||||
enabled. How its refresh works -- rebuilding the whole view, as above, or
|
||||
reconciling what changed in the source -- depends on its LanceDB Enterprise
|
||||
version and configuration, and so do the options
|
||||
[`REFRESH MATERIALIZED VIEW`](/sql/dml/refresh#refresh-materialized-view)
|
||||
accepts.
|
||||
|
||||
`WITH (full = true)` recomputes every row instead of reconciling what changed
|
||||
in the source since the last refresh. `SHOW MATERIALIZED VIEWS` lists them, which is
|
||||
cheaper than `list_materialized_views` -- that method finds views by reading
|
||||
every table's schema, so it costs one open per table.
|
||||
|
||||
<Note>
|
||||
`CASCADE` is accepted by the parser but not yet implemented. A view built on
|
||||
another view is refreshed by refreshing each in dependency order.
|
||||
</Note>
|
||||
`SHOW MATERIALIZED VIEWS` lists them, which is cheaper than
|
||||
`list_materialized_views` -- that method finds views by reading every table's
|
||||
schema, so it costs one open per table.
|
||||
|
||||
## Working with the view {#working-with-the-view}
|
||||
|
||||
|
||||
@@ -52,7 +52,7 @@ See [Tables](/management/table) for creating one and shaping its schema.
|
||||
|
||||
| | Python | TypeScript | SQL |
|
||||
| --- | --- | --- | --- |
|
||||
| create | `create_materialized_view` | `createMaterializedView` | -- |
|
||||
| create | `create_materialized_view` | `createMaterializedView` | `CREATE MATERIALIZED VIEW` |
|
||||
| open | `open_materialized_view` | `openMaterializedView` | -- |
|
||||
| list | `list_materialized_views` | `listMaterializedViews` | `SHOW MATERIALIZED VIEWS` |
|
||||
| refresh | `refresh` | `refresh` | `REFRESH MATERIALIZED VIEW` |
|
||||
|
||||
@@ -49,7 +49,7 @@ separate definition step.
|
||||
|
||||
| | Python | TypeScript | SQL |
|
||||
| --- | --- | --- | --- |
|
||||
| create | `create_materialized_view` | `createMaterializedView` | -- |
|
||||
| create | `create_materialized_view` | `createMaterializedView` | `CREATE MATERIALIZED VIEW` |
|
||||
| open | `open_materialized_view` | `openMaterializedView` | -- |
|
||||
| list | `list_materialized_views` | `listMaterializedViews` | `SHOW MATERIALIZED VIEWS` |
|
||||
| refresh | `refresh` | `refresh` | `REFRESH MATERIALIZED VIEW` |
|
||||
@@ -76,8 +76,11 @@ rows. Both are properties; on an async view, the definition is read with
|
||||
## Against a deployment {#deployment}
|
||||
|
||||
<Badge color="red">Enterprise</Badge> `create_materialized_view` works on a
|
||||
`db://` connection as well as a local one. What changes is that a refresh runs
|
||||
across the cluster rather than in your process, and SQL can drive it.
|
||||
`db://` connection as well as a local one. There the SDK creates and refreshes
|
||||
the view through SQL, and the work runs on the cluster rather than in your
|
||||
process. Materialized views are experimental on a deployment and have to be
|
||||
enabled there; see
|
||||
[Against a deployment](/feature-engineering/materialized-views#availability).
|
||||
|
||||
## What to read next {#next}
|
||||
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
title: "Materialized View"
|
||||
sidebarTitle: "Materialized View"
|
||||
description: "SHOW MATERIALIZED VIEWS — creation and refresh are SDK and REFRESH statements, not CREATE."
|
||||
description: "CREATE MATERIALIZED VIEW and SHOW MATERIALIZED VIEWS; refreshing is a REFRESH statement."
|
||||
icon: table
|
||||
keywords: ["materialized view", "show materialized views", "sql"]
|
||||
keywords: ["materialized view", "create materialized view", "show materialized views", "sql"]
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
|
||||
@@ -12,13 +12,22 @@ A materialized view stores a query's result as rows, kept current by a refresh
|
||||
-- the counterpart to a [view](/sql/ddl/view), which stores only the query.
|
||||
|
||||
<Note>
|
||||
There is no `CREATE MATERIALIZED VIEW` statement. Creation is
|
||||
[`create_materialized_view`](/management/materialized-view#create) in the SDK,
|
||||
since it needs SDK-level source-table validation (stable row ids) that a bare
|
||||
SQL statement has nowhere to check. Once created, it is entirely a SQL object:
|
||||
queried like a table, refreshed with SQL, listed with SQL.
|
||||
Materialized views are experimental on a deployment, which has to have them
|
||||
enabled. Which statements and query shapes a deployment accepts, and how its
|
||||
refresh works, depend on its LanceDB Enterprise version and configuration.
|
||||
</Note>
|
||||
|
||||
## CREATE MATERIALIZED VIEW {#create}
|
||||
|
||||
```sql
|
||||
CREATE MATERIALIZED VIEW <name> AS SELECT ...
|
||||
```
|
||||
|
||||
On a `db://` connection, the SDK's
|
||||
[`create_materialized_view`](/management/materialized-view#create) sends this
|
||||
statement. Once created, a view is queried like a table, refreshed with SQL and
|
||||
listed with SQL.
|
||||
|
||||
## SHOW MATERIALIZED VIEWS {#show}
|
||||
|
||||
```sql
|
||||
@@ -37,7 +46,7 @@ REFRESH MATERIALIZED VIEW chunks
|
||||
```
|
||||
|
||||
See [REFRESH](/sql/dml/refresh#refresh-materialized-view) for the full syntax,
|
||||
including `full = true` and what `CASCADE` does not yet do.
|
||||
including `CASCADE` and `full = true`.
|
||||
|
||||
## Dropping one {#drop}
|
||||
|
||||
@@ -46,17 +55,18 @@ A materialized view **is** a table, so it is dropped with
|
||||
stored rows, unlike dropping a [view](/sql/ddl/view#drop).
|
||||
|
||||
<Note>
|
||||
**Enterprise job type:** `mat_view_refresh` for the refresh; `drop_table` for
|
||||
dropping it, same as any table.
|
||||
**Enterprise job type:** `mat_view_refresh` for a refresh, or `sql` on a
|
||||
deployment that rebuilds views through SQL; `drop_table` for dropping one, same
|
||||
as any table.
|
||||
</Note>
|
||||
|
||||
## What to read next {#next}
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Materialized View" icon="table" href="/management/materialized-view">
|
||||
Creating one from the SDK, and the stable-row-ids requirement.
|
||||
Creating one from the SDK.
|
||||
</Card>
|
||||
<Card title="Refresh a materialized view" icon="arrows-rotate" href="/feature-engineering/materialized-views">
|
||||
Incremental refreshes, and what they cost.
|
||||
What a refresh recomputes, locally and on a deployment.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
@@ -61,7 +61,7 @@ memory and GPU options there.
|
||||
## REFRESH MATERIALIZED VIEW {#refresh-materialized-view}
|
||||
|
||||
```sql
|
||||
REFRESH MATERIALIZED VIEW <name> [WITH (<option> = <value>)]
|
||||
REFRESH MATERIALIZED VIEW <name> [CASCADE] [WITH (<option> = <value>)]
|
||||
```
|
||||
|
||||
```sql
|
||||
@@ -69,13 +69,15 @@ REFRESH MATERIALIZED VIEW chunks
|
||||
REFRESH MATERIALIZED VIEW chunks WITH (full = true)
|
||||
```
|
||||
|
||||
By default a refresh is **incremental** -- it processes what the source has
|
||||
added since the last one. `full = true` recomputes every row, which is what you
|
||||
want after changing something the view's rows depend on.
|
||||
`full = true` asks for every row to be recomputed. Without it, whether a
|
||||
refresh rebuilds the whole view -- as every local SDK refresh does -- or
|
||||
reconciles only what changed in the source depends on the deployment's LanceDB
|
||||
Enterprise version and configuration.
|
||||
|
||||
<Warning>
|
||||
`CASCADE` parses but is **not implemented**. A view built on another view is
|
||||
refreshed by refreshing each in dependency order yourself.
|
||||
`CASCADE` asks for the views this one is built on to be refreshed first, in
|
||||
dependency order. Whether a deployment carries it out depends on its version;
|
||||
where it does not, refresh each view yourself, its sources first.
|
||||
</Warning>
|
||||
|
||||
## Watching a refresh {#watching}
|
||||
|
||||
@@ -99,7 +99,7 @@ insert -- see [What SQL cannot do](#cannot).
|
||||
| [Namespace](/sql/ddl/namespace) | `CREATE SCHEMA`, `DROP SCHEMA` |
|
||||
| [Table](/sql/ddl/table) | `CREATE`, `DESCRIBE`, `ALTER`, `CLONE`, `DROP TABLE` |
|
||||
| [View](/sql/ddl/view) | `CREATE VIEW`, `DROP VIEW` |
|
||||
| [Materialized view](/sql/ddl/materialized-view) | `SHOW MATERIALIZED VIEWS` |
|
||||
| [Materialized view](/sql/ddl/materialized-view) | `CREATE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS` |
|
||||
| [Index](/sql/ddl/index) | `CREATE INDEX`, `DROP INDEX` |
|
||||
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
|
||||
| [Job](/sql/ddl/job) | `SHOW JOBS`, `SHOW ERRORS`, `PAUSE`/`RESUME`/`CANCEL JOB` |
|
||||
|
||||
Reference in New Issue
Block a user