rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-19 18:22:28 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(remote): support materialized view APIs (#4180)

Browse Source 
## Summary

Align the experimental materialized-view HTTP transport with the
equivalent Table API shape and add remote materialized-view support
across Rust, Python, and TypeScript. This is an intentional breaking
change to the experimental materialized-view surface.

Materialized-view creation performs an initial refresh by default. The
create endpoint returns `202 Accepted` with `{ "job_id": "..." }`;
blocking SDK creation waits for that job before returning a populated
view. `with_no_data` / `withNoData` explicitly creates only the
definition and empty backing table.

## Route comparison

| Operation | Materialized-view API | Equivalent Table API |
| --- | --- | --- |
| Create | `POST /v1/materialized_view/{id}/create` | `POST
/v1/table/{id}/create` |
| Describe/open | `POST /v1/materialized_view/{id}/describe` | `POST
/v1/table/{id}/describe` |
| List | `GET /v1/namespace/{id}/materialized_view/list` | `GET
/v1/namespace/{id}/table/list` |
| Refresh | `POST /v1/materialized_view/{id}/refresh` | asynchronous
Table mutation pattern |
| Drop | `POST /v1/materialized_view/{id}/drop` | `POST
/v1/table/{id}/drop` |

Create, describe, refresh, and drop identify the target in the singular
item path instead of duplicating it in the request body. Create and drop
require `202 Accepted` with a valid job ID. List is a namespace-scoped
GET with opaque pagination tokens. The Rust list API now returns view
names, matching Table listing and the existing Python and TypeScript
APIs.

## Python API changes

| Operation | Synchronous API | Asynchronous API | Table/job pattern |
| --- | --- | --- | --- |
| Create and wait | `DBConnection.create_materialized_view(...)` |
`await AsyncConnection.create_materialized_view(...)` | Returns a
materialized-view handle after its initial-population job finishes |
| Submit create | `DBConnection.create_materialized_view_async(...) ->
Job[None]` | `await AsyncConnection.create_materialized_view_async(...)
-> AsyncJob[None]` | Matches job-returning Table mutations such as
`create_index_async` |
| Open | `DBConnection.open_materialized_view(...)` | `await
AsyncConnection.open_materialized_view(...)` | Opens the backing Table
plus its definition |
| List | `DBConnection.list_materialized_views()` | `await
AsyncConnection.list_materialized_views()` | Returns names like Table
listing |
| Refresh and wait | `MaterializedView.refresh(...)` | `await
AsyncMaterializedView.refresh(...)` | Returns the typed refresh result
after the job finishes |
| Submit refresh | `MaterializedView.refresh_async(...) ->
Job[RefreshMaterializedViewResult]` | `await
AsyncMaterializedView.refresh_async(...) ->
AsyncJob[RefreshMaterializedViewResult]` | Matches
`Table.refresh_column_async`; remote job handles expose the server job
ID |
| Drop | `DBConnection.drop_materialized_view(...)` | `await
AsyncConnection.drop_materialized_view(...)` | Matches blocking
`drop_table` |
| Submit drop | `DBConnection.drop_materialized_view_async(...) ->
Job[None]` | `await AsyncConnection.drop_materialized_view_async(...) ->
AsyncJob[None]` | Matches `drop_table_async`; remote handles expose the
server cleanup job ID |

The materialized-view handle exposes its backing Table through `.table`,
so normal Table query, search, and index APIs apply. Definition lookup
and refresh are backend-aware rather than depending on local schema
metadata. TypeScript exposes the equivalent blocking/job drop pair as
`dropMaterializedView` and `dropMaterializedViewAsync`.
This commit is contained in:
Jack Ye
2026-09-15 12:13:39 -07:00
committed by GitHub
parent 2d4622491e
commit f12996557f
24 changed files with 1820 additions and 232 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/src/connection.rs
+67 -3
View File
@@ -381,7 +381,7 @@ impl Connection {
})
}
#[pyo3(signature = (name, source, projections=None, filter=None, limit=None))]
#[pyo3(signature = (name, source, projections=None, filter=None, limit=None, with_no_data=false))]
pub fn create_materialized_view(
self_: PyRef<'_, Self>,
name: String,
@@ -389,6 +389,7 @@ impl Connection {
projections: Option<Vec<(String, String)>>,
filter: Option<String>,
limit: Option<u64>,
with_no_data: bool,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
@@ -402,16 +403,79 @@ impl Connection {
if let Some(limit) = limit {
builder = builder.limit(limit);
}
let view = builder.execute().await.infer_error()?;
builder = builder.with_no_data(with_no_data);
let view = Box::pin(builder.execute()).await.infer_error()?;
Ok(Table::new(view.table().clone()))
})
}
#[pyo3(signature = (name, source, projections=None, filter=None, limit=None, with_no_data=false))]
pub fn create_materialized_view_async(
self_: PyRef<'_, Self>,
name: String,
source: String,
projections: Option<Vec<(String, String)>>,
filter: Option<String>,
limit: Option<u64>,
with_no_data: bool,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
let mut builder = inner.create_materialized_view(name, source);
if let Some(projections) = projections {
builder = builder.select(projections);
}
if let Some(filter) = filter {
builder = builder.only_if(filter);
}
if let Some(limit) = limit {
builder = builder.limit(limit);
}
let job = Box::pin(builder.with_no_data(with_no_data).execute_async())
.await
.infer_error()?;
Ok(crate::job::Job::new(job))
})
}
pub fn list_materialized_views(self_: PyRef<'_, Self>) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
future_into_py(self_.py(), async move {
let views = inner.list_materialized_views().await.infer_error()?;
Ok(views.into_iter().map(|view| view.name).collect::<Vec<_>>())
Ok(views)
})
}
#[pyo3(signature = (name, namespace_path=None))]
pub fn drop_materialized_view(
self_: PyRef<'_, Self>,
name: String,
namespace_path: Option<Vec<String>>,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
let namespace_path = namespace_path.unwrap_or_default();
future_into_py(self_.py(), async move {
inner
.drop_materialized_view(name, &namespace_path)
.await
.infer_error()
})
}
#[pyo3(signature = (name, namespace_path=None))]
pub fn drop_materialized_view_async(
self_: PyRef<'_, Self>,
name: String,
namespace_path: Option<Vec<String>>,
) -> PyResult<Bound<'_, PyAny>> {
let inner = self_.get_inner()?.clone();
let namespace_path = namespace_path.unwrap_or_default();
future_into_py(self_.py(), async move {
inner
.drop_materialized_view_async(name, &namespace_path)
.await
.infer_error()
.map(crate::job::Job::new)
})
}