fix: execute remote materialized view creation through SQL (#4665)

Remote materialized-view creation currently expects a REST response
containing a registry job ID, while refresh already submits SQL. Route
both through the SQL query backend and preserve Python's synchronous and
asynchronous return types.

### Query to Job mapping

`SqlClient::submit_as_job()` submits an `ExecuteQueryRequest` through
the same backend as `execute_query_async()`. Once submission succeeds,
it retains the returned `Query` and starts one client-side task that
consumes its result reader to completion. `Job::from_query_task()` wraps
that task, its cached outcome, a single ID and a cancellation callback.
Python exposes this as `Job[T]` or `AsyncJob[T]`.

The existing `job_type` property maps SQL-backed handles to the new
`"query"` kind. The shared `id` contains that query's UUID string, so
Python callers can inspect it with `db.describe_query(job.id)` on the
submitting connection. No separate `query_id`, backend enum or `type()`
API is added.

| Handle | `job_type` | `id` | Available lookup |
| --- | --- | --- | --- |
| SQL-backed | `"query"`, known at submission | Connection-scoped query
UUID string | `describe_query(job.id)` |
| Registry-backed | Existing server-reported kind, after `status()` or
`refresh()` | Registry job ID string | Registry job APIs, descriptions
and events |
| Local | `None` | `None` | The current in-process handle |

| Job concept | SQL-backed behavior |
| --- | --- |
| Submission | Await initial SQL submission before returning the handle.
Submission errors propagate immediately. |
| `status()` | Tracks the adapter task: running during result
consumption/finalization, finished after both succeed, failed if either
fails, or cancelled after cancellation. |
| `wait()` | Returns the operation's typed terminal result. One task
consumes the query reader once; repeated waits reuse its cached outcome
without resubmitting SQL or reopening the reader. |
| `cancel()` | Delegates to `Query.cancel()`. Rejected or unsupported
cancellation leaves the task running; an accepted request stops the
task. Cancelling a finished operation is a no-op. |
| Registry details | Query-backed handles expose no registry
description, event history, pause/resume or reattachment after a client
restart. `refresh()` updates cached lifecycle state. |

The Job tracks command completion rather than exposing SQL batches. A
wait timeout does not cancel the query. Query IDs remain scoped to the
submitting connection and cannot be used with registry APIs or as
durable resume tokens. Python `describe_query()` accepts both UUID
objects and the string `id` exposed by query-backed jobs. Rust, Python
and TypeScript expose the same classification through their existing
job-type accessors.

### Materialized-view operations

- **CREATE:** Render `CREATE MATERIALIZED VIEW` with the view namespace
and optional auto-refresh settings. After command completion, resolve
the job to `None`. Synchronous creation waits on the job, then opens and
returns the materialized view.
- **REFRESH:** Submit `REFRESH MATERIALIZED VIEW` through the same
adapter. After command completion, read the refreshed table's version
and row count to construct the existing typed refresh result.
Synchronous refresh waits on the same path.

Remote creation requires the SQL service. Execution placement remains a
server decision. REST endpoints are unchanged. The job guide documents
the ID/job-type contract and query-backed limitations.

Regression coverage exercises SQL submission, namespaces, refresh
options, job-type/ID preservation through typed result mapping, repeated
waits, failures and cancellation rejection. No local builds or tests
were run.
This commit is contained in:
Jack Ye
2026-10-25 01:20:13 -07:00
committed by GitHub
parent 24c2669a50
commit 9af7757053
20 changed files with 465 additions and 216 deletions
+8 -5
View File
@@ -17,7 +17,8 @@ extra round trip on every call:
- [Job.refresh](Job.md#refresh) and [Job.status](Job.md#status) fetch the whole record.
- [Job.wait](Job.md#wait) records the terminal state it establishes, but not the
rest of the record.
- Everything is null until one of those runs.
- Registry details are null until one of those runs. Query handles report
jobType "query" immediately.
## Accessors
@@ -57,8 +58,10 @@ get id(): null | string
Identifies the operation on the server that is running it.
Operations that run in this process have no server id. The value is
opaque: parsing it or storing it to resume the job later is not supported.
Registry handles expose a registry job ID; query handles expose a
connection-scoped SQL query UUID string. Local handles have no ID.
Query handles report jobType "query". Their IDs are not registry IDs or
durable resume tokens.
#### Returns
@@ -72,8 +75,8 @@ opaque: parsing it or storing it to resume the job later is not supported.
get jobType(): null | string
```
The job's type, as the server names it. Null for an in-process job, which
has no server-side record.
"query" immediately for SQL-backed handles; otherwise the cached
server-reported job type. Null for in-process handles.
#### Returns
+31 -6
View File
@@ -20,19 +20,20 @@ failures.
Methods such as Python's `create_index_async` and TypeScript's
`createIndexAsync` return job handles. In OSS, the work runs in your application
process; in Enterprise, it runs on server workers. A returned handle can
represent an operation that has already finished, so check its status before
process; in Enterprise, it runs on server workers or through the SQL query
backend. A returned handle can represent an operation that has already finished,
so check its status before
waiting or requesting cancellation. See the operation's API reference for when
the method returns.
## Submit and wait for a job {#waiting}
1. **Submit the operation.** Keep the returned job handle. For an Enterprise
job, also keep its ID if another process needs to look it up later.
registry job, also keep its ID if another process needs to look it up later.
2. **Check progress.** Read the current status while the operation runs.
3. **Wait for completion.** Check that the job succeeded before using its
result. If it failed or was cancelled, inspect the failure details and any
results it committed before stopping.
result. If it failed or was cancelled, inspect the failure details and any
results it committed before stopping.
In Python, `job.status()` reports the current state and `job.wait()` waits for
completion, raising if the job fails or is cancelled. If `job.wait()` times out, the client stops waiting.
@@ -59,13 +60,37 @@ history.
## OSS and Enterprise execution {#in-process}
| | OSS | Enterprise |
The following table describes local operations and Enterprise registry jobs.
SQL-backed job handles follow the query lifecycle described below.
| | OSS | Enterprise registry job |
| --- | --- | --- |
| Execution | In your application process | On server workers |
| Job identity | No server ID | A server ID for looking up the job |
| Event history | No server event history | Available through the jobs API |
| Reattaching after a client restart | No server job to reopen | Reopen by ID while the record is retained |
### SQL-backed job handles
Remote materialized-view CREATE and REFRESH use the SQL query backend. Python's
`create_materialized_view_async()` and `mv.refresh_async()` still return job
handles with `status()`, `wait()` and `cancel()`. Synchronous creation and refresh
wait on the same SQL execution before returning.
The existing `job.job_type` identifies query-backed handles as `"query"` from
submission, and `job.id` contains the connection-scoped SQL query UUID string.
Use `describe_query(job.id)` on the submitting connection to inspect the query.
Registry handles retain their server-reported job types (available after
`status()` or `refresh()`) and registry IDs. Local handles have no ID or job type.
SQL-backed handles do not expose registry records or event history and cannot
be reopened through `open_job()` or reattached after a client restart. Their IDs
are not durable resume tokens. Cancellation follows the SQL server's
capabilities and can be unsupported.
The server decides whether SQL executes on its serving fleet or starts SQL job
workers. That placement does not change the handle's query-backed identity.
## Learn more {#next}
- [Build and repair indexes](/indexing/reindexing): submit index work.