Jack Ye
9af7757053
Remote materialized-view creation currently expects a REST response containing a registry job ID, while refresh already submits SQL. Route both through the SQL query backend and preserve Python's synchronous and asynchronous return types. ### Query to Job mapping `SqlClient::submit_as_job()` submits an `ExecuteQueryRequest` through the same backend as `execute_query_async()`. Once submission succeeds, it retains the returned `Query` and starts one client-side task that consumes its result reader to completion. `Job::from_query_task()` wraps that task, its cached outcome, a single ID and a cancellation callback. Python exposes this as `Job[T]` or `AsyncJob[T]`. The existing `job_type` property maps SQL-backed handles to the new `"query"` kind. The shared `id` contains that query's UUID string, so Python callers can inspect it with `db.describe_query(job.id)` on the submitting connection. No separate `query_id`, backend enum or `type()` API is added. | Handle | `job_type` | `id` | Available lookup | | --- | --- | --- | --- | | SQL-backed | `"query"`, known at submission | Connection-scoped query UUID string | `describe_query(job.id)` | | Registry-backed | Existing server-reported kind, after `status()` or `refresh()` | Registry job ID string | Registry job APIs, descriptions and events | | Local | `None` | `None` | The current in-process handle | | Job concept | SQL-backed behavior | | --- | --- | | Submission | Await initial SQL submission before returning the handle. Submission errors propagate immediately. | | `status()` | Tracks the adapter task: running during result consumption/finalization, finished after both succeed, failed if either fails, or cancelled after cancellation. | | `wait()` | Returns the operation's typed terminal result. One task consumes the query reader once; repeated waits reuse its cached outcome without resubmitting SQL or reopening the reader. | | `cancel()` | Delegates to `Query.cancel()`. Rejected or unsupported cancellation leaves the task running; an accepted request stops the task. Cancelling a finished operation is a no-op. | | Registry details | Query-backed handles expose no registry description, event history, pause/resume or reattachment after a client restart. `refresh()` updates cached lifecycle state. | The Job tracks command completion rather than exposing SQL batches. A wait timeout does not cancel the query. Query IDs remain scoped to the submitting connection and cannot be used with registry APIs or as durable resume tokens. Python `describe_query()` accepts both UUID objects and the string `id` exposed by query-backed jobs. Rust, Python and TypeScript expose the same classification through their existing job-type accessors. ### Materialized-view operations - **CREATE:** Render `CREATE MATERIALIZED VIEW` with the view namespace and optional auto-refresh settings. After command completion, resolve the job to `None`. Synchronous creation waits on the job, then opens and returns the materialized view. - **REFRESH:** Submit `REFRESH MATERIALIZED VIEW` through the same adapter. After command completion, read the refreshed table's version and row count to construct the existing typed refresh result. Synchronous refresh waits on the same path. Remote creation requires the SQL service. Execution placement remains a server decision. REST endpoints are unchanged. The job guide documents the ID/job-type contract and query-backed limitations. Regression coverage exercises SQL submission, namespaces, refresh options, job-type/ID preservation through typed result mapping, repeated waits, failures and cancellation rejection. No local builds or tests were run.
4.0 KiB
@lancedb/lancedb • Docs
@lancedb/lancedb / Job
Class: Job
A handle to an operation that may still be running.
The operation may already be complete when the handle is created.
The detail getters read what the handle last observed. Submitting an operation returns only a job id, so populating them eagerly would cost an extra round trip on every call:
- Job.refresh and Job.status fetch the whole record.
- Job.wait records the terminal state it establishes, but not the rest of the record.
- Registry details are null until one of those runs. Query handles report jobType "query" immediately.
Accessors
creationMs
get creationMs(): null | number
When the job was created, in milliseconds since the epoch.
Returns
null | number
failure
get failure(): null | JobFailureInfo
Why the job failed, when it failed and the server reports a reason.
Returns
null | JobFailureInfo
id
get id(): null | string
Identifies the operation on the server that is running it.
Registry handles expose a registry job ID; query handles expose a connection-scoped SQL query UUID string. Local handles have no ID. Query handles report jobType "query". Their IDs are not registry IDs or durable resume tokens.
Returns
null | string
jobType
get jobType(): null | string
"query" immediately for SQL-backed handles; otherwise the cached server-reported job type. Null for in-process handles.
Returns
null | string
result
get result(): any
The job-type-specific terminal result. Null until the job succeeds, so a job that never terminates reports its progress through Job.events instead.
Returns
any
spec
get spec(): any
The job-type-specific specification it was submitted with.
Returns
any
state
get state(): null | string
The last observed lifecycle state, without contacting the backend.
Returns
null | string
Methods
cancel()
cancel(): Promise<void>
Request cancellation. Cancelling a finished operation is a no-op.
Returns
Promise<void>
events()
events(options?): Promise<Table<any>>
This job's recorded lifecycle events.
Where the getters above report a terminal result only once the job reaches
one, events are written as the job runs and outlive the workers that
produced them. A distributed job records a claim/claim_complete pair
per unit of work, each carrying rows_processed, so a job that never
finishes still accounts for what it did.
The server caps results at 1000 rows by default and 10,000 at most, and
truncates without saying so, so pass limit for a job that emits an event
per fragment. filter is a SQL-like expression over the state,
updated_by, emitted_from, emitted_by, and claim_entity columns.
Parameters
- options?:
JobEventsOptions
Returns
Promise<Table<any>>
refresh()
refresh(): Promise<void>
Ask the backend for this job's current state, and for a server-side job its full record, then cache it for the getters above.
Returns
Promise<void>
status()
status(): Promise<string>
The operation's current lifecycle state: "running", "finished", "failed", or "cancelled".
A point snapshot; unlike Job.wait it does not block or reject on a terminal failure state. Also refreshes the getters above.
Returns
Promise<string>
toString()
toString(): string
Every field the handle currently knows, one per line, with the JSON payloads indented -- a refresh job's spec and result are the point of printing it.
Returns
string
wait()
wait(): Promise<void>
Wait until the operation reaches a terminal state.
Returns
Promise<void>