Files
lancedb/docs/web/basics/jobs.mdx
T
Jack YeandXuanwo 45dbf5eff5 fix: use SQL for all remote materialized view APIs (#4673)
Remote materialized-view APIs now use SQL for the full lifecycle. Create
and refresh already submit SQL; this change moves describe, paginated
listing, and drop onto `DESCRIBE MATERIALIZED VIEW`, `SHOW MATERIALIZED
VIEWS`, and `DROP MATERIALIZED VIEW`.

Async create, refresh, and drop return the existing Job handle with
`job_type = "query"`. The handle has one `id`: the connection-scoped SQL
query ID, also accepted by `describe_query`. Submission starts the
query, the job drains its result once, and repeated waits reuse the
completed result. Synchronous drop waits for catalog removal; physical
cleanup proceeds in the background.

Listing uses namespace-scoped keyset pagination with opaque resume
tokens. Streamed and deferred SQL responses share the same path. Adds
transport coverage and updates Python/TypeScript documentation for
query-backed drop jobs.

---------

Co-authored-by: Xuanwo <github@xuanwo.io>
2026-10-28 04:37:00 +08:00

103 lines
4.9 KiB
Plaintext

---
title: "Jobs"
sidebarTitle: "Job"
description: "Track background work from submission to completion."
icon: hourglass-half
keywords: ["job", "background", "progress", "events", "cancel"]
availability:
oss: available
enterprise: available
summary: >-
Job handles track operations in OSS and Enterprise. Enterprise also
supports listing server jobs, reopening them by ID and reading their events.
---
## Job handles {#convention}
A job handle lets you track an operation, such as an index build or a computed
column refresh. Use it to check progress, wait for completion and inspect
failures.
Methods such as Python's `create_index_async` and TypeScript's
`createIndexAsync` return job handles. In OSS, the work runs in your application
process; in Enterprise, it runs on server workers or through the SQL query
backend. A returned handle can represent an operation that has already finished,
so check its status before
waiting or requesting cancellation. See the operation's API reference for when
the method returns.
## Submit and wait for a job {#waiting}
1. **Submit the operation.** Keep the returned job handle. For an Enterprise
registry job, also keep its ID if another process needs to look it up later.
2. **Check progress.** Read the current status while the operation runs.
3. **Wait for completion.** Check that the job succeeded before using its
result. If it failed or was cancelled, inspect the failure details and any
results it committed before stopping.
In Python, `job.status()` reports the current state and `job.wait()` waits for
completion, raising if the job fails or is cancelled. If `job.wait()` times out, the client stops waiting.
The timeout does not cancel the server job.
<span id="inspecting" />
<span id="events" />
## Inspect and cancel jobs {#managing}
| Task | What to do |
| --- | --- |
| Inspect progress | Check status and refresh the handle to retrieve details. Enterprise job events can help explain ongoing work or failures. |
| Reattach from another process | Open an existing server job with its ID, such as Python's `db.open_job(job_id)`. The server must still retain the job record. |
| Diagnose a failure | Inspect failure details and relevant events. Enterprise SQL also provides `SHOW ERRORS`. |
| Request cancellation | Call the job's cancellation method. Cancellation may take time, and work already committed is not rolled back. |
Reopening a job retrieves the existing operation; it does not restart failed
work. Treat the ID as an opaque identifier and pass it unchanged to the jobs API.
Event responses have [row limits](/basics/limits#jobs). An explicit limit can
request more than the default, but it does not guarantee a complete event
history.
## OSS and Enterprise execution {#in-process}
The following table describes local operations and Enterprise registry jobs.
SQL-backed job handles follow the query lifecycle described below.
| | OSS | Enterprise registry job |
| --- | --- | --- |
| Execution | In your application process | On server workers |
| Job identity | No server ID | A server ID for looking up the job |
| Event history | No server event history | Available through the jobs API |
| Reattaching after a client restart | No server job to reopen | Reopen by ID while the record is retained |
### SQL-backed job handles
Remote materialized-view CREATE, REFRESH, and DROP use the SQL query backend.
Python's `create_materialized_view_async()`, `mv.refresh_async()`, and
`drop_materialized_view_async()` return job handles with `status()`, `wait()`,
and `cancel()`. Synchronous creation, refresh, and drop wait on the same SQL
execution before returning. A completed DROP means the catalog binding has been
removed; physical cleanup continues in the background.
The existing `job.job_type` identifies query-backed handles as `"query"` from
submission, and `job.id` contains the connection-scoped SQL query UUID string.
Use `describe_query(job.id)` on the submitting connection to inspect the query.
Registry handles retain their server-reported job types (available after
`status()` or `refresh()`) and registry IDs. Local handles have no ID or job type.
SQL-backed handles do not expose registry records or event history and cannot
be reopened through `open_job()` or reattached after a client restart. Their IDs
are not durable resume tokens. Cancellation follows the SQL server's
capabilities and can be unsupported.
The server decides whether SQL executes on its serving fleet or starts SQL job
workers. That placement does not change the handle's query-backed identity.
## Learn more {#next}
- [Build and repair indexes](/indexing/reindexing): submit index work.
- [Column refresh](/feature-engineering/refresh-column): populate derived values.
- [Background jobs](/maintenance/background-jobs): understand Enterprise maintenance.
- [SQL job commands](/sql/ddl/job): list jobs and inspect errors.
- [SDKs](/basics/sdk): find the job API for your client.