Files
lancedb/docs/web/sql/ddl/job.mdx
T
Jack YeandClaude Sonnet 5 de95ed3f6d docs: overhaul curation, feature engineering, security and deployment guides (#4748)
A broad documentation pass across the areas LanceDB Enterprise and
Embedded
diverge on most:

**Data Curation** -- GPU Function execution, Slurm scheduling (and what
Slurm is, for readers new to it), blob late materialization as its own
page, joins, aggregation/profiling, sampling, near-duplicate detection,
chunking, distributed shuffle, and IVF-partition clustering.

**Feature Engineering** -- computed columns and materialized views,
restructured to lead with the Function-based path and show the SQL form
side by side with Python throughout; a single Auto Refresh page instead
of
duplicating scheduling details on every refresh page; adaptive
execution;
checkpointing; GPU and Slurm placement for refresh workers; a Training
Cache page for the PyTorch data loader's automatic cache benefit.

**Security & Governance** -- an Audit Log reference (schema, access,
grants), Secrets promoted to an ordinary catalog object with their own
SQL
DDL and a vending/rotation guide, and privilege-table fixes verified
against the authorization source.

**Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry
moved
into a real Deployment area; Automated Cleanup folded into Intelligent
Auto-Maintenance (every other maintenance decision already lived there);
RTE Compaction and Advanced Conflict Resolution documented in depth.

**Landing page** -- the Enterprise feature comparison table reorganized
(several redundant or stale rows removed, several new areas added), and
the
"why Enterprise" bullets replaced with ones tied to a concrete page
(Performance, Scale, Fault tolerance, Multicloud deployment, Simplify
operations) instead of vague promises.

Plus a full pass addressing lancedb-gatekeeper's review: a real
test-safety
bug (a quickstart test could silently overwrite and delete a
deployment's
own `frames` table), a fabricated SQL statement with no backing
implementation, an actual dedup-guarantee bug in `take_row_ids`, and
dozens
of documentation-accuracy fixes verified against source rather than
taken
on faith -- each either fixed, or left with an explicit note on why it
needs a feature or deployment owner's sign-off instead of being guessed
at.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-03 16:17:24 -08:00

175 lines
5.9 KiB
Plaintext

---
title: "Job"
sidebarTitle: "Job"
description: "SHOW JOBS, SHOW JOB EVENTS, DESCRIBE JOB, SHOW ERRORS, and controlling a running job with PAUSE, RESUME and CANCEL."
icon: hourglass-half
keywords: ["show jobs", "show job events", "describe job", "show errors", "pause job", "resume job", "cancel job", "sql"]
availability:
embedded: unavailable
enterprise: available
---
A [job](/basics/jobs) is not a catalog object (it has no schema and is not
namespaced), but it has its own SQL statements, grouped here the same way
the catalog objects are.
## SHOW JOBS {#show}
<CodeGroup>
```sql SQL icon="database"
SHOW JOBS
SHOW JOBS FOR TABLE documents
SHOW JOBS FOR JOB '<parent_job_id>' [LIMIT n]
```
</CodeGroup>
The unfiltered command lists jobs in the connected database, newest first:
id, type, state, creation time -- the SQL counterpart of `list_jobs`. `FOR
TABLE` filters that listing by the current namespace's table name. `FOR JOB`
lists the parent's direct child jobs using the REST API's child model.
Child rows expose their ID, kind, phase, row progress and attempt; unknown
values remain NULL. Children are read-only diagnostic identities.
`FOR JOB` defaults to 1,000 children and accepts `LIMIT` from 1 to 10,000,
following backend continuation pages up to that limit. Child history can be
`available`, `not_yet_recorded`, or `unavailable` after archival.
<Note>
A job you completed weeks ago eventually stops appearing here -- it was
archived, not deleted. Two deployment-scheduled jobs sweep the registry on a
per-database schedule: `job_registry_archive` moves terminal jobs out of the
active registry into archival storage, and `job_events_export` separately
copies terminal jobs' event histories into a curated system table for
metering, where the deployment enables it. A job's events can survive in
that exported table after the job itself is archived.
</Note>
## SHOW JOB EVENTS {#events}
<CodeGroup>
```sql SQL icon="database"
SHOW JOB EVENTS '<parent_or_child_job_id>' [LIMIT n]
```
</CodeGroup>
Events are returned newest first. Parent history does not include child
history. The default limit is 1,000; valid limits range from 1 to 10,000.
The result includes event sequence, state, timestamps, origin, claim and
resource/work fields. Child `kind` and `details` expose the REST API's
allowlisted diagnostics; raw parent payloads are excluded.
## DESCRIBE JOB {#describe}
<CodeGroup>
```sql SQL icon="database"
DESCRIBE JOB '<job_id>' [FORMAT TEXT | TABLE | JSON]
DESC JOB '<job_id>' [FORMAT TEXT | TABLE | JSON]
```
</CodeGroup>
Both spellings show a finished root job's timing profile. `TEXT` is the
default; `TABLE` returns typed phase/step rows and `JSON` returns the
profile as one document. A running job has no profile rows.
## SHOW ERRORS {#errors}
<CodeGroup>
```sql SQL icon="database"
SHOW ERRORS ON <table> [FOR JOB '<job-id>'] [LIMIT <n>]
```
</CodeGroup>
<CodeGroup>
```sql SQL icon="database"
SHOW ERRORS ON documents
SHOW ERRORS ON documents FOR JOB 'job-abc' LIMIT 100
```
</CodeGroup>
Why rows failed. Reach for this when a [refresh](/sql/dml/refresh-column) reports
fewer filled rows than you expected.
### Errors and limits {#errors-and-limits}
| | |
| --- | --- |
| `LIMIT` | defaults to 10,000; must be between 1 and 100,000 |
| Failed jobs scanned without `FOR JOB` | 32 most recent |
| Tables the server does not open directly | not supported |
Without `FOR JOB` this surveys recent failures and stops at 32 jobs. When the
listing stops at your `LIMIT`, the message says so. Narrow with `FOR JOB`
rather than raising the limit blindly.
<Note>
`SHOW JOBS`, `SHOW JOB EVENTS`, `DESCRIBE JOB` and `SHOW ERRORS` require
system `USAGE`, as do the equivalent REST diagnostics. Explicit table
listings additionally use the table-list authorization policy.
</Note>
## PAUSE JOB {#pause}
<CodeGroup>
```sql SQL icon="database"
PAUSE JOB 'j-ci-abc-def'
```
</CodeGroup>
## RESUME JOB {#resume}
<CodeGroup>
```sql SQL icon="database"
RESUME JOB 'j-ci-abc-def'
```
</CodeGroup>
## CANCEL JOB {#cancel}
<CodeGroup>
```sql SQL icon="database"
CANCEL JOB 'j-ci-abc-def'
```
</CodeGroup>
The job id is always a quoted string literal. All three take exactly one and
nothing after it: a second token on the line is a syntax error rather than
being ignored. Pause requires the deployment's pause-write opt-in and a
resumable job type; it may return `pausing` while workers drain, and resume
may return `still_pausing` until drain finishes.
<Note>
`PAUSE JOB`, `RESUME JOB` and `CANCEL JOB` require system `OPERATE` -- a
broader privilege than the `USAGE` that reading job state needs, since
these change what a running job does.
</Note>
<Note>
Cancellation is a request, matching the SDK's
[`cancel()`](/basics/jobs#managing): it does not guarantee the job stops before
doing more work, and asking twice or cancelling a finished job does nothing.
`CANCEL JOB` itself can be unavailable on an Enterprise cluster with no job registry
configured, which is a deployment condition rather than something you can work
around from SQL.
</Note>
## Compatibility {#compatibility}
`SHOW JOBS FOR <table>` (without the `TABLE` keyword), `SHOW JOB SUMMARY`,
and `SHOW JOB SPLITS` retain their existing behavior as compatibility forms.
Prefer `SHOW JOBS FOR TABLE`/`FOR JOB` and `DESCRIBE JOB` going forward. See
[`SHOW JOBS FOR JOB`](#show) for the per-split diagnostics
form -- each split is a child job, listed with its own phase and progress.
See [Watching a refresh](/sql/dml/refresh#watching) for `SHOW JOB SPLITS`'s
full column reference and a `DESCRIBE JOB` example.
## What to read next {#next}
<CardGroup cols={2}>
<Card title="Query and job" icon="hourglass-half" href="/basics/jobs">
The handle these statements are the SQL view of.
</Card>
<Card title="Intelligent Auto-Maintenance" icon="brain" href="/enterprise/intelligent-auto-maintenance#cleanup">
A job type the deployment schedules on its own, without a call.
</Card>
</CardGroup>