lancedb/docs/web/basics/jobs.mdx

---
title: "Jobs"
sidebarTitle: "Job"
description: "Work that outlives the call that started it: submitting, waiting, inspecting, cancelling."
icon: hourglass-half
keywords: ["job", "async", "background", "events", "cancel", "refresh"]
availability:
  oss: available
  enterprise: available
  summary: >-
    Long-running operations return a job handle in the embedded library and on
    a deployment. Listing jobs, reopening them by id and reading their events
    need a deployment, where the work runs on the server.
---

Some operations take longer than a request should. Building an index over a
large table or refreshing a computed column, for example, runs on the server
after the call that started it has returned.

Each is submitted the same way and returns the same handle.

<Note>
A [query](/basics/query) also has an `_async` form, but its handle streams rows
rather than waiting for an outcome. Both answer the same four questions --
which one is this, how is it going, give me the result, stop it -- and differ
only in the third.
</Note>

## The `_async` convention {#convention}

An operation that can run in the background has two forms. The plain name blocks
until the work is done. The `_async` / `Async` name submits the work and returns
a **job** immediately.

| Blocking | Returns a job | SQL |
| --- | --- | --- |
| `create_index` | `create_index_async` | `CREATE INDEX` |
| `refresh_column` | `refresh_column_async` | `REFRESH COLUMN` |
| `create_function` {/* if functions launched */} | `create_function_async` | `CREATE FUNCTION` |
| `drop_table` | `drop_table_async` | `DROP TABLE` |

<Note>
The blocking form is not a different operation -- it submits the same job and
waits. Reach for the `_async` form when you want to do something else meanwhile,
run several at once, or survive the client going away. On a deployment,
`create_index` returns once the build is submitted unless you pass
`wait_timeout`, and `refresh_column` has no blocking form.
</Note>

## Submitting and waiting {#waiting}

<CodeGroup>
```python Python icon="python"
from lancedb.index import IvfPq

job = table.create_index_async("vector", config=IvfPq())

print(job.id)          # the server's id for this operation
print(job.status())    # "running" | "finished" | "failed" | "cancelled"

job.wait()             # blocks until terminal; raises if it failed
```

```typescript TypeScript icon="square-js"
const job = await table.createIndexAsync("vector");

console.log(job.id);              // the server's id for this operation
console.log(await job.status());  // "running" | "finished" | "failed" | "cancelled"

await job.wait();                 // resolves when terminal; rejects if it failed
```
</CodeGroup>

`wait` is the one call that raises on failure. `status` is a point snapshot and
returns the failed state rather than raising, which is what you want in a
polling loop.

In Python `wait` takes an optional timeout and returns the operation's typed
result -- a `RefreshColumnJobResult`, for example, or `None` for an operation
that has no result:

```python
from datetime import timedelta

result = table.refresh_column_async("embedding").wait(timeout=timedelta(minutes=30))
```

`TimeoutError` means the timeout elapsed, not that the job stopped. The job is
still running; the handle just gave up watching.

<Note>
The TypeScript `wait()` resolves to `void`. Read the outcome from `result` after
it resolves.
</Note>

## Inspecting a job {#inspecting}

Submitting an operation returns only an id, so the detail fields are empty until
the handle fetches the record. `refresh()` does that; `status()` does it as a
side effect.

<CodeGroup>
```python Python icon="python"
job.refresh()

job.state        # last observed state, without another round trip
job.job_type     # what kind of job, as the server names it
job.creation_ms  # when it was created, epoch milliseconds
job.spec         # what it was submitted with
job.result       # its terminal result, once it succeeds
job.failure      # why it failed, when it did
```

```typescript TypeScript icon="square-js"
await job.refresh();

job.state;       // last observed state, without another round trip
job.jobType;     // what kind of job, as the server names it
job.creationMs;  // when it was created, epoch milliseconds
job.spec;        // what it was submitted with
job.result;      // its terminal result, once it succeeds
job.failure;     // why it failed, when it did
```
</CodeGroup>

Printing the handle renders every field it knows, which is usually faster than
reading them one at a time.

## Progress, from events {#events}

`result` only tells you anything once the job has finished. Events are written
as it runs, and they outlive the workers that produced them -- so a job that is
still going, or one that never finished, can still account for what it did.

<CodeGroup>
```python Python icon="python"
events = job.events(limit=5000, filter="state = 'claim_complete'")
```

```typescript TypeScript icon="square-js"
const events = await job.events({ limit: 5000, filter: "state = 'claim_complete'" });
```

```sql SQL
SHOW ERRORS ON my_table FOR JOB 'job-id' LIMIT 100
```
</CodeGroup>

A distributed job records a `claim` / `claim_complete` pair per unit of work,
each carrying `rows_processed`. Summing `rows_processed` over the completed
claims is how you get progress on a job that has no percentage.

`filter` is a SQL-like expression over `state`, `updated_by`, `emitted_from`,
`emitted_by`, and `claim_entity`.

<Warning>
The server returns 1000 event rows by default and 10,000 at most, and
**truncates without telling you**. A job that emits an event per fragment will
silently lose its tail unless you pass `limit`.
</Warning>

Events are recorded server-side, so this raises for an operation that ran
in-process.

## Finding and cancelling jobs {#managing}

On a deployment, the connection lists and reopens jobs, so a handle lost to a
restarted process is recoverable from its id. A local database keeps no jobs to
list, and these calls raise there.

<CodeGroup>
```python Python icon="python"
db.list_jobs()             # every job the server knows about
job = db.open_job(job_id)  # reattach to one
db.cancel_job(job_id)      # ask it to stop
```

```typescript TypeScript icon="square-js"
await db.listJobs();
const job = await db.openJob(jobId);
await db.cancelJob(jobId);
```

```sql SQL
SHOW JOBS
```
</CodeGroup>

Cancellation is a request, not a guarantee, and cancelling a job that already
finished does nothing. `JobNotFoundError` means the id does not name a job the
server still has.

<Warning>
A job id is opaque. It correlates with server logs and the calls above, but
parsing it or storing it to resume the job later is not supported.
</Warning>

## In-process jobs {#in-process}

A local database has no server to run anything, so its operations run in your
process. They still return a handle, so the same code works against both -- but
one with no server behind it: `id` is `None` and `events()` raises. An operation
still running in your process reports `"running"`, `wait()` waits for it and
`cancel()` stops it; one that finished before the call returned, such as a local
`drop_table_async`, reports `"finished"` at once.

This is the difference to expect when moving between embedded and deployed --
see [SDKs](/basics/sdk).