mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 14:52:26 +00:00
Brendan Clement
e20664740f
The two Function pages declare `launch: functions`, and every other page marks what it says about Functions, so the assembled site leaves them out until the launch. Before it, the pages that offered Functions for Python work point Enterprise readers at Geneva instead. The SQL Function example now uses Arrow type names, which the server requires, and states the 3.12 default for `python_version`.
74 lines
3.5 KiB
Plaintext
74 lines
3.5 KiB
Plaintext
---
|
|
title: "Background jobs"
|
|
sidebarTitle: "Background Jobs"
|
|
description: "Jobs a deployment runs on its own — visible in SHOW JOBS, not started by a call you make."
|
|
icon: gears
|
|
keywords: ["background jobs", "job type", "wal", "spfresh", "prewarm", "archive"]
|
|
availability:
|
|
oss: unavailable
|
|
enterprise: available
|
|
summary: >-
|
|
Background jobs run only on a deployment, which starts them itself; its
|
|
configuration decides which of them run.
|
|
---
|
|
|
|
The jobs documented elsewhere in these docs -- [Build & Repair](/indexing/reindexing),
|
|
[Compaction](/maintenance/compaction), [Cleanup](/maintenance/cleanup),
|
|
[Column Refresh](/feature-engineering/refresh-column) -- start from a call you
|
|
make or, for compaction, cleanup and index updates, from the deployment's own
|
|
maintenance. This page is the rest: jobs the deployment runs to keep itself
|
|
correct and fast, which you may see named in [`SHOW JOBS`](/sql/ddl/job#show)
|
|
without having asked for any of them.
|
|
|
|
There is no action to take on these beyond recognizing the name. None of them
|
|
has a parameter you set from an SDK; the deployment's operators configure them,
|
|
and start the cache-warming jobs through its admin API.
|
|
|
|
## Write-ahead log maintenance {#wal}
|
|
|
|
For a table with a [write-ahead log](/sql/ddl/table#properties) enabled, two jobs keep the
|
|
log from growing without bound:
|
|
|
|
| Job type | What it does |
|
|
| --- | --- |
|
|
| `sstable_compaction` | Folds a WAL bucket's SSTables into the base table by primary key. Distinct from `compact`, which merges base-table fragments -- the two are unrelated. |
|
|
| `wal_index_catchup` | Extends a base-table index over the rows `sstable_compaction` just folded in, and records how far the index has caught up. Nothing else advances that record. |
|
|
|
|
## Index maintenance {#index}
|
|
|
|
| Job type | What it does |
|
|
| --- | --- |
|
|
| `index_remap` | Reassigns index entries after row ids shift underneath an index -- the kind of remap a delta compaction can require. |
|
|
| `spfresh_merge` | Merges SPFresh-managed segments for indexes that use it. |
|
|
|
|
## Cache warming {#cache}
|
|
|
|
| Job type | What it does |
|
|
| --- | --- |
|
|
| `prewarm_page_cache` | Loads table data into a query node's page cache ahead of a request needing it. |
|
|
| `prewarm_index_cache` | The same, for index structures. |
|
|
| `startup_prewarm_idx` | Runs cache prewarming as part of a plan executor's own startup, rather than on a schedule, when the deployment enables it. |
|
|
|
|
## Registry housekeeping {#registry}
|
|
|
|
| Job type | What it does |
|
|
| --- | --- |
|
|
| `job_registry_archive` | Sweeps terminal jobs out of the active registry and into archival storage, on a per-database schedule. |
|
|
| `job_events_export` | Copies terminal jobs' event histories into a curated system table for metering. Registry-scoped, like the archive sweep. |
|
|
|
|
These two are why an old job you completed weeks ago no longer shows in
|
|
`SHOW JOBS` -- it was archived, not deleted. Where the deployment enables the
|
|
export, its events also survive in the exported table.
|
|
|
|
## Object teardown {#teardown}
|
|
|
|
| Job type | What it does |
|
|
| --- | --- |
|
|
| `drop_object` | Deletes a dropped catalog object's content, then clears it from the database's cleanup backlog. Covers namespaces, views and other catalog objects -- everything [`drop_table`](/management/table#drop) does not, since that one fences a directory with its own marker instead. |
|
|
|
|
You trigger this indirectly, by dropping a [namespace](/management/namespace),
|
|
a [view](/sql/ddl/view#drop),
|
|
or a [function](/management/function#drop), {/* if functions launched */}
|
|
for example -- the teardown itself runs as this job rather than inline with the
|
|
call.
|