mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
Brendan Clement
1db0d3ca5f
Labels early access features and hides features we aren't documenting yet. **Early access label:** Functions (4 pages, plus the GPU refresh, checkpointing and adaptive execution pages), Secrets (3 pages), Tiered Index Cache, Blob V2, and merge insert, which is labelled for Enterprise only. Two sidebar titles are shorter so the tag fits. **Hidden:** branching, JSON columns and geospatial. - The JSON functions page and the agent branch-experiments tutorial go in `.mintignore`. - `tables/branching` stays reachable but out of the nav, as #4859 left it, because SDK error messages link to it. - Other pages' mentions of these features become draft comments, so they're easy to restore. **Secrets page:** drops references to manual KMS setup, which the control plane handles. It keeps the encryption-at-rest facts, states the Azure value-size limit inline, and points a 501 error to support.
94 lines
4.1 KiB
Plaintext
94 lines
4.1 KiB
Plaintext
---
|
|
title: "GPU Refresh Execution"
|
|
tag: "Early access"
|
|
sidebarTitle: "GPU Refresh Execution"
|
|
description: "A computed column or materialized view filled by a GPU Function schedules its refresh workers the same way GPU Function execution does in Data Curation."
|
|
icon: microchip
|
|
keywords: ["gpu", "gpu class", "scheduling", "refresh", "computed column", "function"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
<Badge color="purple">Early access</Badge>
|
|
|
|
A [computed column](/feature-engineering/add-computed-column) or
|
|
[materialized view](/feature-engineering/materialized-views) filled by a
|
|
registered [Function](/curation/create-function) with `gpu=True` refreshes
|
|
through the exact same placement path as
|
|
[GPU Function execution](/curation/gpu-function-execution) in Data
|
|
Curation -- refreshing a column is not a separate scheduling code path
|
|
from calling the Function directly, just a different caller.
|
|
|
|
<div className="hidden dark:block">
|
|
|
|
```mermaid
|
|
%%{init: {"theme": "base", "themeVariables": {
|
|
"background": "#171513", "primaryColor": "#1e1b19", "primaryTextColor": "#f4ebe8",
|
|
"primaryBorderColor": "#4c443f", "lineColor": "#847770", "textColor": "#f4ebe8",
|
|
"edgeLabelBackground": "#171513", "fontFamily": "Inter, sans-serif", "fontSize": "18px"
|
|
}}}%%
|
|
flowchart LR
|
|
F["Function registered with<br/>gpu=True, gpu_class"] --> A{"Who requests<br/>a worker?"}
|
|
A -->|"You, directly"| D["Data Curation:<br/>GPU Function execution"]
|
|
A -->|"A refresh job"| R["Feature Engineering:<br/>computed column / MV refresh"]
|
|
D --> P["Same GPU-class placement,<br/>same capacity-wait behavior"]
|
|
R --> P
|
|
classDef default fill:#1e1b19,stroke:#4c443f,stroke-width:1px,color:#f4ebe8
|
|
classDef accent fill:#2a1810,stroke:#e8582d,stroke-width:1.5px,color:#f4ebe8
|
|
class P accent
|
|
```
|
|
|
|
</div>
|
|
|
|
<div className="block dark:hidden">
|
|
|
|
```mermaid
|
|
%%{init: {"theme": "base", "themeVariables": {
|
|
"background": "#ffffff", "primaryColor": "#ffffff", "primaryTextColor": "#221d19",
|
|
"primaryBorderColor": "#ad9e95", "lineColor": "#6b625c", "textColor": "#221d19",
|
|
"edgeLabelBackground": "#ffffff", "fontFamily": "Inter, sans-serif", "fontSize": "18px"
|
|
}}}%%
|
|
flowchart LR
|
|
F["Function registered with<br/>gpu=True, gpu_class"] --> A{"Who requests<br/>a worker?"}
|
|
A -->|"You, directly"| D["Data Curation:<br/>GPU Function execution"]
|
|
A -->|"A refresh job"| R["Feature Engineering:<br/>computed column / MV refresh"]
|
|
D --> P["Same GPU-class placement,<br/>same capacity-wait behavior"]
|
|
R --> P
|
|
classDef default fill:#ffffff,stroke:#ad9e95,stroke-width:1px,color:#221d19
|
|
classDef accent fill:#fff1ec,stroke:#e8582d,stroke-width:1.5px,color:#221d19
|
|
class P accent
|
|
```
|
|
|
|
</div>
|
|
|
|
The Function's `gpu_class` (or the deployment's `default`) resolves to the
|
|
same node selector, tolerations and resource name either way, and a
|
|
refresh that needs more GPU capacity than is currently free waits the
|
|
same way a direct call would -- see
|
|
[Waiting for capacity](/curation/gpu-function-execution#capacity) for that
|
|
behavior in full. [Adaptive Execution](/enterprise/adaptive-execution)'s
|
|
resource-consumption-based sizing applies here too: a refresh's GPU
|
|
utilization is measured and fed back into sizing the *next* part the same
|
|
way CPU and memory are.
|
|
|
|
A single refresh can also override the Function's registered placement for
|
|
that run -- see
|
|
[Configure workers and resources](/feature-engineering/refresh-column#resources)
|
|
for the `gpu_class`/`gpu_count` options on `refresh_column_async` and the
|
|
equivalent SQL `WITH (...)` clause.
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="GPU Function Execution" icon="microchip" href="/curation/gpu-function-execution">
|
|
The full placement and capacity-wait mechanism this page reuses.
|
|
</Card>
|
|
<Card title="Adaptive Execution" icon="gauge-high" href="/enterprise/adaptive-execution">
|
|
How a refresh's resource class, part size and worker count adjust themselves.
|
|
</Card>
|
|
<Card title="Configure workers and resources" icon="sliders" href="/feature-engineering/refresh-column#resources">
|
|
Override GPU class and count for one refresh call.
|
|
</Card>
|
|
</CardGroup>
|