docs: publish the Function pages only once Functions launch

The two Function pages declare `launch: functions`, and every other page
marks what it says about Functions, so the assembled site leaves them out
until the launch. Before it, the pages that offered Functions for Python
work point Enterprise readers at Geneva instead.

The SQL Function example now uses Arrow type names, which the server
requires, and states the 3.12 default for `python_version`.
This commit is contained in:
Brendan Clement
2026-10-09 13:07:33 -07:00
parent 62a4edc654
commit e20664740f
25 changed files with 142 additions and 48 deletions
@@ -38,9 +38,12 @@ including that you can travel back to before the column existed.
- **SQL expressions in `transforms`** are evaluated *now*, for every row, as
part of the call. Good for a small table, or a column you want populated
immediately. A `FunctionApplication` passed through `transforms` instead
immediately.
{/* if functions launched */}
A `FunctionApplication` passed through `transforms` instead
declares empty Function-bound columns, which a refresh fills; that needs a
deployment with Functions enabled, and the embedded library refuses it.
{/* end if */}
- **`computed`** stores the expression and evaluates nothing. The column is
committed empty and filled later by
[`refresh_column`](/feature-engineering/refresh-column).
@@ -81,11 +84,18 @@ reshape later. Dropping the computed column releases the constraint.
<Badge color="red">Enterprise</Badge>
{/* if functions launched */}
When the value cannot be expressed in SQL -- an embedding, a model call, an
image decode -- the column is filled by a registered
[function](/management/function) instead of an expression. Declaring it looks
the same; what differs is what fills it. It needs a deployment with Functions
enabled.
{/* end if */}
{/* if functions not launched */}
When the value cannot be expressed in SQL -- an embedding, a model call, an
image decode -- [Geneva](/geneva), the feature engineering package for LanceDB
Enterprise, fills the column with a Python function instead.
{/* end if */}
## What to read next {#next}
@@ -93,7 +103,14 @@ enabled.
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
Fill the rows that lack values.
</Card>
{/* if functions launched */}
<Card title="Functions" icon="code" href="/management/function">
Fill a column with Python.
</Card>
{/* end if */}
{/* if functions not launched */}
<Card title="Geneva" icon="layer-group" href="/geneva">
Fill a column with Python on LanceDB Enterprise.
</Card>
{/* end if */}
</CardGroup>
@@ -108,10 +108,17 @@ exist when that does not fit:
- A [computed column](/feature-engineering/add-computed-column) declares the
expression and leaves the column empty, so you choose when the work happens.
That is the right default on a large table.
{/* if functions launched */}
- A [function](/management/function) fills a column with Python, for values SQL
cannot express -- an embedding, a model call, an image decode.
{/* end if */}
{/* if functions not launched */}
- On LanceDB Enterprise, [Geneva](/geneva) fills a column with Python, for
values SQL cannot express -- an embedding, a model call, an image decode.
{/* end if */}
On a deployment, both have to be enabled there.
A deployment has to enable computed columns before you use them.
Functions need enabling there too. {/* if functions launched */}
## What to read next {#next}
+18 -6
View File
@@ -3,15 +3,20 @@ title: "Feature engineering"
sidebarTitle: "Overview"
description: "Derive columns and tables from data already in the lakehouse, without moving it somewhere else to compute."
icon: layer-group
keywords: ["feature engineering", "computed columns", "materialized views", "functions", "lakehouse", "jobs"]
keywords: ["feature engineering", "computed columns", "materialized views", "lakehouse", "jobs"]
---
Feature engineering happens where the data already is. Rather than exporting a
table, computing somewhere else and writing the result back, you declare what a
column or a table should contain and let LanceDB fill it.
{/* if functions launched */}
[Functions](/management/function) and
[materialized views](/feature-engineering/materialized-views) are ordinary
{/* end if */}
{/* if functions not launched */}
[Materialized views](/feature-engineering/materialized-views) are ordinary
{/* end if */}
[catalog objects](/basics/catalog), alongside tables and indexes -- created,
versioned, and refreshed like anything else, not a separate system bolted
alongside.
@@ -48,7 +53,7 @@ idempotent and cheap -- but it will surprise you if you expect a spreadsheet.
| --- | --- | --- | --- |
| [Computed column](/feature-engineering/add-computed-column) | a new column on an existing table | a SQL expression | embedded, and deployments that enable it |
| [Materialized view](/feature-engineering/materialized-views) | a new table derived by a query | a refresh | embedded, and deployments that enable it |
| [Function](/management/function) | a computed column filled by Python | registered code | <Badge color="red">Enterprise</Badge> deployments that enable it |
| [Function](/management/function) {/* if functions launched */} | a computed column filled by Python | registered code | <Badge color="red">Enterprise</Badge> deployments that enable it |
**Computed columns** are the common case: a column declared with a SQL
expression, filled by [a refresh](/feature-engineering/refresh-column).
@@ -57,12 +62,14 @@ expression, filled by [a refresh](/feature-engineering/refresh-column).
aggregated view of a source -- and bring it up to date on each refresh. The
result is an ordinary table: queryable, indexable, searchable.
{/* if functions launched */}
**Functions** run Python where the data is, for work SQL cannot express:
embedding a column, calling a model, decoding an image. Registered once,
versioned, then applied as a column.
They compose. A materialized view can select computed columns, and a computed
column can be filled by a registered function.
{/* end if */}
They compose. A materialized view can select computed columns.
A computed column can be filled by a registered function. {/* if functions launched */}
## Where the work runs {#where}
@@ -83,15 +90,18 @@ from a different process -- the connection does not have to stay open.
That relocation is the main thing to understand about Enterprise feature
engineering. The API is the same; what changes is that the code runs somewhere
else, which is also why a function must declare the environment it needs.
else.
That is also why a function must declare the environment it needs. {/* if functions launched */}
## Recovering from a failure {#failures}
{/* if functions launched */}
A function that raises fails the job. There is no per-row retry or skip policy
in the function definition, so a function calling a flaky external service
should handle that itself, inside the function body.
The alternative is to let the job fail, fix the cause, and run the refresh
{/* end if */}
When a refresh fails, fix the cause and run the refresh
again. The new refresh fills what is still empty or stale. How much of the
failed run was kept depends on where it ran (see
[Refresh a column](/feature-engineering/refresh-column#failures)), and a retry
@@ -110,7 +120,9 @@ only the rows that did not complete.
<Card title="Refresh a materialized view" icon="table" href="/feature-engineering/materialized-views">
Bring a derived table up to date with its source.
</Card>
{/* if functions launched */}
<Card title="Functions" icon="code" href="/management/function">
Register Python to fill a column with.
</Card>
{/* end if */}
</CardGroup>
@@ -69,9 +69,8 @@ REFRESH COLUMN embedding ON documents WHERE created_at > '2026-01-01'
```
<Note>
**Enterprise job type:** `computed_refresh` for a SQL-expression column,
`function_refresh` for one filled by a registered
[function](/management/function).
**Enterprise job type:** `computed_refresh` for a SQL-expression column.
`function_refresh` for one filled by a registered [function](/management/function). {/* if functions launched */}
</Note>
## As a job {#async}
@@ -128,10 +127,12 @@ that were already written -- for example when the record of the inputs they
were computed from was not saved -- so do not count on a retry costing only the
rows that did not complete.
{/* if functions launched */}
For a column filled by a [function](/management/function), the usual cause is
the worker environment rather than the data. See
[environment mismatches](/management/function#mismatch-symptoms) for the
symptoms, which rarely name their cause.
{/* end if */}
## What to read next {#next}