lancedb/docs/web/management/function.mdx

---
title: "Functions"
sidebarTitle: "Function"
description: "Register a Python function once and apply it as a column, for work SQL cannot express."
icon: code
keywords: ["udf", "functions", "create_function", "embedding", "feature engineering"]
availability:
  oss: unavailable
  enterprise: available
  summary: >-
    Enterprise supports Functions when enabled for your Enterprise cluster; the embedded library cannot register or run them.
---

import {
    PyFunctionDefine as FunctionDefine,
    PyFunctionLocalCall as FunctionLocalCall,
    PyFunctionCreateTable as FunctionCreateTable,
    PyFunctionRegister as FunctionRegister,
    PyFunctionApply as FunctionApply,
    PyFunctionRefresh as FunctionRefresh,
    PyFunctionInspect as FunctionInspect,
} from '/snippets/function.mdx';
import { PyConnectEnterprisePlaceholders as FunctionConnect } from '/snippets/connection.mdx';

Some columns cannot be written as a SQL expression: an embedding, a model call,
an image decode. A function is a Python callable registered with the database,
versioned, and then applied as a column. The code runs where the data is
rather than pulling rows to your process.

Registering and running a function needs a LanceDB Enterprise deployment with
Functions enabled, which they are not by default. On a
local connection `create_function` raises `NotImplementedError`; to derive a
column locally, declare a
[SQL computed column](/feature-engineering/add-computed-column) instead.

<Note>
In SQL, a function is registered with [`CREATE FUNCTION`](/sql/ddl/function#create) and listed with [`SHOW FUNCTIONS`](/sql/ddl/function#show).
</Note>

Functions are the way to fill columns with Python in LanceDB Enterprise. If you
already use [Geneva](/geneva), it remains available; see
[Geneva and Functions](/geneva#functions) for how its concepts map onto these.

The steps below take a column of numbers to a filled column of their doubles.

## Define a function {#define}

`@udf` packages a scalar Python callable. Input and output types are inferred
from annotations, so `double` takes one non-nullable `float64` parameter,
`value`, and returns a non-nullable `float64`:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionDefine}
</CodeBlock>

The decorated object stays callable. Calling it runs the original function in
your process, without the database, so unit tests stay ordinary tests:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionLocalCall}
</CodeBlock>

Where annotations cannot express the Arrow type precisely, pass `input_schema`
and `output_schema` together: one without the other is rejected. A scalar
output must be non-nullable, because a physical NULL is what marks a
computed-column row as unfilled. A named-struct output may declare nullable
fields: whether a row has a result at all is the struct's own validity, which
Enterprise keeps in a separate internal column when it expands the result into
sibling columns, so a null field is a value rather than a missing row.

## Connect and create a table {#table}

The remaining steps run against your Enterprise deployment. Replace the database
URI, API key, region and endpoint below with your deployment details. In application
code, load credentials from environment variables or a secrets manager:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionConnect}
</CodeBlock>

For an ordinary stored input column, the Arrow type must match the parameter.
A nullable column requires a nullable parameter, such as `Optional[float]`;
a non-nullable column can bind to either. This example uses `value: float`,
so the table declares `value` non-nullable.

Computed-column inputs depend on the Enterprise server version. Where supported,
they bind to non-nullable parameters even though the physical column is
nullable: refresh skips rows whose computed input is still null. Check your
deployment's support before using another computed column as an input.

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionCreateTable}
</CodeBlock>

## Register it {#register}

Registration returns a job; the immutable version exists once the job succeeds.

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionRegister}
</CodeBlock>

`create_function` is the blocking equivalent. A registered version is fetched
with `db.get_function(name, version=...)`.

<Note>
**Enterprise job type:** `create_function`. Registration returns a job because
the function is baked into an image before the version becomes usable.
</Note>

## Apply it as a column {#apply}

Calling the registered version binds each parameter, by name, to a column with
`col(...)`. `add_columns` declares the result as a new column:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionApply}
</CodeBlock>

The column is declared empty; the function has not run yet. A mapping with one
function application keeps its scalar or named-struct result in the named
column. A bare named-struct application expands its result fields as one atomic
binding, with aliases from `rename(columns=...)`.

## Fill the column {#refresh}

A refresh runs the function on the cluster as a [job](/basics/jobs). `wait()`
blocks until the job finishes and returns its result, including how many rows
it filled:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionRefresh}
</CodeBlock>

The filled values are read like any other column:

<CodeBlock filename="Python" language="Python" icon="python">
{FunctionInspect}
</CodeBlock>

Filling then works as it does for a SQL computed column: a later refresh fills
rows added since the last one and recomputes rows whose inputs changed, and
leaves filled rows with unchanged inputs alone. See
[Refresh a column](/feature-engineering/refresh-column).

## Read and produce blobs {#blobs}

A Function can bind a [Blob v2 column](/management/table#managed-blobs) and
return another blob. In row mode, a top-level blob parameter arrives as Python
`bytes`, or `None` for a nullable input. The worker reads the payload before
calling your function. Return bytes and let the destination table store the
Managed payload and descriptor.

Declare blob inputs and outputs explicitly. This example reverses the bytes
to demonstrate the round trip without requiring an image library. Here `db`
is an Enterprise database and `table` contains an `image` Blob v2 column
with table schema metadata `lance.blob.managed=true`, as in the
[table creation example](/management/table#managed-blobs-read-write):

```python
from datetime import timedelta

import lancedb
from lancedb import col
from lancedb.functions import udf
import pyarrow as pa

@udf(
    name="reverse_image",
    python_version="3.12",
    pip=["pyarrow==26.0.0"],
    input_schema=pa.schema([lancedb.blob("image")]),
    output_schema=lancedb.blob("reversed", nullable=False),
)
def reverse_image(image):
    return b"missing" if image is None else image[::-1]

function = db.create_function_async(reverse_image).wait(
    timeout=timedelta(minutes=15)
)
table.add_columns({"reversed": function(image=col("image"))})
result = table.refresh_column_async("reversed").wait(
    timeout=timedelta(minutes=15)
)
assert result.rows_failed == 0

hits = table.search().select(["id", "reversed"]).with_row_id(True).to_arrow()
values = table.fetch_blobs("reversed", hits).to_pylist()
```

An empty byte string is a filled result. A scalar output cannot be nullable,
because a physical NULL marks an unfilled computed row. To return a nullable
blob as a value, use a named-struct output schema with a nullable blob field;
the struct itself records whether the result has been filled. Read a nested
blob with `table.fetch_blobs("payload.image", hits)`.

On deployments that support computed-column inputs, another Function can bind
to `col("reversed")`. Refresh the producer first, then the consumer. Once
outputs are current, a later refresh fills appended rows. A filtered refresh
preserves payload references for unselected Managed outputs. An unfiltered
refresh can recompute a whole fragment when it cannot establish that the
fragment's outputs are current.

Function output blobs follow the same compaction and cleanup lifecycle as
ingested Managed blobs. Retained versions continue to own their payloads;
cleanup reclaims an object only after its retained references are gone.

### Draft: incremental blob inputs (upcoming)

This feature requires an SDK that exports `BlobReader` and a compatible
Enterprise runtime with reader input support enabled. Coordinate the runtime
upgrade before registering or executing reader Functions.

Annotate a row Function parameter with `BlobReader` to read a Blob v2 input
incrementally. The signature selects the access mode without another
registration option. For example, read the first MP4 box type without loading
the whole video:

```python
from lancedb import BlobReader, udf


@udf
def first_box_type(video: BlobReader) -> bytes:
    video.seek(4)
    return video.read(4)
```

The worker passes a seekable reader backed by the stored blob. `read(size)`
reads from the current cursor; `read_range(offset, length)` reads a byte range
without moving it. `size()` returns the payload size without reading it.
Each row call receives its own reader, which closes when the call returns or
raises. Do not retain it for another invocation.

Use `video: bytes` when the Function needs the complete payload. Bytes inputs
materialize before invocation and remain subject to eager input limits.
Reader inputs keep the unread payload out of the input batch, but calling
`video.read()` without a size loads all remaining bytes in Python. Reader
inputs are supported in row mode. Return values remain ordinary values such
as `bytes`; this feature does not add streamed outputs.

## Declare the environment {#environment}

The function runs remotely, so its dependencies are part of its definition:

| | |
| --- | --- |
| `pip` | requirements for the remote environment |
| `conda`, `conda_channels` | conda packages instead of pip; channels in priority order |
| `env` | environment variables |
| `python_version` | remote Python major/minor, defaulting to yours |
| `gpu` | whether every execution requires a GPU |

These are part of the **immutable function version**. Changing a dependency
produces a new version rather than altering the one already registered, so a
column filled by version 1 is unaffected by version 2.

Declaring them is not optional bookkeeping. If the worker environment differs
from the one the function was written against, the failures are among the
hardest to diagnose in the system, because the error rarely names the cause.

### Symptoms of an environment mismatch {#mismatch-symptoms}

| Symptom | Likely cause |
| --- | --- |
| `TypeError: Enum.__new__() missing 1 required positional argument` | `attrs` version mismatch |
| `TypeError: Can't instantiate abstract class` | package structure differs between versions |
| `ArrowInvalid: cannot cast`, or serialization errors | NumPy 1.x against 2.x |
| `ModuleNotFoundError` on the worker | package installed locally but not declared |
| Model loading failures | PyTorch version mismatch |
| Permission denied | an API key present locally but absent from `env` |

The packages worth pinning explicitly, because a mismatch in them produces the
errors above rather than a clear one: **numpy, torch, pyarrow, attrs,
pydantic**.

<Note>
The client version defaults `python_version`, so a function registered from a
3.11 environment runs on 3.11 unless you say otherwise. That default is usually
right and occasionally surprising. Pin it when the function is sensitive to it.
</Note>

## Failure handling {#failures}

A function that raises fails the job. There is no per-row retry or skip policy
in the function definition: `@udf` takes no error-handling parameter, so a
function that calls a flaky external service should handle that itself, inside
the function body.

Letting the job fail is the other option: fix the cause and refresh again. The
next refresh [skips rows that are filled and current](/feature-engineering/refresh-column#idempotent),
but how much of the failed job's work was kept is not guaranteed, so do not
count on the retry costing only the rows that did not complete.

## What gets packaged {#packaging}

<Warning>
The artifact is a **snapshot**: the function source, plus exactly the
module-level names it references (modules as imports, importable classes and
functions as imports, literals inlined).

Code that reaches the module namespace another way is rejected where it can be
seen and unsupported where it cannot: `globals()`, `eval`, `sys.modules`,
`builtins`. Closures and a non-standard `__builtins__` are rejected outright.
</Warning>

In practice this means a function should be self-contained: take its inputs as
arguments, import what it needs at module level, and avoid capturing state from
the surrounding scope.

## Drop it {#drop}

`db.drop_function(name, version=...)` drops the function's name and the object
it is bound to; `version` must be a version that exists. It returns `False` if
the name was already gone. Deleting the object's content can finish after the
call returns, as a [background job](/maintenance/background-jobs).
`drop_function_async` returns that job with the result, so you can wait for it.

A deployment refuses the drop unless it also allows changes to a function's
lifecycle, which is a separate setting, off by default even where Functions are
enabled.

## What to read next {#next}

<CardGroup cols={2}>
  <Card title="Computed columns" icon="table-columns" href="/feature-engineering/refresh-column">
    Refresh semantics, and the SQL alternative.
  </Card>
  <Card title="Embedding functions" icon="vector-square" href="/embedding">
    Built-in embedding providers, which need no registration.
  </Card>
</CardGroup>