mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-16 07:36:50 +00:00
Migrates SQL-backed catalog listing APIs to `LIMIT ... FROM '<page-token>'` and server-issued Arrow continuation metadata. Namespace-specific APIs send an explicit `IN` schema clause, including `IN "public"` for root, so raw SQL can default to all namespaces. ```sql SHOW VIEWS LIMIT 100; SHOW VIEWS IN team$ml LIMIT 100; SHOW VIEWS IN team$ml LIMIT 100 FROM '<page-token>'; SHOW VIEWS IN other_db.team$ml LIMIT 100; ``` SDK clients forward opaque tokens unchanged, preserve empty continuations, and decode identifier columns by name. Removes legacy cursor reconstruction and updates the SQL reference and Python Flight fixture. Public docs cover optional direct schema paths, all-namespace defaults, and the shared pagination syntax across every catalog listing.
195 lines
7.5 KiB
Plaintext
195 lines
7.5 KiB
Plaintext
---
|
|
title: "Function"
|
|
tag: "Early access"
|
|
sidebarTitle: "Function"
|
|
description: "Create, describe, alter, list, and drop registered functions with SQL."
|
|
icon: function
|
|
keywords: ["create function", "drop function", "show functions", "sql", "udf"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
<Badge color="purple">Early access</Badge>
|
|
|
|
[SQL runs in LanceDB Enterprise.](/sql)
|
|
|
|
## CREATE FUNCTION {#create}
|
|
|
|
{/* Draft for the function exist_ok release.
|
|
|
|
`CREATE FUNCTION IF NOT EXISTS` preserves an existing named Function version,
|
|
including its code, credentials and execution settings. In Python, pass
|
|
`exist_ok=True` to `create_function` or `create_function_async`. The default
|
|
retains the existing version-creation behavior.
|
|
|
|
*/}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE FUNCTION f(x int64) RETURNS int64
|
|
LANGUAGE python
|
|
AS 'def f(x): return x * 2'
|
|
WITH (pip = 'numpy==1.26.4', python_version = '3.11')
|
|
```
|
|
</CodeGroup>
|
|
|
|
A function takes more than one parameter; each one binds a separate column
|
|
or literal where the function is called:
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE FUNCTION ratio(numerator float64, denominator float64) RETURNS float64
|
|
LANGUAGE python
|
|
AS 'def ratio(numerator, denominator): return numerator / denominator'
|
|
```
|
|
</CodeGroup>
|
|
|
|
Parameter and result types are Arrow type names, not SQL ones: `int64`,
|
|
`float64`, `utf8`, `bool`, `date32`, and lists such as `list<float32>` or
|
|
`fixed_size_list<float32, 768>`. `BIGINT` is refused. A parameter accepts nulls
|
|
unless it is declared `NOT NULL`.
|
|
|
|
The body is Python source. Its dependencies are declared with it and are part
|
|
of the **immutable function version**. Changing one produces a new version
|
|
rather than altering the registered one, so a column filled by an earlier
|
|
version is unaffected.
|
|
|
|
| Option | |
|
|
| --- | --- |
|
|
| `pip` | pip requirements for the worker environment |
|
|
| `conda` | conda packages instead of pip |
|
|
| `conda_channels` | channels in priority order; requires `conda` |
|
|
| `apt` | OS packages to install in the worker environment |
|
|
| `image` | a specific container image to run the worker in, such as `'host/repo@sha256:...'` |
|
|
| `python_version` | worker Python version; `3.12` when omitted, whatever Python you send the statement from |
|
|
| `gpu` | `'true'` when every execution needs a GPU |
|
|
| `gpu_class` | which configured GPU class to run on, when `gpu = 'true'` |
|
|
| `result_stability` | `'input_and_context'` or `'per_call'` |
|
|
| `batch_size` | maximum rows per Function call |
|
|
| `row_timeout_seconds` | per-row (or per-batch, for a vectorized Function) execution timeout |
|
|
|
|
**Errors and limits**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `LANGUAGE` other than `python` | refused |
|
|
| `pip` and `conda` together | refused (pick one) |
|
|
| `conda_channels` without `conda` | refused |
|
|
| `gpu` set to anything but `'true'` | refused |
|
|
| An empty body | refused |
|
|
| Any other option | refused, and the message lists what is supported |
|
|
|
|
<Note>
|
|
**Enterprise job type:** `create_function`. Registration returns a job because
|
|
the function is baked into an image before the version becomes usable.
|
|
</Note>
|
|
|
|
## Artifact registration {#artifact}
|
|
|
|
`CREATE FUNCTION name FROM '<registration JSON>'` accepts the artifact,
|
|
signature, runtime, optional secret bindings, and result stability produced by
|
|
the SDK's registration request. Omit its `name` property because the SQL
|
|
identifier supplies it. Escape apostrophes in the JSON as doubled apostrophes.
|
|
|
|
By default, the command waits for its internal image-build job to publish the
|
|
version, then returns one UTF-8 `definition` column containing FunctionVersion
|
|
JSON. `WITH (wait = false)` instead returns the existing image-build registry
|
|
job ID in `job_id`, with `operation` and `status` columns. The SDK's ordinary
|
|
registration handle has type `query` and uses the SQL query UUID as its single
|
|
`id`; the internal image-build job has type `create_function`.
|
|
|
|
## DESCRIBE and ALTER FUNCTION {#describe-alter}
|
|
|
|
```sql
|
|
DESCRIBE FUNCTION embed WITH (version = '1', include_source = true)
|
|
DESCRIBE FUNCTION embed
|
|
```
|
|
|
|
DESCRIBE returns one `definition` JSON column for the exact version. When
|
|
`version` is omitted, it resolves and returns the latest currently visible
|
|
immutable revision.
|
|
`include_source` defaults to false; source-inclusive reads report an omission
|
|
when the stored artifact exceeds the inline source limit.
|
|
|
|
`ALTER FUNCTION name FROM '<pinned update JSON>'` accepts a `reference` with
|
|
name, namespace, object ID, location, version, and manifest digest, plus optional
|
|
`metadata` and `disabled` updates. The command checks that reference before
|
|
appending a revision and returns the resulting `definition` JSON.
|
|
|
|
## Call it {#call}
|
|
|
|
Once registered, a function is callable by name in any query, the same as a
|
|
built-in scalar function:
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
SELECT id, f(x) AS doubled FROM my_table
|
|
```
|
|
</CodeGroup>
|
|
|
|
This is also the fastest way to check a new registration before binding it to
|
|
a column: call it over a few rows, or over literal arguments, before
|
|
committing to [a computed column](/sql/ddl/table#computed-column) that fills
|
|
from it.
|
|
|
|
## SHOW FUNCTIONS {#show}
|
|
|
|
```sql
|
|
SHOW FUNCTIONS IN analytics.`team$ml` LIMIT 100;
|
|
SHOW FUNCTIONS IN analytics.`team$ml` LIMIT 100 FROM '<page-token>';
|
|
```
|
|
|
|
Returns two non-null string columns: `namespace_path`, a compact JSON array
|
|
of parent namespace components (`[]` for root), and `name`, the unqualified
|
|
object name. Omitting `IN` lists all authorized namespaces in the current database;
|
|
`IN team$ml` selects that exact path without a database qualifier; `IN public` selects
|
|
the current database's root. Listings require the relevant database and
|
|
namespace `USAGE` privileges.
|
|
|
|
`SHOW FUNCTIONS` lists each function once. Add `HISTORY` to list every
|
|
published revision with its metadata and definition, as `list_functions()`
|
|
does. History uses the same pagination suffix; numeric revision order is
|
|
encoded in the token, so revision `2` precedes `10`.
|
|
|
|
```sql
|
|
SHOW FUNCTIONS HISTORY IN analytics.`team$ml` LIMIT 100;
|
|
SHOW FUNCTIONS HISTORY IN analytics.`team$ml` LIMIT 100 FROM '<page-token>';
|
|
```
|
|
|
|
Page through results using `LIMIT n [FROM '<page-token>']`. Read the opaque
|
|
`lancedb.next_page_token` from the Arrow result schema metadata and pass it back
|
|
unchanged with the same object type and scope. Stop when no token is returned;
|
|
a short or empty page can still carry a continuation. See
|
|
[catalog listing pagination](/sql/pagination).
|
|
|
|
## DROP FUNCTION {#drop}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DROP FUNCTION embed WITH (version = '1')
|
|
DROP FUNCTION IF EXISTS embed WITH (version = '1')
|
|
```
|
|
</CodeGroup>
|
|
|
|
DROP removes only the requested revision from lookup and listing. Other
|
|
published revisions and their grants remain available. Dropping the last visible
|
|
revision releases the name and object binding; physical cleanup continues
|
|
separately. The result reports `dropped` or `not_found`. `IF EXISTS` allows a
|
|
missing name or a repeated drop of a previously published revision. A revision
|
|
that was never published under an existing name is an error.
|
|
|
|
All function object commands retain shared results even when general query-result
|
|
copying is disabled, so asynchronous polling can resume on another query node.
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Function" icon="function" href="/management/function">
|
|
Registering one from the SDK, and diagnosing environment mismatches.
|
|
</Card>
|
|
<Card title="Computed columns" icon="table-columns" href="/feature-engineering/add-computed-column">
|
|
Filling a column with a function instead of an expression.
|
|
</Card>
|
|
</CardGroup>
|