mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 07:12:27 +00:00
docs: publish the Function pages only once Functions launch
The two Function pages declare `launch: functions`, and every other page marks what it says about Functions, so the assembled site leaves them out until the launch. Before it, the pages that offered Functions for Python work point Enterprise readers at Geneva instead. The SQL Function example now uses Arrow type names, which the server requires, and states the 3.12 default for `python_version`.
This commit is contained in:
Brendan Clement
@@ -90,7 +90,8 @@ path.
|
||||
scales for interactive traffic without also scaling indexing capacity, and heavy
|
||||
background work does not steal resources from user queries. Because a server
|
||||
exists, so do the things that need one: [SQL](/sql),
|
||||
[functions](/management/function), server-side jobs, and
|
||||
[functions](/management/function), {/* if functions launched */}
|
||||
server-side jobs, and
|
||||
[authorization](/governance/authorization) per object rather than per bucket.
|
||||
|
||||
**What it is good at.** Many concurrent queries, continuous ingestion and
|
||||
@@ -108,7 +109,7 @@ terabytes.
|
||||
| Indexing and compaction | when you call it | dedicated indexers |
|
||||
| [Jobs](/basics/jobs) | run in-process, no server record | real jobs: id, status, events, cancel |
|
||||
| [SQL](/sql) | unavailable | Flight SQL endpoint |
|
||||
| [Functions](/management/function) | unavailable | run on workers |
|
||||
| [Functions](/management/function) {/* if functions launched */} | unavailable | run on workers |
|
||||
| Access control | object-store permissions | [per-object grants](/governance/authorization) |
|
||||
| Scale bound | your process | the cluster |
|
||||
| What you operate | nothing | a deployment |
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "Lance Catalog"
|
||||
sidebarTitle: "Lance Catalog"
|
||||
description: "The naming layer: what objects exist, how they nest, and who may reach them."
|
||||
icon: sitemap
|
||||
keywords: ["catalog", "namespace", "hierarchy", "governance", "database", "table", "view", "function"]
|
||||
keywords: ["catalog", "namespace", "hierarchy", "governance", "database", "table", "view"]
|
||||
---
|
||||
|
||||
A catalog answers one question: **what exists, and what is it called.** It holds
|
||||
@@ -25,12 +25,23 @@ resolves the same names to the same tables.
|
||||
| [Table](/management/table) | Rows and columns, versioned. What most operations act on. |
|
||||
| [Materialized view](/management/materialized-view) | A table whose contents come from a query, computed and stored. |
|
||||
| [View](/management/view) | A saved query. Computed when read, not stored. |
|
||||
| [Function](/management/function) | Registered code, applied to produce column values. |
|
||||
| [Function](/management/function) {/* if functions launched */} | Registered code, applied to produce column values. |
|
||||
|
||||
Views and functions exist only on an Enterprise deployment that enables them.
|
||||
Views exist only on an Enterprise deployment that enables them.
|
||||
So do functions. {/* if functions launched */}
|
||||
Materialized views also exist in the embedded library, and a deployment has to
|
||||
enable them too.
|
||||
|
||||
{/* if functions not launched */}
|
||||
```
|
||||
database
|
||||
└── namespace
|
||||
├── table
|
||||
├── materialized view (a table, defined by a query)
|
||||
└── view (a query, not stored)
|
||||
```
|
||||
{/* end if */}
|
||||
{/* if functions launched */}
|
||||
```
|
||||
database
|
||||
└── namespace
|
||||
@@ -39,6 +50,7 @@ database
|
||||
├── view (a query, not stored)
|
||||
└── function
|
||||
```
|
||||
{/* end if */}
|
||||
|
||||
A namespace can contain further namespaces, so the hierarchy goes as deep as
|
||||
your organisation needs. A table is reached by its path through it:
|
||||
|
||||
@@ -14,8 +14,8 @@ availability:
|
||||
---
|
||||
|
||||
Some operations take longer than a request should. Building an index over a
|
||||
large table, refreshing a computed column, registering a function -- these run
|
||||
on the server after the call that started them has returned.
|
||||
large table or refreshing a computed column, for example, runs on the server
|
||||
after the call that started it has returned.
|
||||
|
||||
Each is submitted the same way and returns the same handle.
|
||||
|
||||
@@ -36,7 +36,7 @@ a **job** immediately.
|
||||
| --- | --- | --- |
|
||||
| `create_index` | `create_index_async` | `CREATE INDEX` |
|
||||
| `refresh_column` | `refresh_column_async` | `REFRESH COLUMN` |
|
||||
| `create_function` | `create_function_async` | `CREATE FUNCTION` |
|
||||
| `create_function` {/* if functions launched */} | `create_function_async` | `CREATE FUNCTION` |
|
||||
| `drop_table` | `drop_table_async` | `DROP TABLE` |
|
||||
|
||||
<Note>
|
||||
@@ -76,8 +76,8 @@ returns the failed state rather than raising, which is what you want in a
|
||||
polling loop.
|
||||
|
||||
In Python `wait` takes an optional timeout and returns the operation's typed
|
||||
result -- a `RefreshColumnJobResult`, a `FunctionVersion`, or `None` for an
|
||||
operation that has no result:
|
||||
result -- a `RefreshColumnJobResult`, for example, or `None` for an operation
|
||||
that has no result:
|
||||
|
||||
```python
|
||||
from datetime import timedelta
|
||||
|
||||
@@ -54,10 +54,10 @@ operation, not a separate service holding a second copy of your embeddings. See
|
||||
### Compute {#compute}
|
||||
|
||||
Deriving new columns and tables from what is already stored:
|
||||
[computed columns](/feature-engineering/add-computed-column),
|
||||
[functions](/management/function) that run Python where the data is, and
|
||||
[computed columns](/feature-engineering/add-computed-column) and
|
||||
[materialized views](/management/materialized-view) that persist a query's
|
||||
result.
|
||||
[Functions](/management/function) run Python where the data is. {/* if functions launched */}
|
||||
|
||||
The reason this belongs in the lakehouse rather than in a pipeline outside it is
|
||||
the format. Adding a derived column writes only that column -- the images and
|
||||
|
||||
@@ -97,7 +97,7 @@ not another.
|
||||
| SQL on a local database | unavailable; `execute_query` raises |
|
||||
| SQL on a deployment | needs its Flight SQL endpoint enabled and reachable |
|
||||
| SQL methods in the TypeScript SDK | none; use a Flight SQL client |
|
||||
| Function methods in the TypeScript SDK | none; use Python or SQL |
|
||||
| Function methods in the TypeScript SDK {/* if functions launched */} | none; use Python or SQL |
|
||||
| Vector search in SQL | [depends on the deployment](/basics/sql#no-vector-sql); the query builder always has it |
|
||||
| Geospatial `st_*` functions | behind a build feature LanceDB does not ship |
|
||||
| `tableExists` / `namespaceExists` in TypeScript | not present |
|
||||
|
||||
@@ -90,7 +90,7 @@ hybrid search has no SQL form. See [Semantic Search](/search).
|
||||
|
||||
## When the work outlives the call {#jobs}
|
||||
|
||||
Some operations take longer than a query should: building an index, refreshing a
|
||||
computed column, registering a function. Those return a **job** rather than
|
||||
Some operations take longer than a query should: building an index, or
|
||||
refreshing a computed column. Those return a **job** rather than
|
||||
rows, and a job is a different handle with a different shape. See
|
||||
[Jobs](/basics/jobs).
|
||||
|
||||
@@ -42,9 +42,9 @@ Not every SDK covers the same surface, though:
|
||||
| Feature engineering, jobs | yes | partly |
|
||||
| Runs against a local database | yes | yes |
|
||||
|
||||
Neither SDK aggregates or joins, and TypeScript has no methods for
|
||||
[functions](/management/function). Reach those with SQL on a deployment
|
||||
instead -- see [SQL](/basics/sql).
|
||||
Neither SDK aggregates or joins.
|
||||
TypeScript has no methods for [functions](/management/function). {/* if functions launched */}
|
||||
Reach those with SQL on a deployment instead -- see [SQL](/basics/sql).
|
||||
|
||||
## REST API {#rest}
|
||||
|
||||
|
||||
@@ -21,7 +21,7 @@ What it adds over the embedded form:
|
||||
| | |
|
||||
| --- | --- |
|
||||
| [SQL](/sql) | an Arrow Flight SQL endpoint |
|
||||
| [Functions](/management/function) | Python running on workers |
|
||||
| [Functions](/management/function) {/* if functions launched */} | Python running on workers |
|
||||
| [Jobs](/basics/jobs) | server-side work with ids, status and events |
|
||||
| [Authorization](/governance/authorization) | per-object grants |
|
||||
| Scale | a cluster rather than your process |
|
||||
|
||||
@@ -38,9 +38,12 @@ including that you can travel back to before the column existed.
|
||||
|
||||
- **SQL expressions in `transforms`** are evaluated *now*, for every row, as
|
||||
part of the call. Good for a small table, or a column you want populated
|
||||
immediately. A `FunctionApplication` passed through `transforms` instead
|
||||
immediately.
|
||||
{/* if functions launched */}
|
||||
A `FunctionApplication` passed through `transforms` instead
|
||||
declares empty Function-bound columns, which a refresh fills; that needs a
|
||||
deployment with Functions enabled, and the embedded library refuses it.
|
||||
{/* end if */}
|
||||
- **`computed`** stores the expression and evaluates nothing. The column is
|
||||
committed empty and filled later by
|
||||
[`refresh_column`](/feature-engineering/refresh-column).
|
||||
@@ -81,11 +84,18 @@ reshape later. Dropping the computed column releases the constraint.
|
||||
|
||||
<Badge color="red">Enterprise</Badge>
|
||||
|
||||
{/* if functions launched */}
|
||||
When the value cannot be expressed in SQL -- an embedding, a model call, an
|
||||
image decode -- the column is filled by a registered
|
||||
[function](/management/function) instead of an expression. Declaring it looks
|
||||
the same; what differs is what fills it. It needs a deployment with Functions
|
||||
enabled.
|
||||
{/* end if */}
|
||||
{/* if functions not launched */}
|
||||
When the value cannot be expressed in SQL -- an embedding, a model call, an
|
||||
image decode -- [Geneva](/geneva), the feature engineering package for LanceDB
|
||||
Enterprise, fills the column with a Python function instead.
|
||||
{/* end if */}
|
||||
|
||||
## What to read next {#next}
|
||||
|
||||
@@ -93,7 +103,14 @@ enabled.
|
||||
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
|
||||
Fill the rows that lack values.
|
||||
</Card>
|
||||
{/* if functions launched */}
|
||||
<Card title="Functions" icon="code" href="/management/function">
|
||||
Fill a column with Python.
|
||||
</Card>
|
||||
{/* end if */}
|
||||
{/* if functions not launched */}
|
||||
<Card title="Geneva" icon="layer-group" href="/geneva">
|
||||
Fill a column with Python on LanceDB Enterprise.
|
||||
</Card>
|
||||
{/* end if */}
|
||||
</CardGroup>
|
||||
|
||||
@@ -108,10 +108,17 @@ exist when that does not fit:
|
||||
- A [computed column](/feature-engineering/add-computed-column) declares the
|
||||
expression and leaves the column empty, so you choose when the work happens.
|
||||
That is the right default on a large table.
|
||||
{/* if functions launched */}
|
||||
- A [function](/management/function) fills a column with Python, for values SQL
|
||||
cannot express -- an embedding, a model call, an image decode.
|
||||
{/* end if */}
|
||||
{/* if functions not launched */}
|
||||
- On LanceDB Enterprise, [Geneva](/geneva) fills a column with Python, for
|
||||
values SQL cannot express -- an embedding, a model call, an image decode.
|
||||
{/* end if */}
|
||||
|
||||
On a deployment, both have to be enabled there.
|
||||
A deployment has to enable computed columns before you use them.
|
||||
Functions need enabling there too. {/* if functions launched */}
|
||||
|
||||
## What to read next {#next}
|
||||
|
||||
|
||||
@@ -3,15 +3,20 @@ title: "Feature engineering"
|
||||
sidebarTitle: "Overview"
|
||||
description: "Derive columns and tables from data already in the lakehouse, without moving it somewhere else to compute."
|
||||
icon: layer-group
|
||||
keywords: ["feature engineering", "computed columns", "materialized views", "functions", "lakehouse", "jobs"]
|
||||
keywords: ["feature engineering", "computed columns", "materialized views", "lakehouse", "jobs"]
|
||||
---
|
||||
|
||||
Feature engineering happens where the data already is. Rather than exporting a
|
||||
table, computing somewhere else and writing the result back, you declare what a
|
||||
column or a table should contain and let LanceDB fill it.
|
||||
|
||||
{/* if functions launched */}
|
||||
[Functions](/management/function) and
|
||||
[materialized views](/feature-engineering/materialized-views) are ordinary
|
||||
{/* end if */}
|
||||
{/* if functions not launched */}
|
||||
[Materialized views](/feature-engineering/materialized-views) are ordinary
|
||||
{/* end if */}
|
||||
[catalog objects](/basics/catalog), alongside tables and indexes -- created,
|
||||
versioned, and refreshed like anything else, not a separate system bolted
|
||||
alongside.
|
||||
@@ -48,7 +53,7 @@ idempotent and cheap -- but it will surprise you if you expect a spreadsheet.
|
||||
| --- | --- | --- | --- |
|
||||
| [Computed column](/feature-engineering/add-computed-column) | a new column on an existing table | a SQL expression | embedded, and deployments that enable it |
|
||||
| [Materialized view](/feature-engineering/materialized-views) | a new table derived by a query | a refresh | embedded, and deployments that enable it |
|
||||
| [Function](/management/function) | a computed column filled by Python | registered code | <Badge color="red">Enterprise</Badge> deployments that enable it |
|
||||
| [Function](/management/function) {/* if functions launched */} | a computed column filled by Python | registered code | <Badge color="red">Enterprise</Badge> deployments that enable it |
|
||||
|
||||
**Computed columns** are the common case: a column declared with a SQL
|
||||
expression, filled by [a refresh](/feature-engineering/refresh-column).
|
||||
@@ -57,12 +62,14 @@ expression, filled by [a refresh](/feature-engineering/refresh-column).
|
||||
aggregated view of a source -- and bring it up to date on each refresh. The
|
||||
result is an ordinary table: queryable, indexable, searchable.
|
||||
|
||||
{/* if functions launched */}
|
||||
**Functions** run Python where the data is, for work SQL cannot express:
|
||||
embedding a column, calling a model, decoding an image. Registered once,
|
||||
versioned, then applied as a column.
|
||||
|
||||
They compose. A materialized view can select computed columns, and a computed
|
||||
column can be filled by a registered function.
|
||||
{/* end if */}
|
||||
They compose. A materialized view can select computed columns.
|
||||
A computed column can be filled by a registered function. {/* if functions launched */}
|
||||
|
||||
## Where the work runs {#where}
|
||||
|
||||
@@ -83,15 +90,18 @@ from a different process -- the connection does not have to stay open.
|
||||
|
||||
That relocation is the main thing to understand about Enterprise feature
|
||||
engineering. The API is the same; what changes is that the code runs somewhere
|
||||
else, which is also why a function must declare the environment it needs.
|
||||
else.
|
||||
That is also why a function must declare the environment it needs. {/* if functions launched */}
|
||||
|
||||
## Recovering from a failure {#failures}
|
||||
|
||||
{/* if functions launched */}
|
||||
A function that raises fails the job. There is no per-row retry or skip policy
|
||||
in the function definition, so a function calling a flaky external service
|
||||
should handle that itself, inside the function body.
|
||||
|
||||
The alternative is to let the job fail, fix the cause, and run the refresh
|
||||
{/* end if */}
|
||||
When a refresh fails, fix the cause and run the refresh
|
||||
again. The new refresh fills what is still empty or stale. How much of the
|
||||
failed run was kept depends on where it ran (see
|
||||
[Refresh a column](/feature-engineering/refresh-column#failures)), and a retry
|
||||
@@ -110,7 +120,9 @@ only the rows that did not complete.
|
||||
<Card title="Refresh a materialized view" icon="table" href="/feature-engineering/materialized-views">
|
||||
Bring a derived table up to date with its source.
|
||||
</Card>
|
||||
{/* if functions launched */}
|
||||
<Card title="Functions" icon="code" href="/management/function">
|
||||
Register Python to fill a column with.
|
||||
</Card>
|
||||
{/* end if */}
|
||||
</CardGroup>
|
||||
|
||||
@@ -69,9 +69,8 @@ REFRESH COLUMN embedding ON documents WHERE created_at > '2026-01-01'
|
||||
```
|
||||
|
||||
<Note>
|
||||
**Enterprise job type:** `computed_refresh` for a SQL-expression column,
|
||||
`function_refresh` for one filled by a registered
|
||||
[function](/management/function).
|
||||
**Enterprise job type:** `computed_refresh` for a SQL-expression column.
|
||||
`function_refresh` for one filled by a registered [function](/management/function). {/* if functions launched */}
|
||||
</Note>
|
||||
|
||||
## As a job {#async}
|
||||
@@ -128,10 +127,12 @@ that were already written -- for example when the record of the inputs they
|
||||
were computed from was not saved -- so do not count on a retry costing only the
|
||||
rows that did not complete.
|
||||
|
||||
{/* if functions launched */}
|
||||
For a column filled by a [function](/management/function), the usual cause is
|
||||
the worker environment rather than the data. See
|
||||
[environment mismatches](/management/function#mismatch-symptoms) for the
|
||||
symptoms, which rarely name their cause.
|
||||
{/* end if */}
|
||||
|
||||
## What to read next {#next}
|
||||
|
||||
|
||||
@@ -44,6 +44,17 @@ not a narrower one.
|
||||
|
||||
Objects are addressed by kind and path:
|
||||
|
||||
{/* if functions not launched */}
|
||||
```
|
||||
system
|
||||
database:db
|
||||
namespace:db:ns
|
||||
table:db:ns$table
|
||||
view:db:ns$view
|
||||
secret:db:ns$secret
|
||||
```
|
||||
{/* end if */}
|
||||
{/* if functions launched */}
|
||||
```
|
||||
system
|
||||
database:db
|
||||
@@ -53,6 +64,7 @@ view:db:ns$view
|
||||
function:db:ns$function
|
||||
secret:db:ns$secret
|
||||
```
|
||||
{/* end if */}
|
||||
|
||||
A nested namespace joins its path with `$`, as in `namespace:db:parent$child`
|
||||
or `table:db:parent$child$table`.
|
||||
@@ -69,7 +81,7 @@ table alone is not enough to traverse to it.
|
||||
| `USAGE` | system | listing jobs and their events, and looking up other identities |
|
||||
| `USAGE` | database | accessing the namespaces inside |
|
||||
| `USAGE` | namespace | accessing the tables inside |
|
||||
| `USAGE` | function | binding it to a column, and refreshing a column it computes |
|
||||
| `USAGE` {/* if functions launched */} | function | binding it to a column, and refreshing a column it computes |
|
||||
| `USAGE` | secret | binding and resolving it |
|
||||
| `SELECT` | table, view | reading rows |
|
||||
| `INSERT` | table | appending rows |
|
||||
@@ -81,7 +93,7 @@ table alone is not enough to traverse to it.
|
||||
| `CREATE_TABLE` | namespace | creating a table in it |
|
||||
| `CREATE_MATERIALIZED_VIEW` | namespace | creating a materialized view in it |
|
||||
| `CREATE_VIEW` | namespace | creating a non-materialized view in it |
|
||||
| `CREATE_FUNCTION` | namespace | registering a function in it |
|
||||
| `CREATE_FUNCTION` {/* if functions launched */} | namespace | registering a function in it |
|
||||
| `CREATE_SECRET` | namespace | creating a secret in it |
|
||||
| `OPERATE` | system | describing and controlling jobs, managing caches, and reading server configuration |
|
||||
|
||||
|
||||
@@ -64,8 +64,10 @@ export, its events also survive in the exported table.
|
||||
|
||||
| Job type | What it does |
|
||||
| --- | --- |
|
||||
| `drop_object` | Deletes a dropped catalog object's content, then clears it from the database's cleanup backlog. Covers namespaces, views, and functions -- everything [`drop_table`](/management/table#drop) does not, since that one fences a directory with its own marker instead. |
|
||||
| `drop_object` | Deletes a dropped catalog object's content, then clears it from the database's cleanup backlog. Covers namespaces, views and other catalog objects -- everything [`drop_table`](/management/table#drop) does not, since that one fences a directory with its own marker instead. |
|
||||
|
||||
You trigger this indirectly, by dropping a [namespace](/management/namespace),
|
||||
[view](/sql/ddl/view#drop), or [function](/management/function#drop) -- the teardown
|
||||
itself runs as this job rather than inline with the call.
|
||||
a [view](/sql/ddl/view#drop),
|
||||
or a [function](/management/function#drop), {/* if functions launched */}
|
||||
for example -- the teardown itself runs as this job rather than inline with the
|
||||
call.
|
||||
|
||||
@@ -49,7 +49,7 @@ storage and does the work itself. The third talks to a **deployment**.
|
||||
| Table handle | `LanceTable` | `RemoteTable` |
|
||||
| [SQL](/basics/sql) | -- | where enabled |
|
||||
| Server-side [jobs](/basics/jobs) | -- | yes |
|
||||
| [Functions](/management/function) | -- | where enabled |
|
||||
| [Functions](/management/function) {/* if functions launched */} | -- | where enabled |
|
||||
| [Authorization](/governance/authorization) | storage permissions | granted per object, where configured |
|
||||
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ sidebarTitle: "Function"
|
||||
description: "Register a Python function once and apply it as a column, for work SQL cannot express."
|
||||
icon: code
|
||||
keywords: ["udf", "functions", "create_function", "embedding", "feature engineering"]
|
||||
launch: functions
|
||||
availability:
|
||||
oss: unavailable
|
||||
enterprise: varies
|
||||
|
||||
@@ -79,6 +79,7 @@ it current.
|
||||
enables them; the SDK methods send the view's SQL query to the deployment. See
|
||||
[Views](/management/view).
|
||||
|
||||
{/* if functions launched */}
|
||||
### Functions {#functions}
|
||||
|
||||
| | Python | TypeScript | SQL |
|
||||
@@ -92,6 +93,7 @@ enables them; the SDK methods send the view's SQL query to the deployment. See
|
||||
a deployment that enables functions, so there is no local form. The TypeScript
|
||||
SDK has no function methods; use Python or SQL.
|
||||
See [Functions](/management/function).
|
||||
{/* end if */}
|
||||
|
||||
### Jobs {#jobs}
|
||||
|
||||
|
||||
@@ -225,8 +225,8 @@ also makes shallow clones only.
|
||||
**Enterprise job type:** `drop_table`. On a deployment that drops tables in the
|
||||
background, fences the table's directory with a marker and deletes its content
|
||||
as a job -- see
|
||||
[Background jobs](/maintenance/background-jobs#teardown) for the namespace,
|
||||
view, and function equivalent.
|
||||
[Background jobs](/maintenance/background-jobs#teardown) for the namespace
|
||||
and view equivalent.
|
||||
</Note>
|
||||
|
||||
## Dropping {#drop}
|
||||
|
||||
@@ -410,9 +410,8 @@ job = table.refresh_column_async("combat_score")
|
||||
job.wait()
|
||||
```
|
||||
|
||||
See [Computed columns](/feature-engineering/add-computed-column) for expressions,
|
||||
and [Functions](/management/function) for filling a column with Python
|
||||
-- an embedding model, a captioner, anything you can register.
|
||||
See [Computed columns](/feature-engineering/add-computed-column) for expressions.
|
||||
See [Functions](/management/function) for filling a column with Python. {/* if functions launched */}
|
||||
|
||||
## 7\. Store multimodal data {#7-store-multimodal-data}
|
||||
|
||||
|
||||
@@ -4,6 +4,7 @@ sidebarTitle: "Function"
|
||||
description: "CREATE FUNCTION, SHOW FUNCTIONS, and DROP FUNCTION."
|
||||
icon: function
|
||||
keywords: ["create function", "drop function", "show functions", "sql", "udf"]
|
||||
launch: functions
|
||||
---
|
||||
|
||||
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
|
||||
@@ -11,12 +12,17 @@ keywords: ["create function", "drop function", "show functions", "sql", "udf"]
|
||||
## CREATE FUNCTION {#create}
|
||||
|
||||
```sql
|
||||
CREATE FUNCTION f(x BIGINT) RETURNS BIGINT
|
||||
CREATE FUNCTION f(x int64) RETURNS int64
|
||||
LANGUAGE python
|
||||
AS 'def f(x): return x * 2'
|
||||
WITH (pip = 'numpy==1.26.4', python_version = '3.11')
|
||||
```
|
||||
|
||||
Parameter and result types are Arrow type names, not SQL ones: `int64`,
|
||||
`float64`, `utf8`, `bool`, `date32`, and lists such as `list<float32>` or
|
||||
`fixed_size_list<float32, 768>`. `BIGINT` is refused. A parameter accepts nulls
|
||||
unless it is declared `NOT NULL`.
|
||||
|
||||
The body is Python source. Its dependencies are declared with it and are part
|
||||
of the **immutable function version** -- changing one produces a new version
|
||||
rather than altering the registered one, so a column filled by an earlier
|
||||
@@ -27,7 +33,7 @@ version is unaffected.
|
||||
| `pip` | pip requirements for the worker environment |
|
||||
| `conda` | conda packages instead of pip |
|
||||
| `conda_channels` | channels in priority order; requires `conda` |
|
||||
| `python_version` | worker Python version, defaulting to yours |
|
||||
| `python_version` | worker Python version; `3.12` when omitted, whatever Python you send the statement from |
|
||||
| `gpu` | `'true'` when every execution needs a GPU |
|
||||
|
||||
**Errors and limits**
|
||||
|
||||
@@ -36,7 +36,7 @@ DROP SCHEMA `a$b` RESTRICT
|
||||
|
||||
<Warning>
|
||||
`CASCADE` is rejected: **"DROP SCHEMA CASCADE is not supported; drop the
|
||||
objects inside first."** A namespace's tables, views and functions must be
|
||||
objects inside first."** A namespace's tables, views and other objects must be
|
||||
dropped explicitly before the namespace itself can go. `RESTRICT` -- the
|
||||
default -- states that requirement rather than changing it.
|
||||
</Warning>
|
||||
|
||||
@@ -30,8 +30,8 @@ The table's columns and types, without a plan -- the SQL counterpart of
|
||||
|
||||
## ALTER TABLE {#alter}
|
||||
|
||||
`ALTER TABLE ... ADD COLUMN` covers three different things, distinguished by
|
||||
what follows the column name.
|
||||
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
||||
name.
|
||||
|
||||
### A plain column {#plain-column}
|
||||
|
||||
@@ -52,6 +52,7 @@ The column is committed empty and the expression is stored, not evaluated --
|
||||
declaring one on a billion-row table costs what declaring one on an empty table
|
||||
costs. [`REFRESH COLUMN`](/sql/dml/refresh) fills it.
|
||||
|
||||
{/* if functions launched */}
|
||||
### A function column {#function-column}
|
||||
|
||||
```sql
|
||||
@@ -86,6 +87,7 @@ ALTER TABLE t ADD COLUMN c AS f(x) WITH (num_workers = '2')
|
||||
`WITH` on a plain computed column accepts only `version`. Resource options such
|
||||
as `num_workers` apply to function columns, where there are workers to size.
|
||||
</Note>
|
||||
{/* end if */}
|
||||
|
||||
### Table properties {#properties}
|
||||
|
||||
|
||||
@@ -52,11 +52,13 @@ REFRESH COLUMN embedding ON documents
|
||||
`num_workers` sets a count; the budget options size each worker. They are
|
||||
alternatives rather than a pair.
|
||||
|
||||
{/* if functions launched */}
|
||||
<Note>
|
||||
`error_policy` and `num_workers` are refused for a column filled by a registered
|
||||
[function](/management/function) with its own resource budget -- use the CPU,
|
||||
memory and GPU options there.
|
||||
</Note>
|
||||
{/* end if */}
|
||||
|
||||
## REFRESH MATERIALIZED VIEW {#refresh-materialized-view}
|
||||
|
||||
|
||||
@@ -101,7 +101,7 @@ operations that have no SQL form, such as a merge insert -- see
|
||||
| [View](/sql/ddl/view) | `CREATE VIEW`, `DROP VIEW` |
|
||||
| [Materialized view](/sql/ddl/materialized-view) | `CREATE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS` |
|
||||
| [Index](/sql/ddl/index) | `CREATE INDEX`, `DROP INDEX` |
|
||||
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
|
||||
| [Function](/sql/ddl/function) {/* if functions launched */} | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
|
||||
| [Job](/sql/ddl/job) | `SHOW JOBS`, `SHOW ERRORS`, `PAUSE`/`RESUME`/`CANCEL JOB` |
|
||||
|
||||
### DML {#dml}
|
||||
|
||||
Reference in New Issue
Block a user