mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-20 18:52:28 +00:00
Brendan Clement
198812fd53
The existing docs have 254 findings from our shared Vale rules. Contributors would otherwise inherit these when editing the same lines. Clean up the prose across 80 pages: correct product names and a repeated word, simplify a few phrases, and replace dashes with punctuation that fits each sentence. The pinned Vale scan now reports zero findings, with no suppressions.
125 lines
4.5 KiB
Plaintext
125 lines
4.5 KiB
Plaintext
---
|
|
title: "Add a computed column"
|
|
sidebarTitle: "Computed Columns"
|
|
description: "Declare a column from a SQL expression. Nothing is evaluated until you refresh it."
|
|
icon: table-columns
|
|
keywords: ["computed columns", "add_columns", "transforms", "schema"]
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Both offerings support computed columns. Enterprise requires the deployment to enable them; the embedded library evaluates them in your process.
|
|
---
|
|
|
|
A computed column is declared with an expression and starts with no values.
|
|
Declaring it is cheap on any table size, because nothing is evaluated until
|
|
[a refresh](/feature-engineering/refresh-column) asks for it.
|
|
|
|
## Add a computed column {#declare}
|
|
|
|
```python
|
|
import lancedb
|
|
|
|
db = lancedb.connect("./.lancedb")
|
|
table = db.create_table("computed_demo", [{"x": 1}, {"x": 2}])
|
|
|
|
table.add_columns(computed={"doubled": "x * 2"})
|
|
# AddColumnsResult(version=2)
|
|
```
|
|
|
|
That commits a new table version with the column present and empty. It is an
|
|
ordinary table version, so the usual [versioning](/management/table#versioning) applies,
|
|
including that you can travel back to before the column existed.
|
|
|
|
## Compute now or on refresh {#computed-vs-transforms}
|
|
|
|
`add_columns` takes either, and the difference matters:
|
|
|
|
- **SQL expressions in `transforms`** are evaluated *now*, for every row, as
|
|
part of the call. Good for a small table, or a column you want populated
|
|
immediately.
|
|
{/* Draft for the Functions release:
|
|
A `FunctionApplication` passed through `transforms` instead
|
|
declares empty Function-bound columns, which a refresh fills; that needs a
|
|
deployment with Functions enabled, and the embedded library refuses it. */}
|
|
- **`computed`** stores the expression and evaluates nothing. The column is
|
|
committed empty and filled later by
|
|
[`refresh_column`](/feature-engineering/refresh-column).
|
|
|
|
`computed` and `transforms` cannot be combined in one call. Reach for `computed` when the table is
|
|
large enough that you would rather control when the work happens.
|
|
|
|
## The expression {#expressions}
|
|
|
|
The expression is SQL, evaluated over the row. It can reference any column in
|
|
the table, including another computed column:
|
|
|
|
```python
|
|
table.add_columns(computed={
|
|
"power_score": "cast(x as double) / 2.0",
|
|
"is_strong": "power_score > 0.5",
|
|
})
|
|
```
|
|
|
|
Declaration order is dependency order: a computed column may reference one
|
|
declared earlier in the same mapping, but not one declared after it.
|
|
|
|
The functions available are the ones the
|
|
[SQL dialect](/sql/functions) documents.
|
|
|
|
A literal in the expression has to fit the column it meets: `x / 2.0` is
|
|
refused when `x` is an integer column, so write `cast(x as double) / 2.0`.
|
|
|
|
<Warning>
|
|
In LanceDB Enterprise, keep the expression to top-level columns. A column
|
|
refresh can fail on an expression that reads a field inside a struct, such as
|
|
`stats.magic + stats.strength`, which the embedded library computes. Try a new
|
|
expression on a small table first.
|
|
</Warning>
|
|
|
|
## Change columns used by an expression {#input-constraints}
|
|
|
|
<Warning>
|
|
While a computed column's declaration references another column, that column
|
|
cannot be renamed, retyped, or dropped. Drop the computed column first
|
|
if you need to change one of its inputs.
|
|
</Warning>
|
|
|
|
This is worth knowing before deriving a column from something you expect to
|
|
reshape later. Dropping the computed column releases the constraint.
|
|
|
|
## Filling with Python instead of SQL {#functions}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
When the value cannot be expressed in SQL (an embedding, a model call, an
|
|
image decode), [Geneva](/geneva), the feature engineering package for LanceDB
|
|
Enterprise, fills the column with a Python function instead.
|
|
|
|
{/*
|
|
Draft for the Functions release. Replaces the paragraph above:
|
|
|
|
When the value cannot be expressed in SQL -- an embedding, a model call, an
|
|
image decode -- the column is filled by a registered
|
|
[function](/management/function) instead of an expression. Declaring it looks
|
|
the same; what differs is what fills it. It needs a deployment with Functions
|
|
enabled.
|
|
*/}
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
|
|
Fill the rows that lack values.
|
|
</Card>
|
|
<Card title="Geneva" icon="layer-group" href="/geneva">
|
|
Fill a column with Python on LanceDB Enterprise.
|
|
</Card>
|
|
{/*
|
|
Draft for the Functions release. Replaces the Geneva card above:
|
|
<Card title="Functions" icon="code" href="/management/function">
|
|
Fill a column with Python.
|
|
</Card>
|
|
*/}
|
|
</CardGroup>
|