mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 15:26:52 +00:00
## Summary Adopts the product terminology guide across the docs site pages, SDK docstrings and error messages. "LanceDB OSS" and local mode become **LanceDB Embedded**. "Remote" as the execution model becomes **LanceDB Distributed**, which LanceDB Enterprise provides. Where free vs paid is the point, pages contrast "LanceDB Embedded (open source)" with "LanceDB Enterprise". This is a first trial of the Embedded name, so reviewers can judge it in context. - **Embedded vs Distributed** for where execution runs (`optimize()`, consistency, branching, maintenance). Main pages define it once: "LanceDB Enterprise runs LanceDB Distributed: …" - **LanceDB Cloud** is kept only for legacy-Cloud specifics (`db://` region, the brute-force KNN limit) and marked legacy. "Cloud and Enterprise" becomes Enterprise or Distributed. - **Plan executor** becomes "executor" or the fleet name. - **Error messages** change, e.g. `optimize is not supported for remote tables.` → `... in LanceDB Distributed.` Tests that match on them are updated. - **Vale rules** (`EmbeddedName`, `OSSProduct`, `RemoteExecution`, `DistributedEnterprise`, `LegacyCloud`, `InternalNames`) enforce the terms in prose. Unchanged: identifiers (`availability.oss`, the `remote` module and feature, `RemoteTable`, `db://`), heading anchors, internal code comments, and Geneva. ## Evidence - Rust lib tests for the changed messages and the Python tests in `test_remote_db.py`, `test_materialized_views.py` and `test_table.py` pass locally. - Not run locally: - The Node tests: `remote.test.ts` and `table.test.ts` have updated assertions. - Vale. - The `experimental_functions`-marked materialized-view tests. - `docs/src` API reference was hand-edited to match the docstrings rather than regenerated. ## Merge Risk **Door:** two-way; prose and message text only. **Blast Radius:** `docs`, `error-messages`: callers matching on old error strings would break. Companion PRs: lancedb/docs (assembler labels, STYLE.md) and sophon (Enterprise pages). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
132 lines
4.8 KiB
Plaintext
132 lines
4.8 KiB
Plaintext
---
|
|
title: "Add a computed column"
|
|
sidebarTitle: "Computed Columns"
|
|
description: "Declare a column from a SQL expression. Nothing is evaluated until you refresh it."
|
|
icon: table-columns
|
|
keywords: ["computed columns", "add_columns", "transforms", "schema"]
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Local computed columns require a source build with the experimental-functions Cargo feature. Enterprise requires the deployment to enable them.
|
|
---
|
|
|
|
A computed column is declared with an expression and starts with no values.
|
|
Declaring it is cheap on any table size, because nothing is evaluated until
|
|
[a refresh](/feature-engineering/refresh-column) asks for it.
|
|
|
|
<Note>
|
|
In the 0.41 SDK release, computed-column declaration and refresh in LanceDB Embedded require a
|
|
source build with the `experimental-functions` Cargo feature. Published Python
|
|
and TypeScript packages leave it disabled. LanceDB Enterprise operations
|
|
do not require this feature. Ordinary eager `add_columns` transforms remain available.
|
|
</Note>
|
|
|
|
## Add a computed column {#declare}
|
|
|
|
```python
|
|
import lancedb
|
|
|
|
db = lancedb.connect("./.lancedb")
|
|
table = db.create_table("computed_demo", [{"x": 1}, {"x": 2}])
|
|
|
|
table.add_columns(computed={"doubled": "x * 2"})
|
|
# AddColumnsResult(version=2)
|
|
```
|
|
|
|
That commits a new table version with the column present and empty. It is an
|
|
ordinary table version, so the usual [versioning](/management/table#versioning) applies,
|
|
including that you can travel back to before the column existed.
|
|
|
|
## Compute now or on refresh {#computed-vs-transforms}
|
|
|
|
`add_columns` takes either, and the difference matters:
|
|
|
|
- **SQL expressions in `transforms`** are evaluated *now*, for every row, as
|
|
part of the call. Good for a small table, or a column you want populated
|
|
immediately.
|
|
{/* Draft for the Functions release:
|
|
A `FunctionApplication` passed through `transforms` instead
|
|
declares empty Function-bound columns, which a refresh fills; that needs a
|
|
deployment with Functions enabled, and the embedded library refuses it. */}
|
|
- **`computed`** stores the expression and evaluates nothing. The column is
|
|
committed empty and filled later by
|
|
[`refresh_column`](/feature-engineering/refresh-column).
|
|
|
|
`computed` and `transforms` cannot be combined in one call. Reach for `computed` when the table is
|
|
large enough that you would rather control when the work happens.
|
|
|
|
## The expression {#expressions}
|
|
|
|
The expression is SQL, evaluated over the row. It can reference any column in
|
|
the table, including another computed column:
|
|
|
|
```python
|
|
table.add_columns(computed={
|
|
"power_score": "cast(x as double) / 2.0",
|
|
"is_strong": "power_score > 0.5",
|
|
})
|
|
```
|
|
|
|
Declaration order is dependency order: a computed column may reference one
|
|
declared earlier in the same mapping, but not one declared after it.
|
|
|
|
The functions available are the ones the
|
|
[SQL dialect](/sql/functions) documents.
|
|
|
|
A literal in the expression has to fit the column it meets: `x / 2.0` is
|
|
refused when `x` is an integer column, so write `cast(x as double) / 2.0`.
|
|
|
|
<Warning>
|
|
In LanceDB Enterprise, keep the expression to top-level columns. A column
|
|
refresh can fail on an expression that reads a field inside a struct, such as
|
|
`stats.magic + stats.strength`, which the embedded library computes. Try a new
|
|
expression on a small table first.
|
|
</Warning>
|
|
|
|
## Change columns used by an expression {#input-constraints}
|
|
|
|
<Warning>
|
|
While a computed column's declaration references another column, that column
|
|
cannot be renamed, retyped, or dropped. Drop the computed column first
|
|
if you need to change one of its inputs.
|
|
</Warning>
|
|
|
|
This is worth knowing before deriving a column from something you expect to
|
|
reshape later. Dropping the computed column releases the constraint.
|
|
|
|
## Filling with Python instead of SQL {#functions}
|
|
|
|
<Badge color="red">Enterprise</Badge>
|
|
|
|
When the value cannot be expressed in SQL (an embedding, a model call, an
|
|
image decode), [Geneva](/geneva), the feature engineering package for LanceDB
|
|
Enterprise, fills the column with a Python function instead.
|
|
|
|
{/*
|
|
Draft for the Functions release. Replaces the paragraph above:
|
|
|
|
When the value cannot be expressed in SQL -- an embedding, a model call, an
|
|
image decode -- the column is filled by a registered
|
|
[function](/management/function) instead of an expression. Declaring it looks
|
|
the same; what differs is what fills it. It needs a deployment with Functions
|
|
enabled.
|
|
*/}
|
|
|
|
## Related guides {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Refresh a column" icon="arrows-rotate" href="/feature-engineering/refresh-column">
|
|
Fill the rows that lack values.
|
|
</Card>
|
|
<Card title="Geneva" icon="layer-group" href="/geneva">
|
|
Fill a column with Python on LanceDB Enterprise.
|
|
</Card>
|
|
{/*
|
|
Draft for the Functions release. Replaces the Geneva card above:
|
|
<Card title="Functions" icon="code" href="/management/function">
|
|
Fill a column with Python.
|
|
</Card>
|
|
*/}
|
|
</CardGroup>
|