mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-24 20:52:29 +00:00
Add `auto_refresh_enabled` and `auto_refresh_interval_seconds` to the
Rust builder and Python sync/async add-column APIs for computed columns
and registered Function outputs.
Python users can configure refresh when declaring a column:
```python
table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
Omit the interval to let the server determine when to refresh:
```python
table.add_columns(computed={"discounted_total": "total * 0.9"})
```
The async API accepts the same options:
```python
await async_table.add_columns(
computed={"total": "price * quantity"},
auto_refresh_enabled=True,
auto_refresh_interval_seconds=300,
)
```
For an existing registered `FunctionApplication`, the options also apply
to Function columns. Disabling automatic refresh still permits explicit
refresh:
```python
table.add_columns(
{"features": function_application},
auto_refresh_enabled=False,
auto_refresh_interval_seconds=300,
)
table.refresh_column("features")
```
Options apply to every output declared by a call. Automatic refresh is
enabled by default; when the interval is omitted, the server determines
when to refresh. An explicit interval sets a minimum delay after a
successful refresh. The interval does not force recomputation of
unchanged inputs. Invalid intervals and options on one-time transforms
are rejected before declaration.
Native declarations persist `computed_column.auto_refresh_enabled` and
an explicitly supplied `computed_column.auto_refresh_interval_seconds`
in each output field’s metadata in the same commit as the declaration.
An omitted interval is neither stored nor sent in remote requests.
Remote declarations forward the settings to Cloud/Enterprise servers.
Local tables store these settings; automatic execution requires a
supporting server with scheduling enabled.
228 lines
7.6 KiB
Plaintext
228 lines
7.6 KiB
Plaintext
---
|
|
title: "Table"
|
|
sidebarTitle: "Table"
|
|
description: "CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE."
|
|
icon: table
|
|
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
|
|
|
|
## CREATE TABLE {#create}
|
|
|
|
```sql
|
|
CREATE TABLE documents (id BIGINT, title VARCHAR, created_at TIMESTAMP)
|
|
CREATE TABLE recent AS SELECT * FROM documents WHERE created_at > now() - INTERVAL '7 days'
|
|
```
|
|
|
|
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
|
A table inside a [namespace](/sql/ddl/namespace) is written as a dotted path:
|
|
`prod.events.clicks`.
|
|
|
|
## DESCRIBE {#describe}
|
|
|
|
```sql
|
|
DESCRIBE documents
|
|
```
|
|
|
|
The table's columns and types, without a plan. The SQL counterpart of
|
|
`table.schema`.
|
|
|
|
## ALTER TABLE {#alter}
|
|
|
|
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
|
name.
|
|
|
|
### A plain column {#plain-column}
|
|
|
|
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
|
|
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
|
|
with the SDK instead. See
|
|
[Add nullable columns](/management/table#schema-add-nullable-columns).
|
|
|
|
### A computed column {#computed-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
```
|
|
|
|
`AS <expression>` declares a [computed column](/feature-engineering/add-computed-column).
|
|
The column is committed empty and the expression is stored, not evaluated:
|
|
declaring one on a billion-row table costs what declaring one on an empty table
|
|
costs. [`REFRESH COLUMN`](/sql/dml/refresh) fills it.
|
|
|
|
{/*
|
|
Draft for automatic computed-column refresh. Enable this section when the
|
|
server feature is released.
|
|
|
|
### Automatic refresh options {#automatic-refresh-options}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
|
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
|
```
|
|
|
|
Automatic refresh is enabled by default. An explicitly configured interval is
|
|
stored in each computed output's field metadata; when omitted, the server
|
|
determines when to refresh. The interval must be a positive integer
|
|
and is minimum pacing after a successful refresh of that output; unchanged
|
|
inputs do not force work. A new column is immediately eligible for its first
|
|
refresh. The server's separate automatic-refresh switch must also be enabled.
|
|
|
|
These options also work beside a Function binding's `version` and `branch`:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text)
|
|
WITH (version = 'fv_1', auto_refresh_enabled = true,
|
|
auto_refresh_interval_seconds = 300);
|
|
ALTER TABLE t ALTER COLUMN score SET auto_refresh_enabled = false;
|
|
```
|
|
|
|
Declaration settings initialize every output in the group; later SET commands
|
|
change one output. Disabling automatic refresh prevents new submissions while
|
|
allowing inflight jobs to finish and explicit REFRESH COLUMN to run. Column
|
|
SUSPEND/RESUME syntax is not supported.
|
|
*/}
|
|
|
|
{/*
|
|
Draft for the Functions release.
|
|
|
|
### A function column {#function-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN embedded AS embed(name) WITH (version = 'fv_1')
|
|
```
|
|
|
|
When the expression calls a [registered function](/sql/ddl/function), the
|
|
column is filled by running that function on workers. `version` pins which
|
|
immutable function version fills it, so a later registration does not silently
|
|
change what the column means.
|
|
|
|
Several modifiers narrow what gets computed:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `FILTER (WHERE pred)` | only rows matching `pred` are computed |
|
|
| `IGNORE NULLS` | rows with null inputs are skipped |
|
|
| `OVER (PARTITION BY col)` | the function sees rows grouped by `col` |
|
|
|
|
A function returning several values fills several columns at once:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1')
|
|
```
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
|
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
|
```
|
|
|
|
<Note>
|
|
Declaration `WITH` accepts `auto_refresh_enabled` and
|
|
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
|
`branch`. Worker resource options such as `num_workers` belong on
|
|
`REFRESH COLUMN ... WITH (...)`.
|
|
</Note>
|
|
*/}
|
|
|
|
### Table properties {#properties}
|
|
|
|
```sql
|
|
ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
|
|
```
|
|
|
|
Records which column identifies a row. **Unenforced** is not a hedge in the
|
|
naming: uniqueness is not checked on write, and declaring it does not make
|
|
duplicates impossible. It tells the planner what to assume. The column must be
|
|
declared `NOT NULL`.
|
|
|
|
```sql
|
|
ALTER TABLE t SET RTE
|
|
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
|
|
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
|
|
ALTER TABLE t UNSET RTE
|
|
```
|
|
|
|
The [realtime engine](/tables/realtime-engine) puts a write-ahead log in front
|
|
of the table, making small frequent writes cheap by batching them out of the
|
|
commit path. `sharding` and `buckets` spread the log so concurrent writers do
|
|
not contend on one place; `bucket` sharding names its `column`, which must be
|
|
the table's unenforced primary key, and inserts into the table then need a
|
|
scalar index on that key. `maintained_indexes` chooses which indexes are kept
|
|
current against WAL rows; omitted, every supported index is kept current,
|
|
including ones created later. A table with a computed column cannot take a
|
|
write-ahead log. See
|
|
[Background jobs](/maintenance/background-jobs#wal) for how the log gets
|
|
folded back into the table.
|
|
|
|
<Warning>
|
|
[`TABLESAMPLE`](/sql/dql/sampling) is refused on a WAL-backed table, because
|
|
rows a sample would draw from are not all in the fragments yet.
|
|
</Warning>
|
|
|
|
**Errors and limits**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `ALTER TABLE ... DROP COLUMN` | not supported |
|
|
|
|
Dropping, renaming, or retyping a column is an SDK operation
|
|
([`drop_columns`](/management/table#schema) and
|
|
[`alter_columns`](/management/table#schema)), not a SQL statement.
|
|
|
|
## CLONE {#clone}
|
|
|
|
```sql
|
|
CREATE TABLE docs_snapshot CLONE documents
|
|
CREATE TABLE docs_at_v42 CLONE documents AT 42
|
|
CREATE TABLE docs_at_release CLONE documents AT TAG 'release-1'
|
|
CREATE TABLE docs_yesterday CLONE documents AT TIMESTAMP '2026-09-01T00:00:00Z'
|
|
```
|
|
|
|
A **shallow clone**: the new table shares the source's data files rather than
|
|
copying them, so it costs one metadata commit regardless of the source's size.
|
|
|
|
`AT` pins the point in the source's history to clone. A bare integer is a
|
|
version and a bare quoted string is a timestamp; `VERSION`, `TAG` and
|
|
`TIMESTAMP` say so explicitly, and a tag always needs its keyword: a bare
|
|
quoted string cannot be both a tag name and a timestamp.
|
|
|
|
<Warning>
|
|
The clone depends on the source's files continuing to exist. If the source is
|
|
[compacted](/maintenance/cleanup) past the retention window, the version the
|
|
clone points at can be pruned. The SDK's
|
|
[`clone_table`](/management/table#rename-clone) also offers a deep clone for
|
|
when the copy has to outlive the source's history.
|
|
</Warning>
|
|
|
|
## DROP TABLE {#drop}
|
|
|
|
```sql
|
|
DROP TABLE documents
|
|
```
|
|
|
|
<Warning>
|
|
Removes the table and every version of it. There is no undrop.
|
|
</Warning>
|
|
|
|
<Note>
|
|
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
|
|
deployment stores its catalog. See
|
|
[Background jobs](/maintenance/background-jobs#teardown).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Table" icon="table" href="/management/table">
|
|
The SDK equivalent, and what a table carries.
|
|
</Card>
|
|
<Card title="Schema evolution" icon="boxes-stacked" href="/management/table#schema">
|
|
Renaming, retyping and dropping columns from the SDK.
|
|
</Card>
|
|
</CardGroup>
|