mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-30 15:26:52 +00:00
## Summary - Complete the SQL reference: `ATTACH DATABASE`, the `CHANGES` clause, and `dedupe()`/`duplicate_pairs()` (verified against current sophon source; these were recently renamed from `vector_dedup`/`vector_duplicate_pairs`). Fix the blob-column default-projection claim and add the previously undocumented `blob_descriptor()`. - Enrich the gated Functions draft (still `.mintignore`'d, pre-release): chained Functions over blob outputs, GPU-produced blobs, Secrets, multi-input/output binding, `code=[module]` packaging. - Upgrade the materialized-view and computed-column auto-refresh drafts with precise, verified catch-up semantics, and add the missing `ALTER MATERIALIZED VIEW` / `CREATE ... WITH(...)` sections (also gated — the automatic-refresh scheduler is still dev/staging-only). - Thread one running example (video frames → GPU extract/caption → dedup → auto-refreshing materialized view) through Functions, materialized views, and the LeRobotDataset integration guide instead of a separate tutorial. - Rename "LanceDB OSS"/"OSS" to "LanceDB Embedded"/"Embedded" throughout (52 files); activate the existing availability-comparison mechanism on `basics/offerings.mdx` for a full feature-by-feature comparison table.
246 lines
8.5 KiB
Plaintext
246 lines
8.5 KiB
Plaintext
---
|
|
title: "Table"
|
|
sidebarTitle: "Table"
|
|
description: "CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE."
|
|
icon: table
|
|
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
## CREATE TABLE {#create}
|
|
|
|
```sql
|
|
CREATE TABLE documents (id BIGINT, title VARCHAR, created_at TIMESTAMP)
|
|
CREATE TABLE recent AS SELECT * FROM documents WHERE created_at > now() - INTERVAL '7 days'
|
|
```
|
|
|
|
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
|
|
|
A table inside a [namespace](/sql/ddl/namespace) is written as a dotted path:
|
|
`prod.events.clicks`.
|
|
|
|
## DESCRIBE {#describe}
|
|
|
|
```sql
|
|
DESCRIBE documents
|
|
```
|
|
|
|
The table's columns and types, without a plan. The SQL counterpart of
|
|
`table.schema`.
|
|
|
|
## ALTER TABLE {#alter}
|
|
|
|
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
|
name.
|
|
|
|
### A plain column {#plain-column}
|
|
|
|
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
|
|
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
|
|
with the SDK instead. See
|
|
[Add nullable columns](/management/table#schema-add-nullable-columns).
|
|
|
|
### A computed column {#computed-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
```
|
|
|
|
`AS <expression>` declares a [computed column](/feature-engineering/add-computed-column).
|
|
The column is committed empty and the expression is stored, not evaluated:
|
|
declaring one on a billion-row table costs what declaring one on an empty table
|
|
costs. [`REFRESH COLUMN`](/sql/dml/refresh-column) fills it.
|
|
|
|
{/*
|
|
Draft for automatic computed-column refresh. Enable this section when the
|
|
server feature is released.
|
|
|
|
### Automatic refresh options {#automatic-refresh-options}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
|
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
|
```
|
|
|
|
Automatic refresh is enabled by default. An explicitly configured interval is
|
|
stored in each computed output's field metadata; when omitted, the server
|
|
determines when to refresh. The interval must be a positive integer
|
|
and is minimum pacing after a successful refresh of that output; unchanged
|
|
inputs do not force work. A new column is immediately eligible for its first
|
|
refresh. The server's separate automatic-refresh switch must also be enabled.
|
|
|
|
These options also work beside a Function binding's `version` and `branch`,
|
|
continuing the [`caption` Function](/management/function#chain) from Functions:
|
|
|
|
```sql
|
|
ALTER TABLE frames ADD COLUMN caption AS caption(extract_frame(video, ts))
|
|
WITH (version = 'fv_1', auto_refresh_enabled = true,
|
|
auto_refresh_interval_seconds = 300);
|
|
ALTER TABLE frames ALTER COLUMN caption SET auto_refresh_enabled = false;
|
|
```
|
|
|
|
Declaration settings initialize every output in the group; later SET commands
|
|
change one output. SET takes effect immediately and does not itself read or
|
|
write any row:
|
|
|
|
- **Disabling** prevents new automatic submissions. Inflight jobs may still
|
|
finish, explicit `REFRESH COLUMN` keeps working, and rows written while
|
|
disabled commit with no computed value -- they do not queue anywhere
|
|
separate.
|
|
- **Re-enabling catches up the entire backlog in one pass.** The next
|
|
scheduler tick processes every row left unfilled since the column was
|
|
disabled, immediately, with no further write required to trigger it: a
|
|
column disabled for an hour and re-enabled fills everything that
|
|
accumulated during that hour, not just rows written after the moment it
|
|
is re-enabled.
|
|
|
|
Column SUSPEND/RESUME syntax is not supported; `SET auto_refresh_enabled` is
|
|
the only toggle. The current setting is in the column's own field metadata,
|
|
`computed_column.auto_refresh_enabled`, readable from `DESCRIBE` or the
|
|
SDK's `table.schema` without submitting a job to ask.
|
|
*/}
|
|
|
|
{/*
|
|
Draft for the Functions release.
|
|
|
|
### A function column {#function-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN embedded AS embed(name) WITH (version = 'fv_1')
|
|
```
|
|
|
|
When the expression calls a [registered function](/sql/ddl/function), the
|
|
column is filled by running that function on workers. `version` pins which
|
|
immutable function version fills it, so a later registration does not silently
|
|
change what the column means.
|
|
|
|
Several modifiers narrow what gets computed:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `FILTER (WHERE pred)` | only rows matching `pred` are computed |
|
|
| `IGNORE NULLS` | rows with null inputs are skipped |
|
|
| `OVER (PARTITION BY col)` | the function sees rows grouped by `col` |
|
|
|
|
A function returning several values fills several columns at once:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1')
|
|
```
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
|
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
|
```
|
|
|
|
<Note>
|
|
Declaration `WITH` accepts `auto_refresh_enabled` and
|
|
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
|
`branch`. Worker resource options such as `num_workers` belong on
|
|
`REFRESH COLUMN ... WITH (...)`.
|
|
</Note>
|
|
*/}
|
|
|
|
### Table properties {#properties}
|
|
|
|
```sql
|
|
ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
|
|
```
|
|
|
|
Records which column identifies a row. **Unenforced** is not a hedge in the
|
|
naming: uniqueness is not checked on write, and declaring it does not make
|
|
duplicates impossible. It tells the planner what to assume. The column must be
|
|
declared `NOT NULL`.
|
|
|
|
```sql
|
|
ALTER TABLE t SET RTE
|
|
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
|
|
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
|
|
ALTER TABLE t UNSET RTE
|
|
```
|
|
|
|
The [real time engine](/tables/realtime-engine) puts a write-ahead log in front
|
|
of the table, making small frequent writes cheap by batching them out of the
|
|
commit path. `sharding` and `buckets` spread the log so concurrent writers do
|
|
not contend on one place; `bucket` sharding names its `column`, which must be
|
|
the table's unenforced primary key, and inserts into the table then need a
|
|
scalar index on that key. `maintained_indexes` chooses which indexes are kept
|
|
current against WAL rows; omitted, every supported index is kept current,
|
|
including ones created later. A table with a computed column cannot take a
|
|
write-ahead log. See
|
|
[Background jobs](/maintenance/background-jobs#wal) for how the log gets
|
|
folded back into the table.
|
|
|
|
<Warning>
|
|
[`TABLESAMPLE`](/sql/dql/sampling) is refused on a WAL-backed table, because
|
|
rows a sample would draw from are not all in the fragments yet.
|
|
</Warning>
|
|
|
|
**Errors and limits**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `ALTER TABLE ... DROP COLUMN` | not supported |
|
|
|
|
Dropping, renaming, or retyping a column is an SDK operation
|
|
([`drop_columns`](/management/table#schema) and
|
|
[`alter_columns`](/management/table#schema)), not a SQL statement.
|
|
|
|
## CLONE {#clone}
|
|
|
|
```sql
|
|
CREATE TABLE docs_snapshot CLONE documents
|
|
CREATE TABLE docs_at_v42 CLONE documents AT 42
|
|
CREATE TABLE docs_at_release CLONE documents AT TAG 'release-1'
|
|
CREATE TABLE docs_yesterday CLONE documents AT TIMESTAMP '2026-09-01T00:00:00Z'
|
|
```
|
|
|
|
A **shallow clone**: the new table shares the source's data files rather than
|
|
copying them, so it costs one metadata commit regardless of the source's size.
|
|
|
|
`AT` pins the point in the source's history to clone. A bare integer is a
|
|
version and a bare quoted string is a timestamp; `VERSION`, `TAG` and
|
|
`TIMESTAMP` say so explicitly, and a tag always needs its keyword: a bare
|
|
quoted string cannot be both a tag name and a timestamp.
|
|
|
|
<Warning>
|
|
The clone depends on the source's files continuing to exist. If the source is
|
|
[compacted](/maintenance/cleanup) past the retention window, the version the
|
|
clone points at can be pruned. The SDK's
|
|
[`clone_table`](/management/table#rename-clone) also offers a deep clone for
|
|
when the copy has to outlive the source's history.
|
|
</Warning>
|
|
|
|
## DROP TABLE {#drop}
|
|
|
|
```sql
|
|
DROP TABLE documents
|
|
```
|
|
|
|
<Warning>
|
|
Removes the table and every version of it. There is no undrop.
|
|
</Warning>
|
|
|
|
<Note>
|
|
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
|
|
deployment stores its catalog. See
|
|
[Background jobs](/maintenance/background-jobs#teardown).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Table" icon="table" href="/management/table">
|
|
The SDK equivalent, and what a table carries.
|
|
</Card>
|
|
<Card title="Schema evolution" icon="boxes-stacked" href="/management/table#schema">
|
|
Renaming, retyping and dropping columns from the SDK.
|
|
</Card>
|
|
</CardGroup>
|