mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-16 07:36:50 +00:00
The Enterprise feature name was inconsistently written as "Real Time Engine" and "real time engine". Standardize it to **Realtime Engine** in eight website guides, including page and sidebar titles, descriptions, prose, link labels, cards, and search keywords. The remaining MemWAL references describe Lance's open source format or literal API/job identifiers, including `memwal_compaction` and `memwal_catchup_<type>`, and are preserved. Only `docs/web/*.mdx` pages change; SDK source, generated API reference, code examples, URL destinations, and section anchors are unchanged. Validation: - `git diff --check` passes. - Reviewed all MemWAL and Realtime Engine references under `docs/`. - Verified each changed file differs only by the intended naming substitutions and preserves code blocks, inline code, links, anchors, and MemWAL identifiers. - Confirmed no split or hyphenated "real time engine" names remain in website guides. - No snippets needed regeneration because code examples are unchanged. Vale and Mintlify are not installed locally, so their checks were not run locally.
261 lines
8.8 KiB
Plaintext
261 lines
8.8 KiB
Plaintext
---
|
|
title: "Table"
|
|
sidebarTitle: "Table"
|
|
description: "CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE."
|
|
icon: table
|
|
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
## CREATE TABLE {#create}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE TABLE documents (id BIGINT, title VARCHAR, created_at TIMESTAMP)
|
|
CREATE TABLE recent AS
|
|
SELECT * FROM documents
|
|
WHERE created_at > now() - INTERVAL '7 days'
|
|
```
|
|
</CodeGroup>
|
|
|
|
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
|
|
|
A table inside a [namespace](/sql/ddl/namespace) is written with its
|
|
[schema](/basics/catalog#schema) before the name: `` `prod$events`.clicks ``.
|
|
|
|
## DESCRIBE {#describe}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DESCRIBE documents
|
|
```
|
|
</CodeGroup>
|
|
|
|
The table's columns and types, without a plan. The SQL counterpart of
|
|
`table.schema`.
|
|
|
|
## ALTER TABLE {#alter}
|
|
|
|
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
|
name.
|
|
|
|
### A plain column {#plain-column}
|
|
|
|
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
|
|
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
|
|
with the SDK instead. See
|
|
[Add nullable columns](/management/table#schema-add-nullable-columns).
|
|
|
|
### A computed column {#computed-column}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
```
|
|
</CodeGroup>
|
|
|
|
`AS <expression>` declares a [computed column](/feature-engineering/add-computed-column).
|
|
The column is committed empty and the expression is stored, not evaluated:
|
|
declaring one on a billion-row table costs what declaring one on an empty table
|
|
costs. [`REFRESH COLUMN`](/sql/dml/refresh-column) fills it.
|
|
|
|
{/*
|
|
Draft for automatic computed-column refresh. Enable this section when the
|
|
server feature is released.
|
|
|
|
### Automatic refresh options {#automatic-refresh-options}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
|
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
|
```
|
|
|
|
Automatic refresh is enabled by default. An explicitly configured interval is
|
|
stored in each computed output's field metadata; when omitted, the server
|
|
determines when to refresh. The interval must be a positive integer
|
|
and is minimum pacing after a successful refresh of that output; unchanged
|
|
inputs do not force work. A new column is immediately eligible for its first
|
|
refresh. The server's separate automatic-refresh switch must also be enabled.
|
|
|
|
These options also work beside a Function binding's `version` and `branch`,
|
|
continuing the [`caption` Function](/curation/create-function#chain) from Create a Function:
|
|
|
|
```sql
|
|
ALTER TABLE frames ADD COLUMN caption AS caption(extract_frame(video, ts))
|
|
WITH (
|
|
version = 'fv_1',
|
|
auto_refresh_enabled = true,
|
|
auto_refresh_interval_seconds = 300
|
|
);
|
|
ALTER TABLE frames ALTER COLUMN caption SET auto_refresh_enabled = false;
|
|
```
|
|
|
|
Declaration settings initialize every output in the group; later SET commands
|
|
change one output. SET takes effect immediately and does not itself read or
|
|
write any row:
|
|
|
|
- **Disabling** prevents new automatic submissions. Inflight jobs may still
|
|
finish, explicit `REFRESH COLUMN` keeps working, and rows written while
|
|
disabled commit with no computed value -- they do not queue anywhere
|
|
separate.
|
|
- **Re-enabling catches up the entire backlog in one pass.** The next
|
|
scheduler tick processes every row left unfilled since the column was
|
|
disabled, immediately, with no further write required to trigger it: a
|
|
column disabled for an hour and re-enabled fills everything that
|
|
accumulated during that hour, not just rows written after the moment it
|
|
is re-enabled.
|
|
|
|
Column SUSPEND/RESUME syntax is not supported; `SET auto_refresh_enabled` is
|
|
the only toggle. The current setting is in the column's own field metadata,
|
|
`computed_column.auto_refresh_enabled`, readable from `DESCRIBE` or the
|
|
SDK's `table.schema` without submitting a job to ask.
|
|
*/}
|
|
|
|
### A function column {#function-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN embedded AS embed(name) WITH (version = 'fv_1')
|
|
```
|
|
|
|
When the expression calls a [registered function](/sql/ddl/function), the
|
|
column is filled by running that function on workers. `version` pins which
|
|
immutable function version fills it, so a later registration does not silently
|
|
change what the column means.
|
|
|
|
Several modifiers narrow what gets computed:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `FILTER (WHERE pred)` | only rows matching `pred` are computed |
|
|
| `IGNORE NULLS` | rows with null inputs are skipped |
|
|
| `OVER (PARTITION BY col)` | the function sees rows grouped by `col` |
|
|
|
|
A function returning several values fills several columns at once:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1')
|
|
```
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
|
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
|
```
|
|
|
|
<Note>
|
|
Declaration `WITH` accepts `auto_refresh_enabled` and
|
|
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
|
`branch`. Worker resource options such as `num_workers` belong on
|
|
`REFRESH COLUMN ... WITH (...)`.
|
|
</Note>
|
|
|
|
### Table properties {#properties}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
|
|
```
|
|
</CodeGroup>
|
|
|
|
Records which column identifies a row. **Unenforced** is not a hedge in the
|
|
naming: uniqueness is not checked on write, and declaring it does not make
|
|
duplicates impossible. It tells the planner what to assume. The column must be
|
|
declared `NOT NULL`.
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t SET RTE
|
|
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
|
|
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
|
|
ALTER TABLE t UNSET RTE
|
|
```
|
|
</CodeGroup>
|
|
|
|
The [Realtime Engine](/tables/realtime-engine) puts a write-ahead log in front
|
|
of the table, making small frequent writes cheap by batching them out of the
|
|
commit path. `sharding` and `buckets` spread the log so concurrent writers do
|
|
not contend on one place; `bucket` sharding names its `column`, which must be
|
|
the table's unenforced primary key, and inserts into the table then need a
|
|
scalar index on that key. `maintained_indexes` chooses which indexes are kept
|
|
current against WAL rows; omitted, every supported index is kept current,
|
|
including ones created later. A table with a computed column cannot take a
|
|
write-ahead log. See
|
|
[RTE Compaction](/enterprise/realtime-engine-compaction) for how
|
|
the log gets folded back into the table.
|
|
|
|
<Warning>
|
|
[`TABLESAMPLE`](/sql/dql/sampling) is refused on a WAL-backed table, because
|
|
rows a sample would draw from are not all in the fragments yet.
|
|
</Warning>
|
|
|
|
### Errors and limits {#errors-and-limits}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `ALTER TABLE ... DROP COLUMN` | not supported |
|
|
|
|
Dropping, renaming, or retyping a column is an SDK operation
|
|
([`drop_columns`](/management/table#schema) and
|
|
[`alter_columns`](/management/table#schema)), not a SQL statement.
|
|
|
|
## CLONE {#clone}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE TABLE docs_snapshot CLONE documents
|
|
CREATE TABLE docs_at_v42 CLONE documents AT 42
|
|
CREATE TABLE docs_at_release CLONE documents AT TAG 'release-1'
|
|
CREATE TABLE docs_yesterday CLONE documents AT TIMESTAMP '2026-09-01T00:00:00Z'
|
|
```
|
|
</CodeGroup>
|
|
|
|
A **shallow clone**: the new table shares the source's data files rather than
|
|
copying them, so it costs one metadata commit regardless of the source's size.
|
|
|
|
`AT` pins the point in the source's history to clone. A bare integer is a
|
|
version and a bare quoted string is a timestamp; `VERSION`, `TAG` and
|
|
`TIMESTAMP` say so explicitly, and a tag always needs its keyword: a bare
|
|
quoted string cannot be both a tag name and a timestamp.
|
|
|
|
<Warning>
|
|
The clone depends on the source's files continuing to exist. If the source is
|
|
[compacted](/maintenance/cleanup) past the retention window, the version the
|
|
clone points at can be pruned. The SDK's
|
|
[`clone_table`](/management/table#rename-clone) also offers a deep clone for
|
|
when the copy has to outlive the source's history.
|
|
</Warning>
|
|
|
|
## DROP TABLE {#drop}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DROP TABLE documents
|
|
```
|
|
</CodeGroup>
|
|
|
|
<Warning>
|
|
Removes the table and every version of it. There is no undrop.
|
|
</Warning>
|
|
|
|
<Note>
|
|
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
|
|
deployment stores its catalog. See
|
|
[Object teardown](/basics/jobs#teardown).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Table" icon="table" href="/management/table">
|
|
The SDK equivalent, and what a table carries.
|
|
</Card>
|
|
<Card title="Schema evolution" icon="boxes-stacked" href="/management/table#schema">
|
|
Renaming, retyping and dropping columns from the SDK.
|
|
</Card>
|
|
</CardGroup>
|