mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-16 07:36:50 +00:00
Migrates SQL-backed catalog listing APIs to `LIMIT ... FROM '<page-token>'` and server-issued Arrow continuation metadata. Namespace-specific APIs send an explicit `IN` schema clause, including `IN "public"` for root, so raw SQL can default to all namespaces. ```sql SHOW VIEWS LIMIT 100; SHOW VIEWS IN team$ml LIMIT 100; SHOW VIEWS IN team$ml LIMIT 100 FROM '<page-token>'; SHOW VIEWS IN other_db.team$ml LIMIT 100; ``` SDK clients forward opaque tokens unchanged, preserve empty continuations, and decode identifier columns by name. Removes legacy cursor reconstruction and updates the SQL reference and Python Flight fixture. Public docs cover optional direct schema paths, all-namespace defaults, and the shared pagination syntax across every catalog listing.
291 lines
10 KiB
Plaintext
291 lines
10 KiB
Plaintext
---
|
|
title: "Table"
|
|
sidebarTitle: "Table"
|
|
description: "CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE."
|
|
icon: table
|
|
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
## CREATE TABLE {#create}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE TABLE documents (id BIGINT, title VARCHAR, created_at TIMESTAMP)
|
|
CREATE TABLE recent AS
|
|
SELECT * FROM documents
|
|
WHERE created_at > now() - INTERVAL '7 days'
|
|
```
|
|
</CodeGroup>
|
|
|
|
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
|
|
|
A table inside a [namespace](/sql/ddl/namespace) is written with its
|
|
[schema](/basics/catalog#schema) before the name: `` `prod$events`.clicks ``.
|
|
|
|
## DESCRIBE {#describe}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DESCRIBE documents
|
|
```
|
|
</CodeGroup>
|
|
|
|
The table's columns and types, without a plan. The SQL counterpart of
|
|
`table.schema`.
|
|
|
|
## ALTER TABLE {#alter}
|
|
|
|
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
|
name.
|
|
|
|
### A plain column {#plain-column}
|
|
|
|
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
|
|
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
|
|
with the SDK instead. See
|
|
[Add nullable columns](/management/table#schema-add-nullable-columns).
|
|
|
|
### A computed column {#computed-column}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
```
|
|
</CodeGroup>
|
|
|
|
`AS <expression>` declares a [computed column](/feature-engineering/add-computed-column).
|
|
The column is committed empty and the expression is stored, not evaluated:
|
|
declaring one on a billion-row table costs what declaring one on an empty table
|
|
costs. [`REFRESH COLUMN`](/sql/dml/refresh-column) fills it.
|
|
|
|
{/*
|
|
Draft for automatic computed-column refresh. Enable this section when the
|
|
server feature is released.
|
|
|
|
### Automatic refresh options {#automatic-refresh-options}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
|
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
|
```
|
|
|
|
Automatic refresh is enabled by default. An explicitly configured interval is
|
|
stored in each computed output's field metadata; when omitted, the server
|
|
determines when to refresh. The interval must be a positive integer
|
|
and is minimum pacing after a successful refresh of that output; unchanged
|
|
inputs do not force work. A new column is immediately eligible for its first
|
|
refresh. The server's separate automatic-refresh switch must also be enabled.
|
|
|
|
These options also work beside a Function binding's `version` and `branch`,
|
|
continuing the [`caption` Function](/curation/create-function#chain) from Create a Function:
|
|
|
|
```sql
|
|
ALTER TABLE frames ADD COLUMN caption AS caption(extract_frame(video, ts))
|
|
WITH (
|
|
version = 'fv_1',
|
|
auto_refresh_enabled = true,
|
|
auto_refresh_interval_seconds = 300
|
|
);
|
|
ALTER TABLE frames ALTER COLUMN caption SET auto_refresh_enabled = false;
|
|
```
|
|
|
|
Declaration settings initialize every output in the group; later SET commands
|
|
change one output. SET takes effect immediately and does not itself read or
|
|
write any row:
|
|
|
|
- **Disabling** prevents new automatic submissions. Inflight jobs may still
|
|
finish, explicit `REFRESH COLUMN` keeps working, and rows written while
|
|
disabled commit with no computed value -- they do not queue anywhere
|
|
separate.
|
|
- **Re-enabling catches up the entire backlog in one pass.** The next
|
|
scheduler tick processes every row left unfilled since the column was
|
|
disabled, immediately, with no further write required to trigger it: a
|
|
column disabled for an hour and re-enabled fills everything that
|
|
accumulated during that hour, not just rows written after the moment it
|
|
is re-enabled.
|
|
|
|
Column SUSPEND/RESUME syntax is not supported; `SET auto_refresh_enabled` is
|
|
the only toggle. The current setting is in the column's own field metadata,
|
|
`computed_column.auto_refresh_enabled`, readable from `DESCRIBE` or the
|
|
SDK's `table.schema` without submitting a job to ask.
|
|
*/}
|
|
|
|
### A function column {#function-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN embedded AS embed(name) WITH (version = 'fv_1')
|
|
```
|
|
|
|
When the expression calls a [registered function](/sql/ddl/function), the
|
|
column is filled by running that function on workers. `version` pins which
|
|
immutable function version fills it, so a later registration does not silently
|
|
change what the column means.
|
|
|
|
Several modifiers narrow what gets computed:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `FILTER (WHERE pred)` | only rows matching `pred` are computed |
|
|
| `IGNORE NULLS` | rows with null inputs are skipped |
|
|
| `OVER (PARTITION BY col)` | the function sees rows grouped by `col` |
|
|
|
|
A function returning several values fills several columns at once:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1')
|
|
```
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
|
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
|
```
|
|
|
|
<Note>
|
|
Declaration `WITH` accepts `auto_refresh_enabled` and
|
|
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
|
`branch`. Worker resource options such as `num_workers` belong on
|
|
`REFRESH COLUMN ... WITH (...)`.
|
|
</Note>
|
|
|
|
### Table properties {#properties}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
|
|
```
|
|
</CodeGroup>
|
|
|
|
Records which column identifies a row. **Unenforced** is not a hedge in the
|
|
naming: uniqueness is not checked on write, and declaring it does not make
|
|
duplicates impossible. It tells the planner what to assume. The column must be
|
|
declared `NOT NULL`.
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
ALTER TABLE t SET RTE
|
|
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
|
|
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
|
|
ALTER TABLE t UNSET RTE
|
|
```
|
|
</CodeGroup>
|
|
|
|
The [Realtime Engine](/tables/realtime-engine) puts a write-ahead log in front
|
|
of the table, making small frequent writes cheap by batching them out of the
|
|
commit path. `sharding` and `buckets` spread the log so concurrent writers do
|
|
not contend on one place; `bucket` sharding names its `column`, which must be
|
|
the table's unenforced primary key, and inserts into the table then need a
|
|
scalar index on that key. `maintained_indexes` chooses which indexes are kept
|
|
current against WAL rows; omitted, every supported index is kept current,
|
|
including ones created later. A table with a computed column cannot take a
|
|
write-ahead log. See
|
|
[RTE Compaction](/enterprise/realtime-engine-compaction) for how
|
|
the log gets folded back into the table.
|
|
|
|
<Warning>
|
|
[`TABLESAMPLE`](/sql/dql/sampling) is refused on a WAL-backed table, because
|
|
rows a sample would draw from are not all in the fragments yet.
|
|
</Warning>
|
|
|
|
### Errors and limits {#errors-and-limits}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `ALTER TABLE ... DROP COLUMN` | not supported |
|
|
|
|
Dropping, renaming, or retyping a column is an SDK operation
|
|
([`drop_columns`](/management/table#schema) and
|
|
[`alter_columns`](/management/table#schema)), not a SQL statement.
|
|
|
|
## CLONE {#clone}
|
|
|
|
{/* Draft for the table-clone exist_ok release.
|
|
|
|
Use `CREATE TABLE IF NOT EXISTS target CLONE source` to preserve an existing
|
|
target unchanged. The Python equivalent is
|
|
`db.clone_table("target", source_uri, exist_ok=True)`. Omitting the option reports
|
|
an error for an existing target. A skipped clone does not copy rows or change
|
|
the target's version history.
|
|
|
|
*/}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
CREATE TABLE docs_snapshot CLONE documents
|
|
CREATE TABLE docs_at_v42 CLONE documents AT 42
|
|
CREATE TABLE docs_at_release CLONE documents AT TAG 'release-1'
|
|
CREATE TABLE docs_yesterday CLONE documents AT TIMESTAMP '2026-09-01T00:00:00Z'
|
|
```
|
|
</CodeGroup>
|
|
|
|
A **shallow clone**: the new table shares the source's data files rather than
|
|
copying them, so it costs one metadata commit regardless of the source's size.
|
|
|
|
`AT` pins the point in the source's history to clone. A bare integer is a
|
|
version and a bare quoted string is a timestamp; `VERSION`, `TAG` and
|
|
`TIMESTAMP` say so explicitly, and a tag always needs its keyword: a bare
|
|
quoted string cannot be both a tag name and a timestamp.
|
|
|
|
<Warning>
|
|
The clone depends on the source's files continuing to exist. If the source is
|
|
[compacted](/maintenance/cleanup) past the retention window, the version the
|
|
clone points at can be pruned. The SDK's
|
|
[`clone_table`](/management/table#rename-clone) also offers a deep clone for
|
|
when the copy has to outlive the source's history.
|
|
</Warning>
|
|
|
|
## DROP TABLE {#drop}
|
|
|
|
<CodeGroup>
|
|
```sql SQL icon="database"
|
|
DROP TABLE documents
|
|
```
|
|
</CodeGroup>
|
|
|
|
<Warning>
|
|
Removes the table and every version of it. There is no undrop.
|
|
</Warning>
|
|
|
|
<Note>
|
|
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
|
|
deployment stores its catalog. See
|
|
[Object teardown](/basics/jobs#teardown).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Table" icon="table" href="/management/table">
|
|
The SDK equivalent, and what a table carries.
|
|
</Card>
|
|
<Card title="Schema evolution" icon="boxes-stacked" href="/management/table#schema">
|
|
Renaming, retyping and dropping columns from the SDK.
|
|
</Card>
|
|
</CardGroup>
|
|
|
|
## SHOW TABLES {#show}
|
|
|
|
```sql
|
|
SHOW TABLES IN analytics.`team$ml` LIMIT 100;
|
|
SHOW TABLES IN analytics.`team$ml` LIMIT 100 FROM '<page-token>';
|
|
```
|
|
|
|
Returns two non-null string columns: `namespace_path`, a compact JSON array
|
|
of parent namespace components (`[]` for root), and `name`, the unqualified
|
|
object name. Omitting `IN` lists all authorized namespaces in the current database;
|
|
`IN team$ml` selects that exact path without a database qualifier; `IN public` selects
|
|
the current database's root. Listings require the relevant database and
|
|
namespace `USAGE` privileges.
|
|
|
|
Page through results using `LIMIT n [FROM '<page-token>']`. Read the opaque
|
|
`lancedb.next_page_token` from the Arrow result schema metadata and pass it back
|
|
unchanged with the same object type and scope. Stop when no token is returned;
|
|
a short or empty page can still carry a continuation. See
|
|
[catalog listing pagination](/sql/pagination).
|