mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 14:16:54 +00:00
## Summary Fixes the problems a page-by-page review of the docs found before launch, after checking each one against the current source and, where noted, the code. The matching fixes for the dataset and Geneva pages, which live in `lancedb/docs`, are in lancedb/docs#402. **Tables** - `integrations/embedding/cohere`: each input-type description was split across two table rows and wrapped in literal quotes (`"`search_document`"`). Now one row per type. - `integrations/embedding/gemini`: removes the same stray quotes from the task-type table. - `integrations/ai/langchain`: the MMR search table's `kwargs` row was missing a cell, so its description sat under Type. - Six reranker pages (answerdotai, cohere, colbert, jina, openai, voyageai): closes the backtick before `_relevance_score`. - `indexing/vector-index`: finishes a cell cut off at "with scalar". - `sql/dql/dedupe`: labels the empty header cell in the resolution table. **Corrections** - `curation/sampling`: sampling works in LanceDB Embedded through the Python SDK, so the page is available in both, with the SQL sections marked Enterprise. - `feature-engineering/index`, `add-computed-column`: automatic refresh is on by default for each computed column and takes effect once a deployment enables its scheduler. `refresh-column` already said so; these two implied it was off. - `indexing/scalar-index`, `indexing/index`: `LABEL_LIST` accelerates `array_has_all`/`array_has_any`; `array_contains_*` doesn't exist. Also merges scalar-index's two "nested fields" sections into one. - `integrations/embedding/ibm`: lists the four current models from `CURRENT_MODELS` (it listed a deprecated one) and explains that the default stays a legacy ID so existing tables reload with their model. - `integrations/reranking/openai`: the class is `OpenaiReranker`, not `OpenAI()`. - `integrations/ai/synthetic-data-kit`: the CLI is `synthetic-data-kit` throughout, each step runs one command, each step passes the file it writes to the next, and `save-as` uses `-f`/`-o`. - `integrations/data/dlt`: `secrets.toml` defines `api_token`, which the pipeline reads, and the run command runs the file the page creates. - `search/vector-search`: the multivector walkthrough connected to Enterprise in step 1 and a local path in step 2. It now runs locally throughout, with a note on connecting to Enterprise, and is renamed so it doesn't read as a duplicate of the short section above. **Structure and wording** - `integrations/embedding/nvidia`: adds headings, and drops test-suite details aimed at contributors. - `integrations/ai/genkit`: makes the retrieval test a real heading and describes the query step. - `integrations/data/voxel51`: replaces a raw MkDocs `!!! note` with a `<Note>`. - `sql/ddl/table`, `sql/ddl/job`: "Errors and limits" is a real heading, not bold text. - `curation/windowing`: sidebar title matches the curation overview ("Rank and Deduplicate"). - `deployment/storage`, `basics/agentic-skills`: fixes a garbled sentence and a stale reference to the quickstart. **Checked:** no new Vale alerts on the changed pages. Assembled together with lancedb/docs#402, the site passes `mint validate` and `mint broken-links --check-anchors --check-redirects`. Every changed page was reviewed in a local preview. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
246 lines
8.5 KiB
Plaintext
246 lines
8.5 KiB
Plaintext
---
|
|
title: "Table"
|
|
sidebarTitle: "Table"
|
|
description: "CREATE, DESCRIBE, ALTER, CLONE and DROP TABLE."
|
|
icon: table
|
|
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
|
|
availability:
|
|
embedded: unavailable
|
|
enterprise: available
|
|
---
|
|
|
|
## CREATE TABLE {#create}
|
|
|
|
```sql
|
|
CREATE TABLE documents (id BIGINT, title VARCHAR, created_at TIMESTAMP)
|
|
CREATE TABLE recent AS SELECT * FROM documents WHERE created_at > now() - INTERVAL '7 days'
|
|
```
|
|
|
|
Column types are Arrow types under SQL names. See [Data types](/sql/data-types).
|
|
|
|
A table inside a [namespace](/sql/ddl/namespace) is written as a dotted path:
|
|
`prod.events.clicks`.
|
|
|
|
## DESCRIBE {#describe}
|
|
|
|
```sql
|
|
DESCRIBE documents
|
|
```
|
|
|
|
The table's columns and types, without a plan. The SQL counterpart of
|
|
`table.schema`.
|
|
|
|
## ALTER TABLE {#alter}
|
|
|
|
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
|
|
name.
|
|
|
|
### A plain column {#plain-column}
|
|
|
|
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
|
|
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
|
|
with the SDK instead. See
|
|
[Add nullable columns](/management/table#schema-add-nullable-columns).
|
|
|
|
### A computed column {#computed-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
```
|
|
|
|
`AS <expression>` declares a [computed column](/feature-engineering/add-computed-column).
|
|
The column is committed empty and the expression is stored, not evaluated:
|
|
declaring one on a billion-row table costs what declaring one on an empty table
|
|
costs. [`REFRESH COLUMN`](/sql/dml/refresh-column) fills it.
|
|
|
|
{/*
|
|
Draft for automatic computed-column refresh. Enable this section when the
|
|
server feature is released.
|
|
|
|
### Automatic refresh options {#automatic-refresh-options}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN doubled AS value * 2
|
|
WITH (auto_refresh_enabled = true, auto_refresh_interval_seconds = 300);
|
|
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = false;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_interval_seconds = 600;
|
|
ALTER TABLE t ALTER COLUMN doubled SET auto_refresh_enabled = true;
|
|
```
|
|
|
|
Automatic refresh is enabled by default. An explicitly configured interval is
|
|
stored in each computed output's field metadata; when omitted, the server
|
|
determines when to refresh. The interval must be a positive integer
|
|
and is minimum pacing after a successful refresh of that output; unchanged
|
|
inputs do not force work. A new column is immediately eligible for its first
|
|
refresh. The server's separate automatic-refresh switch must also be enabled.
|
|
|
|
These options also work beside a Function binding's `version` and `branch`,
|
|
continuing the [`caption` Function](/management/function#chain) from Functions:
|
|
|
|
```sql
|
|
ALTER TABLE frames ADD COLUMN caption AS caption(extract_frame(video, ts))
|
|
WITH (version = 'fv_1', auto_refresh_enabled = true,
|
|
auto_refresh_interval_seconds = 300);
|
|
ALTER TABLE frames ALTER COLUMN caption SET auto_refresh_enabled = false;
|
|
```
|
|
|
|
Declaration settings initialize every output in the group; later SET commands
|
|
change one output. SET takes effect immediately and does not itself read or
|
|
write any row:
|
|
|
|
- **Disabling** prevents new automatic submissions. Inflight jobs may still
|
|
finish, explicit `REFRESH COLUMN` keeps working, and rows written while
|
|
disabled commit with no computed value -- they do not queue anywhere
|
|
separate.
|
|
- **Re-enabling catches up the entire backlog in one pass.** The next
|
|
scheduler tick processes every row left unfilled since the column was
|
|
disabled, immediately, with no further write required to trigger it: a
|
|
column disabled for an hour and re-enabled fills everything that
|
|
accumulated during that hour, not just rows written after the moment it
|
|
is re-enabled.
|
|
|
|
Column SUSPEND/RESUME syntax is not supported; `SET auto_refresh_enabled` is
|
|
the only toggle. The current setting is in the column's own field metadata,
|
|
`computed_column.auto_refresh_enabled`, readable from `DESCRIBE` or the
|
|
SDK's `table.schema` without submitting a job to ask.
|
|
*/}
|
|
|
|
{/*
|
|
Draft for the Functions release.
|
|
|
|
### A function column {#function-column}
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN embedded AS embed(name) WITH (version = 'fv_1')
|
|
```
|
|
|
|
When the expression calls a [registered function](/sql/ddl/function), the
|
|
column is filled by running that function on workers. `version` pins which
|
|
immutable function version fills it, so a later registration does not silently
|
|
change what the column means.
|
|
|
|
Several modifiers narrow what gets computed:
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `FILTER (WHERE pred)` | only rows matching `pred` are computed |
|
|
| `IGNORE NULLS` | rows with null inputs are skipped |
|
|
| `OVER (PARTITION BY col)` | the function sees rows grouped by `col` |
|
|
|
|
A function returning several values fills several columns at once:
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN (label, score) AS classify(text) WITH (version = 'fv_1')
|
|
```
|
|
|
|
```sql
|
|
ALTER TABLE t ADD COLUMN c AS f(x) FILTER (WHERE keep) WITH (version = 'fv_1')
|
|
REFRESH COLUMN c ON t WITH (num_workers = 2)
|
|
```
|
|
|
|
<Note>
|
|
Declaration `WITH` accepts `auto_refresh_enabled` and
|
|
`auto_refresh_interval_seconds`. Function bindings also accept `version` and
|
|
`branch`. Worker resource options such as `num_workers` belong on
|
|
`REFRESH COLUMN ... WITH (...)`.
|
|
</Note>
|
|
*/}
|
|
|
|
### Table properties {#properties}
|
|
|
|
```sql
|
|
ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
|
|
```
|
|
|
|
Records which column identifies a row. **Unenforced** is not a hedge in the
|
|
naming: uniqueness is not checked on write, and declaring it does not make
|
|
duplicates impossible. It tells the planner what to assume. The column must be
|
|
declared `NOT NULL`.
|
|
|
|
```sql
|
|
ALTER TABLE t SET RTE
|
|
ALTER TABLE t SET RTE WITH (sharding = 'bucket', column = 'id', buckets = 4)
|
|
ALTER TABLE t SET RTE WITH (maintained_indexes = '')
|
|
ALTER TABLE t UNSET RTE
|
|
```
|
|
|
|
The [real time engine](/tables/realtime-engine) puts a write-ahead log in front
|
|
of the table, making small frequent writes cheap by batching them out of the
|
|
commit path. `sharding` and `buckets` spread the log so concurrent writers do
|
|
not contend on one place; `bucket` sharding names its `column`, which must be
|
|
the table's unenforced primary key, and inserts into the table then need a
|
|
scalar index on that key. `maintained_indexes` chooses which indexes are kept
|
|
current against WAL rows; omitted, every supported index is kept current,
|
|
including ones created later. A table with a computed column cannot take a
|
|
write-ahead log. See
|
|
[Background jobs](/maintenance/background-jobs#wal) for how the log gets
|
|
folded back into the table.
|
|
|
|
<Warning>
|
|
[`TABLESAMPLE`](/sql/dql/sampling) is refused on a WAL-backed table, because
|
|
rows a sample would draw from are not all in the fragments yet.
|
|
</Warning>
|
|
|
|
### Errors and limits {#errors-and-limits}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| `ALTER TABLE ... DROP COLUMN` | not supported |
|
|
|
|
Dropping, renaming, or retyping a column is an SDK operation
|
|
([`drop_columns`](/management/table#schema) and
|
|
[`alter_columns`](/management/table#schema)), not a SQL statement.
|
|
|
|
## CLONE {#clone}
|
|
|
|
```sql
|
|
CREATE TABLE docs_snapshot CLONE documents
|
|
CREATE TABLE docs_at_v42 CLONE documents AT 42
|
|
CREATE TABLE docs_at_release CLONE documents AT TAG 'release-1'
|
|
CREATE TABLE docs_yesterday CLONE documents AT TIMESTAMP '2026-09-01T00:00:00Z'
|
|
```
|
|
|
|
A **shallow clone**: the new table shares the source's data files rather than
|
|
copying them, so it costs one metadata commit regardless of the source's size.
|
|
|
|
`AT` pins the point in the source's history to clone. A bare integer is a
|
|
version and a bare quoted string is a timestamp; `VERSION`, `TAG` and
|
|
`TIMESTAMP` say so explicitly, and a tag always needs its keyword: a bare
|
|
quoted string cannot be both a tag name and a timestamp.
|
|
|
|
<Warning>
|
|
The clone depends on the source's files continuing to exist. If the source is
|
|
[compacted](/maintenance/cleanup) past the retention window, the version the
|
|
clone points at can be pruned. The SDK's
|
|
[`clone_table`](/management/table#rename-clone) also offers a deep clone for
|
|
when the copy has to outlive the source's history.
|
|
</Warning>
|
|
|
|
## DROP TABLE {#drop}
|
|
|
|
```sql
|
|
DROP TABLE documents
|
|
```
|
|
|
|
<Warning>
|
|
Removes the table and every version of it. There is no undrop.
|
|
</Warning>
|
|
|
|
<Note>
|
|
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
|
|
deployment stores its catalog. See
|
|
[Background jobs](/maintenance/background-jobs#teardown).
|
|
</Note>
|
|
|
|
## What to read next {#next}
|
|
|
|
<CardGroup cols={2}>
|
|
<Card title="Table" icon="table" href="/management/table">
|
|
The SDK equivalent, and what a table carries.
|
|
</Card>
|
|
<Card title="Schema evolution" icon="boxes-stacked" href="/management/table#schema">
|
|
Renaming, retyping and dropping columns from the SDK.
|
|
</Card>
|
|
</CardGroup>
|