Files
lancedb/docs/web/sql/index.mdx
T
Jack YeandClaude Sonnet 5 de95ed3f6d docs: overhaul curation, feature engineering, security and deployment guides (#4748)
A broad documentation pass across the areas LanceDB Enterprise and
Embedded
diverge on most:

**Data Curation** -- GPU Function execution, Slurm scheduling (and what
Slurm is, for readers new to it), blob late materialization as its own
page, joins, aggregation/profiling, sampling, near-duplicate detection,
chunking, distributed shuffle, and IVF-partition clustering.

**Feature Engineering** -- computed columns and materialized views,
restructured to lead with the Function-based path and show the SQL form
side by side with Python throughout; a single Auto Refresh page instead
of
duplicating scheduling details on every refresh page; adaptive
execution;
checkpointing; GPU and Slurm placement for refresh workers; a Training
Cache page for the PyTorch data loader's automatic cache benefit.

**Security & Governance** -- an Audit Log reference (schema, access,
grants), Secrets promoted to an ordinary catalog object with their own
SQL
DDL and a vending/rotation guide, and privilege-table fixes verified
against the authorization source.

**Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry
moved
into a real Deployment area; Automated Cleanup folded into Intelligent
Auto-Maintenance (every other maintenance decision already lived there);
RTE Compaction and Advanced Conflict Resolution documented in depth.

**Landing page** -- the Enterprise feature comparison table reorganized
(several redundant or stale rows removed, several new areas added), and
the
"why Enterprise" bullets replaced with ones tied to a concrete page
(Performance, Scale, Fault tolerance, Multicloud deployment, Simplify
operations) instead of vague promises.

Plus a full pass addressing lancedb-gatekeeper's review: a real
test-safety
bug (a quickstart test could silently overwrite and delete a
deployment's
own `frames` table), a fabricated SQL statement with no backing
implementation, an actual dedup-guarantee bug in `take_row_ids`, and
dozens
of documentation-accuracy fixes verified against source rather than
taken
on faith -- each either fixed, or left with an explicit note on why it
needs a feature or deployment owner's sign-off instead of being guessed
at.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-03 16:17:24 -08:00

104 lines
4.9 KiB
Plaintext

---
title: "SQL reference"
sidebarTitle: "Overview"
description: "SQL statements, expressions and data types supported by LanceDB Enterprise."
icon: table-list
keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"]
---
LanceDB Enterprise supports SQL through an Arrow Flight SQL endpoint. This
reference covers statement syntax, data types and expressions. To set up a
client and run your first query, see [SQL](/basics/sql).
## The dialect {#dialect}
LanceDB uses [Apache DataFusion](https://datafusion.apache.org) for SQL planning
and execution. These pages describe LanceDB's supported statements and
extensions. DataFusion's reference is useful for expression syntax, but does
not establish that every DataFusion statement or feature is exposed by
LanceDB.
## Connecting {#connecting}
Use Python's `execute_query` or a Flight SQL client after configuring the
[Enterprise connection](/basics/sql#sql-is-remote). TypeScript applications
use a Flight SQL client for SQL queries.
<CodeGroup>
```python Python icon="python"
reader = db.execute_query("SELECT id, text FROM my_docs LIMIT 10")
for batch in reader:
print(batch)
```
</CodeGroup>
SQL results arrive as Arrow record batches. Iterating over the reader avoids
collecting the entire result in application memory. See
[Read and manage results](/basics/sql#results) for more.
## Usage {#usage}
The endpoint executes one statement per request and is meant for the queries
and DDL/DML pages below. Reach for the SDKs instead for table-management
operations that have no SQL form, such as a merge insert. See
[What SQL cannot do](#cannot).
## Reference {#reference}
### DDL, by object {#ddl}
| | |
| --- | --- |
| [Database](/sql/ddl/database) | `CREATE`, `DESCRIBE`, `SHOW`, `ATTACH`, `DROP DATABASE` |
| [Namespace](/sql/ddl/namespace) | `CREATE`, `ALTER`, `DESCRIBE`, `SHOW`, `DROP SCHEMA`/`NAMESPACE` |
| [Table](/sql/ddl/table) | `CREATE`, `DESCRIBE`, `ALTER`, `CLONE`, `DROP TABLE` |
| [View](/sql/ddl/view) | `CREATE VIEW`, `DROP VIEW` |
| [Materialized view](/sql/ddl/materialized-view) | `CREATE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS` |
| [Secret](/sql/ddl/secret) | `CREATE`, `ALTER`, `DESCRIBE`, `SHOW`, `DROP SECRET` |
| [Storage Integration](/sql/ddl/storage-integration) | `CREATE`, `DESCRIBE`, `SHOW`, `DROP STORAGE INTEGRATION` |
| [Index](/sql/ddl/index) | `CREATE INDEX`, `DROP INDEX` |
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
| [Job](/sql/ddl/job) | `SHOW JOBS`, `SHOW JOB EVENTS`, `DESCRIBE JOB`, `SHOW ERRORS`, `PAUSE`/`RESUME`/`CANCEL JOB` |
### DML {#dml}
| | |
| --- | --- |
| [Insert](/sql/dml/insert) | `INSERT INTO`, including `INSERT ... SELECT` |
| [Update](/sql/dml/update) | `UPDATE ... SET` |
| [Delete](/sql/dml/delete) | `DELETE FROM` |
| [Copy](/sql/dml/copy) | `COPY` (refused by recent deployment versions) |
| [Refresh column](/sql/dml/refresh-column) | filling a computed column |
| [Refresh materialized view](/sql/dml/refresh-materialized-view) | bringing a view's stored result up to date |
### DQL {#dql}
| | |
| --- | --- |
| [Data types](/sql/data-types) | how Lance columns appear to SQL, and pseudo-columns |
| [Select](/sql/dql/select) | projection, filtering, ordering, set operations, CTEs |
| [Subqueries](/sql/dql/subqueries) | scalar, `IN`, `EXISTS`, correlated, derived tables |
| [Joins](/sql/dql/join) | inner, outer, cross, and namespace-qualified paths |
| [Aggregation](/sql/dql/aggregation) | `GROUP BY`, `HAVING`, the aggregate set |
| [Window functions](/sql/dql/window) | `OVER`, framing, ranking and offset functions |
| [Sampling](/sql/dql/sampling) | `TABLESAMPLE`, by rows or by fragments |
| [Changes](/sql/dql/changes) | `CHANGES`: rows appended since an earlier version |
| [Full-Text Search](/sql/dql/fts) | the `fts()` table function, and its query forms |
| [Operators](/sql/dql/operators) | comparison, logical, arithmetic, literals, precedence |
| [Functions](/sql/dql/functions) | where to find each kind, and table functions |
| [Scalar functions](/sql/dql/scalar-functions) | the per-row catalogue, and what Lance accelerates |
| [Explain](/sql/dql/explain) | `EXPLAIN`, `EXPLAIN ANALYZE`, `EXPLAIN VERBOSE` |
## SQL limitations and alternatives {#cannot}
| | |
| --- | --- |
| Vector search | See [SQL vector search](/basics/sql#no-vector-sql) for supported query forms and server-version requirements. The [SDK query builder](/search/vector-search) also supports vector search. |
| Merge insert | no SQL form: it needs a source relation and a per-match policy |
| `information_schema` | not enabled, so `SHOW TABLES` and `SHOW COLUMNS` fail |
| `COPY` | refused by recent deployment versions (see [Copy](/sql/dml/copy)) |
| `PREPARE` / `EXECUTE` | preparation is a [protocol feature](/basics/sql#prepared-statements), not a statement |
| Run on a local database | no engine to plan the statement |
See [Limits](/basics/limits#sql) for the bounds these pages enforce.