mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-31 07:46:51 +00:00
Jack Ye
ddb318567a
## Summary - Complete the SQL reference: `ATTACH DATABASE`, the `CHANGES` clause, and `dedupe()`/`duplicate_pairs()` (verified against current sophon source; these were recently renamed from `vector_dedup`/`vector_duplicate_pairs`). Fix the blob-column default-projection claim and add the previously undocumented `blob_descriptor()`. - Enrich the gated Functions draft (still `.mintignore`'d, pre-release): chained Functions over blob outputs, GPU-produced blobs, Secrets, multi-input/output binding, `code=[module]` packaging. - Upgrade the materialized-view and computed-column auto-refresh drafts with precise, verified catch-up semantics, and add the missing `ALTER MATERIALIZED VIEW` / `CREATE ... WITH(...)` sections (also gated — the automatic-refresh scheduler is still dev/staging-only). - Thread one running example (video frames → GPU extract/caption → dedup → auto-refreshing materialized view) through Functions, materialized views, and the LeRobotDataset integration guide instead of a separate tutorial. - Rename "LanceDB OSS"/"OSS" to "LanceDB Embedded"/"Embedded" throughout (52 files); activate the existing availability-comparison mechanism on `basics/offerings.mdx` for a full feature-by-feature comparison table.
110 lines
4.7 KiB
Plaintext
110 lines
4.7 KiB
Plaintext
---
|
|
title: "SQL reference"
|
|
sidebarTitle: "Overview"
|
|
description: "SQL statements, expressions and data types supported by LanceDB Enterprise."
|
|
icon: table-list
|
|
keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"]
|
|
---
|
|
|
|
LanceDB Enterprise supports SQL through an Arrow Flight SQL endpoint. This
|
|
reference covers statement syntax, data types and expressions. To set up a
|
|
client and run your first query, see [SQL](/basics/sql).
|
|
|
|
## The dialect {#dialect}
|
|
|
|
LanceDB uses [Apache DataFusion](https://datafusion.apache.org) for SQL planning
|
|
and execution. These pages describe LanceDB's supported statements and
|
|
extensions. DataFusion's reference is useful for expression syntax, but does
|
|
not establish that every DataFusion statement or feature is exposed by
|
|
LanceDB.
|
|
|
|
## Connecting {#connecting}
|
|
|
|
Use Python's `execute_query` or a Flight SQL client after configuring the
|
|
[Enterprise connection](/basics/sql#sql-is-remote). TypeScript applications
|
|
use a Flight SQL client for SQL queries.
|
|
|
|
```python
|
|
reader = db.execute_query("SELECT id, text FROM my_docs LIMIT 10")
|
|
for batch in reader:
|
|
print(batch)
|
|
```
|
|
|
|
SQL results arrive as Arrow record batches. Iterating over the reader avoids
|
|
collecting the entire result in application memory. See
|
|
[Read and manage results](/basics/sql#results) for more.
|
|
|
|
## Usage {#usage}
|
|
|
|
The endpoint executes one statement per request and is meant for the queries
|
|
and DDL/DML pages below. Reach for the SDKs instead for table-management
|
|
operations that have no SQL form, such as a merge insert. See
|
|
[What SQL cannot do](#cannot).
|
|
|
|
## Reference {#reference}
|
|
|
|
### DDL, by object {#ddl}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Database](/sql/ddl/database) | `ATTACH DATABASE` |
|
|
| [Namespace](/sql/ddl/namespace) | `CREATE SCHEMA`, `DROP SCHEMA` |
|
|
| [Table](/sql/ddl/table) | `CREATE`, `DESCRIBE`, `ALTER`, `CLONE`, `DROP TABLE` |
|
|
| [View](/sql/ddl/view) | `CREATE VIEW`, `DROP VIEW` |
|
|
| [Materialized view](/sql/ddl/materialized-view) | `CREATE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS` |
|
|
| [Index](/sql/ddl/index) | `CREATE INDEX`, `DROP INDEX` |
|
|
| [Job](/sql/ddl/job) | `SHOW JOBS`, `SHOW ERRORS`, `PAUSE`/`RESUME`/`CANCEL JOB` |
|
|
|
|
{/*
|
|
Draft for the Functions release. Add this row to the table above, after Index:
|
|
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
|
|
*/}
|
|
|
|
### DML {#dml}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Insert](/sql/dml/insert) | `INSERT INTO`, including `INSERT ... SELECT` |
|
|
| [Update](/sql/dml/update) | `UPDATE ... SET` |
|
|
| [Delete](/sql/dml/delete) | `DELETE FROM` |
|
|
| [Copy](/sql/dml/copy) | `COPY` (refused by recent deployment versions) |
|
|
| [Refresh column](/sql/dml/refresh-column) | filling a computed column |
|
|
| [Refresh materialized view](/sql/dml/refresh-materialized-view) | bringing a view's stored result up to date |
|
|
|
|
### DQL {#dql}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Data types](/sql/data-types) | how Lance columns appear to SQL, and pseudo-columns |
|
|
| [Select](/sql/dql/select) | projection, filtering, ordering, set operations, CTEs |
|
|
| [Subqueries](/sql/dql/subqueries) | scalar, `IN`, `EXISTS`, correlated, derived tables |
|
|
| [Joins](/sql/dql/join) | inner, outer, cross, and namespace-qualified paths |
|
|
| [Aggregation](/sql/dql/aggregation) | `GROUP BY`, `HAVING`, the aggregate set |
|
|
| [Window functions](/sql/dql/window) | `OVER`, framing, ranking and offset functions |
|
|
| [Sampling](/sql/dql/sampling) | `TABLESAMPLE`, by rows or by fragments |
|
|
| [Changes](/sql/dql/changes) | `CHANGES`: rows appended since an earlier version |
|
|
| [Full-Text Search](/sql/dql/fts) | the `fts()` table function, and its query forms |
|
|
| [Explain](/sql/dql/explain) | `EXPLAIN`, `EXPLAIN ANALYZE`, `EXPLAIN VERBOSE` |
|
|
|
|
### Expressions {#expressions}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Operators](/sql/operators) | comparison, logical, arithmetic, literals, precedence |
|
|
| [Functions](/sql/functions) | where to find each kind, and table functions |
|
|
| [Scalar functions](/sql/scalar-functions) | the per-row catalogue, and what Lance accelerates |
|
|
| [Prepared statements](/sql/prepared-statements) | preparing through a Flight SQL client |
|
|
|
|
## SQL limitations and alternatives {#cannot}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| Vector search | See [SQL vector search](/basics/sql#no-vector-sql) for supported query forms and server-version requirements. The [SDK query builder](/search/vector-search) also supports vector search. |
|
|
| Merge insert | no SQL form: it needs a source relation and a per-match policy |
|
|
| `information_schema` | not enabled, so `SHOW TABLES` and `SHOW COLUMNS` fail |
|
|
| `COPY` | refused by recent deployment versions (see [Copy](/sql/dml/copy)) |
|
|
| `PREPARE` / `EXECUTE` | preparation is a [protocol feature](/sql/prepared-statements), not a statement |
|
|
| Run on a local database | no engine to plan the statement |
|
|
|
|
See [Limits](/basics/limits#sql) for the bounds these pages enforce.
|