mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-13 07:12:27 +00:00
Jack Ye
5b5ed930ac
Every group now answers exactly one question. Get started and Concepts cover what the system is; the verb groups cover doing something with it; Operate, Secure and Deploy cover running it; SQL reference is lookup. Maintenance becomes Operate, Governance becomes Secure, and the rest take verbs to match. Three new pages in Concepts and Get started: Guarantees states what a write promises when it returns, which version a reader sees, what survives a failure, and -- the half usually missing -- what is not guaranteed: no multi-table transactions, no read-your-writes across processes without an interval, unenforced primary keys that do not enforce, no background compaction. Limits collects bounds that were stated in passing across a dozen pages, with the numbers verified against the code rather than inferred: name lengths, job event caps, SHOW ERRORS bounds, the sampling range that is refused rather than clamped. Architecture illustrates embedded against distributed as the one deployment decision, and is written to cover both rather than only Enterprise. Indexing splits by question. Choosing an index is a Query decision and moves there; building, repairing and GPU-building it stay in Operate. SQL becomes a top-level reference of fifteen pages instead of a sub-group of Query. Auditing sophon-sql turned up a large undocumented surface: ALTER TABLE ADD COLUMN in its plain, computed and function forms with FILTER, IGNORE NULLS and OVER modifiers; SET UNENFORCED PRIMARY KEY; SET and UNSET WAL with sharding; CREATE FUNCTION with its dependency options; REFRESH COLUMN and REFRESH MATERIALIZED VIEW; and the SHOW family. None of it was written down.
86 lines
3.3 KiB
Plaintext
86 lines
3.3 KiB
Plaintext
---
|
|
title: "SQL reference"
|
|
sidebarTitle: "Overview"
|
|
description: "The SQL surface: dialect, where a query runs, and a page per construct."
|
|
icon: table-list
|
|
keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> SQL runs on a deployment. A local database
|
|
has no engine to plan a statement, so `execute_query` raises there -- see
|
|
[SDKs and SQL](/basics/apis).
|
|
|
|
This is a reference rather than a guide: one page per construct, syntax first.
|
|
The guides elsewhere in these docs show the same operations from the SDKs.
|
|
|
|
## The dialect {#dialect}
|
|
|
|
LanceDB executes SQL with [Apache DataFusion](https://datafusion.apache.org),
|
|
currently **version 54**. The grammar, built-in functions and type system are
|
|
DataFusion's, so anything its reference documents for that version works here
|
|
unchanged and is not reproduced.
|
|
|
|
What these pages add is the part that is LanceDB's: how a query reaches the
|
|
engine, how Lance columns appear to it, and the statements a lakehouse needs
|
|
that DataFusion has no concept of.
|
|
|
|
## Where a query runs {#execution}
|
|
|
|
Queries execute on the server and results come back as Arrow record batches --
|
|
nothing is converted to pandas or Python objects unless you ask. `execute_query`
|
|
hands back a reader as soon as the first batch is ready rather than when the
|
|
query finishes, so a large result set does not have to fit in memory before you
|
|
start reading it. See [Query and job](/basics/jobs#queries).
|
|
|
|
Served over an [Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html)
|
|
endpoint, so any Flight SQL client reaches it.
|
|
|
|
## Reference {#reference}
|
|
|
|
**Reading data**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Data types](/sql/data-types) | how Lance columns appear to SQL, and pseudo-columns |
|
|
| [SELECT](/sql/select) | projection, filtering, ordering, set operations, CTEs |
|
|
| [Subqueries](/sql/subqueries) | scalar, `IN`, `EXISTS`, correlated, derived tables |
|
|
| [Joins](/sql/join) | inner, outer, cross, and namespace-qualified paths |
|
|
| [Aggregation](/sql/aggregation) | `GROUP BY`, `HAVING`, the aggregate set |
|
|
| [Window functions](/sql/window) | `OVER`, framing, ranking and offset functions |
|
|
| [Sampling](/sql/sampling) | `TABLESAMPLE`, by rows or by fragments |
|
|
|
|
**Changing things**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [DDL](/sql/ddl) | tables, views, indexes, functions, columns, WAL |
|
|
| [DML](/sql/dml) | `INSERT`, `UPDATE`, `DELETE`, `COPY` |
|
|
| [REFRESH](/sql/refresh) | filling computed columns and materialized views |
|
|
|
|
**Introspection**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [SHOW](/sql/show) | jobs, functions, materialized views, errors |
|
|
| [EXPLAIN](/sql/explain) | plans, and measuring them |
|
|
|
|
**Expressions**
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Operators](/sql/operators) | comparison, logical, arithmetic, literals, precedence |
|
|
| [Functions](/sql/functions) | the function set, and what Lance accelerates |
|
|
|
|
## What SQL cannot do {#cannot}
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| Vector search | no SQL form -- a query-builder operation |
|
|
| Merge insert | no SQL form -- needs a source relation and a per-match policy |
|
|
| Run on a local database | no engine to plan the statement |
|
|
|
|
Full-text search **is** available, as the [`fts()`](/sql/functions#fts) table
|
|
function and the [`contains_tokens`](/sql/functions#contains-tokens) predicate.
|
|
|
|
See [Limits](/basics/limits#sql) for the bounds these pages enforce.
|