rust/lancedb
1
0
Fork 0
mirror of https://github.com/lancedb/lancedb.git synced 2026-10-16 16:52:34 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: correct data workflows, SQL and operations (#4534)

Browse Source 
This PR is part of the documentation revamp. Once the full set of PRs is
merged, we’ll publish the combined site to staging for review before
updating docs.lancedb.com.

The curation, SQL and operations pages contain outdated claims and
examples that do not reliably work as written.

This corrects those pages against the reviewed SDK and local server
evidence, including prepared statements, background jobs, cleanup and
monitoring. Integration guides are grouped with their examples in #4533.

Function and Secret-object documentation stays unpublished until its
release. The two Function pages are listed in `.mintignore` and left out
of the navigation, and the sections about them on other pages are MDX
comments.

Stacked on #4533. Each MDX file has one PR owner. Rebase onto main after
#4533 is squash-merged.

Local assembly, Mint validation and link checks pass, and the assembled
site shows no Function or Secret-object content. A local release
rehearsal that uncomments the drafts and restores the navigation also
passes. Hold publication until this PR and lancedb/sophon#8688 are on
`main`.
This commit is contained in:
Brendan Clement
2026-10-15 14:57:20 -07:00
committed by GitHub
parent 758f738b8c
commit 06aa526ae1
72 changed files with 1997 additions and 1254 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/web/sql/data-types.mdx
+2 -2
View File
@@ -6,7 +6,7 @@ icon: shapes
keywords: ["sql types", "arrow", "cast", "pseudo-columns", "vector", "blob", "json"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
There is no SQL type layer over Lance storage. The table provider hands
DataFusion the dataset's Arrow schema directly, so a column is the same object
@@ -77,7 +77,7 @@ SELECT id FROM documents WHERE vector IS NOT NULL
<Warning>
A list representation does not imply that ordinary arithmetic operators
perform vector arithmetic. Use supported distance functions for similarity
calculations. SQL vector search depends on the deployment version and query
calculations. SQL vector search depends on the Enterprise server version and query
plan; see [SQL](/basics/sql#no-vector-sql).
</Warning>
docs/web/sql/ddl/function.mdx
+8 -3
View File
@@ -6,17 +6,22 @@ icon: function
keywords: ["create function", "drop function", "show functions", "sql", "udf"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
## CREATE FUNCTION {#create}
```sql
CREATE FUNCTION f(x BIGINT) RETURNS BIGINT
CREATE FUNCTION f(x int64) RETURNS int64
LANGUAGE python
AS 'def f(x): return x * 2'
WITH (pip = 'numpy==1.26.4', python_version = '3.11')
```
Parameter and result types are Arrow type names, not SQL ones: `int64`,
`float64`, `utf8`, `bool`, `date32`, and lists such as `list<float32>` or
`fixed_size_list<float32, 768>`. `BIGINT` is refused. A parameter accepts nulls
unless it is declared `NOT NULL`.
The body is Python source. Its dependencies are declared with it and are part
of the **immutable function version** -- changing one produces a new version
rather than altering the registered one, so a column filled by an earlier
@@ -27,7 +32,7 @@ version is unaffected.
| `pip` | pip requirements for the worker environment |
| `conda` | conda packages instead of pip |
| `conda_channels` | channels in priority order; requires `conda` |
| `python_version` | worker Python version, defaulting to yours |
| `python_version` | worker Python version; `3.12` when omitted, whatever Python you send the statement from |
| `gpu` | `'true'` when every execution needs a GPU |
**Errors and limits**
docs/web/sql/ddl/index.mdx
+21 -11
View File
@@ -1,30 +1,38 @@
---
title: "Index"
sidebarTitle: "Index"
description: "CREATE INDEX and DROP INDEX — one column, server-chosen name."
description: "CREATE INDEX and DROP INDEX — one column, with the algorithm chosen by USING."
icon: bolt
keywords: ["create index", "drop index", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
## CREATE INDEX {#create}
```sql
CREATE INDEX ON documents (title)
CREATE INDEX ON documents USING btree (title)
CREATE INDEX text_search ON documents USING fts (text)
```
`USING` is required: it names the algorithm. The scalar algorithms are `btree`,
`bitmap` and `label_list`; `fts` builds a full-text index; and the vector
algorithms are `ivf_pq`, `ivf_rq`, `ivf_flat`, `ivf_hnsw_sq` and `ivf_hnsw_pq`,
where the deployment supports them. Index options go in a `WITH (...)` clause.
<Note>
SQL `CREATE INDEX` covers **one column** and takes **no user-chosen name** --
the server names it. For a multi-column or explicitly-named index, or to
choose the algorithm, use the SDK's
[`create_index`](/indexing).
In LanceDB Enterprise, SQL `CREATE INDEX` covers **one column**. A name between
`INDEX` and `ON` names the index, where the deployment's configuration allows
it; without one, the server names it after the column, as in `title_idx`. For a
multi-column index, use the SDK's [`create_index`](/indexing).
</Note>
<Note>
**Enterprise job type:** `create_ivf_pq`, `create_btree`, `create_fts`, and one
more per algorithm -- the type names which algorithm is building, not a
different feature. See [Build & Repair](/indexing/reindexing).
**Enterprise job type:** `reindex_` followed by the algorithm, such as
`reindex_btree` or `reindex_ivf_pq`; the name says which algorithm is building,
not a different feature. An SDK build that must not replace an existing index
runs as `create_` followed by the algorithm instead. See
[Build & Repair](/indexing/reindexing).
</Note>
## DROP INDEX {#drop}
@@ -35,7 +43,9 @@ DROP INDEX documents.title_idx
Takes the qualified `table.index` name. Find it with the SDK's
[`list_indices`](/indexing) if you created the index without
naming it yourself.
naming it yourself. A deployment builds the index in the background after
`CREATE INDEX` returns; until that build finishes, the index is not listed and
`DROP INDEX` reports it as not found.
## What to read next {#next}
docs/web/sql/ddl/job.mdx
+2 -2
View File
@@ -6,7 +6,7 @@ icon: hourglass-half
keywords: ["show jobs", "show errors", "pause job", "resume job", "cancel job", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A [job](/basics/jobs) is not a catalog object -- it has no schema and is not
namespaced -- but it has its own SQL statements, grouped here the same way the
@@ -73,7 +73,7 @@ being ignored.
Cancellation is a request, matching the SDK's
[`cancel()`](/basics/jobs#managing): it does not guarantee the job stops before
doing more work, and asking twice or cancelling a finished job does nothing.
`CANCEL JOB` itself can be unavailable on a deployment with no job registry
`CANCEL JOB` itself can be unavailable on an Enterprise cluster with no job registry
configured, which is a deployment condition rather than something you can work
around from SQL.
</Note>
docs/web/sql/ddl/materialized-view.mdx
+11 -12
View File
@@ -6,13 +6,13 @@ icon: table
keywords: ["materialized view", "create materialized view", "show materialized views", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A materialized view stores a query's result as rows, kept current by a refresh
-- the counterpart to a [view](/sql/ddl/view), which stores only the query.
<Note>
Materialized views are off by default on a deployment and have to be enabled
Materialized views are off by default in LanceDB Enterprise and have to be enabled
there. Which statements and query shapes a deployment accepts, whether it needs
the source table to have stable row ids, and how it refreshes a view depend on
its LanceDB Enterprise version and configuration.
@@ -24,10 +24,9 @@ its LanceDB Enterprise version and configuration.
CREATE MATERIALIZED VIEW <name> AS SELECT ...
```
On a `db://` connection, the SDK's
[`create_materialized_view`](/management/materialized-view#create) sends this
statement. Once created, a view is queried like a table, refreshed with SQL and
listed with SQL.
Once created, a materialized view can be queried like a table. Use the
statements below to list, refresh and drop it. For SDK methods, see
[Materialized views](/management/materialized-view#create).
## SHOW MATERIALIZED VIEWS {#show}
@@ -36,8 +35,8 @@ SHOW MATERIALIZED VIEWS
```
<Note>
Cheaper than the SDK equivalent. `list_materialized_views` finds views by
reading every table's schema, costing one open per table; this asks the server.
This asks the server for its views, as the SDK's `list_materialized_views` does
on a `db://` connection.
</Note>
## Refreshing one {#refresh}
@@ -47,7 +46,7 @@ REFRESH MATERIALIZED VIEW chunks
```
See [REFRESH](/sql/dml/refresh#refresh-materialized-view) for the full syntax,
including `CASCADE` and `full = true`.
and compatibility notes for older server versions.
## Dropping one {#drop}
@@ -57,8 +56,8 @@ stored rows, unlike dropping a [view](/sql/ddl/view#drop).
<Note>
**Enterprise job type:** `mat_view_refresh` for a refresh, or `sql` on a
deployment that rebuilds views through SQL; `drop_table` for dropping one, same
as any table.
deployment that rebuilds views through SQL. Dropping one runs the same job as
dropping any table, `drop_table` or `drop_object`.
</Note>
## What to read next {#next}
@@ -68,6 +67,6 @@ as any table.
Creating one from the SDK.
</Card>
<Card title="Refresh a materialized view" icon="arrows-rotate" href="/feature-engineering/materialized-views">
How a refresh works, locally and on a deployment.
How a refresh works, locally and in LanceDB Enterprise.
</Card>
</CardGroup>
docs/web/sql/ddl/namespace.mdx
+10 -8
View File
@@ -6,7 +6,7 @@ icon: folder-tree
keywords: ["namespace", "schema", "create schema", "drop schema", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A [namespace](/management/namespace) is addressed in SQL as a `SCHEMA` -- the
standard's name for the same concept.
@@ -14,14 +14,16 @@ standard's name for the same concept.
## CREATE SCHEMA {#create}
```sql
CREATE SCHEMA public
CREATE SCHEMA db1.ns1.ns2
CREATE SCHEMA analytics
CREATE SCHEMA db1.`ns1$ns2`
CREATE SCHEMA IF NOT EXISTS `ns1$ns2`
```
A dotted path creates the whole chain: `db1.ns1.ns2` creates `ns2` inside
`ns1` inside `db1`, provided the parents already exist. A name containing the
namespace path delimiter itself is backtick-quoted, as in `` `ns1$ns2` `` above.
A nested namespace is one backtick-quoted name with `$` between its
components: `` db1.`ns1$ns2` `` creates `ns2` inside `ns1` in database `db1`,
provided `ns1` already exists. The part before the dot is the database, so a
three-part path such as `db1.ns1.ns2` is rejected. `public` is the database's
root namespace, which is never created or dropped.
`IF NOT EXISTS` makes the statement a no-op rather than an error when the
namespace is already there.
@@ -29,14 +31,14 @@ namespace is already there.
## DROP SCHEMA {#drop}
```sql
DROP SCHEMA db1.public
DROP SCHEMA db1.analytics
DROP SCHEMA IF EXISTS `a$b`
DROP SCHEMA `a$b` RESTRICT
```
<Warning>
`CASCADE` is rejected: **"DROP SCHEMA CASCADE is not supported; drop the
objects inside first."** A namespace's tables, views and functions must be
objects inside first."** A namespace's tables, views and other objects must be
dropped explicitly before the namespace itself can go. `RESTRICT` -- the
default -- states that requirement rather than changing it.
</Warning>
docs/web/sql/ddl/table.mdx
+21 -13
View File
@@ -6,7 +6,7 @@ icon: table
keywords: ["create table", "alter table", "drop table", "describe", "clone", "ddl"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
## CREATE TABLE {#create}
@@ -30,16 +30,15 @@ The table's columns and types, without a plan -- the SQL counterpart of
## ALTER TABLE {#alter}
`ALTER TABLE ... ADD COLUMN` covers three different things, distinguished by
what follows the column name.
What `ALTER TABLE ... ADD COLUMN` does depends on what follows the column
name.
### A plain column {#plain-column}
```sql
ALTER TABLE t ADD COLUMN c INT
```
Added empty and nullable. Nothing is computed.
SQL `ADD COLUMN` requires an `AS` binding, so a plain column such as
`ALTER TABLE t ADD COLUMN c INT` is rejected. Add an empty, nullable column
with the SDK instead -- see
[Add nullable columns](/management/table#schema-add-nullable-columns).
### A computed column {#computed-column}
@@ -52,6 +51,9 @@ The column is committed empty and the expression is stored, not evaluated --
declaring one on a billion-row table costs what declaring one on an empty table
costs. [`REFRESH COLUMN`](/sql/dml/refresh) fills it.
{/*
Draft for the Functions release.
### A function column {#function-column}
```sql
@@ -86,6 +88,7 @@ ALTER TABLE t ADD COLUMN c AS f(x) WITH (num_workers = '2')
`WITH` on a plain computed column accepts only `version`. Resource options such
as `num_workers` apply to function columns, where there are workers to size.
</Note>
*/}
### Table properties {#properties}
@@ -95,19 +98,23 @@ ALTER TABLE t SET UNENFORCED PRIMARY KEY (id)
Records which column identifies a row. **Unenforced** is not a hedge in the
naming: uniqueness is not checked on write, and declaring it does not make
duplicates impossible. It tells the planner what to assume.
duplicates impossible. It tells the planner what to assume. The column must be
declared `NOT NULL`.
```sql
ALTER TABLE t SET WAL
ALTER TABLE t SET WAL WITH (sharding = 'bucket', buckets = 4)
ALTER TABLE t SET WAL WITH (sharding = 'bucket', column = 'id', buckets = 4)
ALTER TABLE t SET WAL WITH (maintained_indexes = '')
ALTER TABLE t UNSET WAL
```
A write-ahead log makes small frequent writes cheap by batching them out of the
commit path. `sharding` and `buckets` spread the log so concurrent writers do
not contend on one place; `maintained_indexes` chooses which indexes are kept
current against WAL rows. See
not contend on one place; `bucket` sharding names its `column`, which must be
the table's unenforced primary key, and inserts into the table then need a
scalar index on that key. `maintained_indexes` chooses which indexes are kept
current against WAL rows. A table with a computed column cannot take a
write-ahead log. See
[Background jobs](/maintenance/background-jobs#wal) for how the log gets
folded back into the table.
@@ -162,7 +169,8 @@ Removes the table and every version of it. There is no undrop.
</Warning>
<Note>
**Enterprise job type:** `drop_table`. See
**Enterprise job type:** `drop_table` or `drop_object`, depending on how the
deployment stores its catalog. See
[Background jobs](/maintenance/background-jobs#teardown).
</Note>
docs/web/sql/ddl/view.mdx
+1 -1
View File
@@ -6,7 +6,7 @@ icon: eye
keywords: ["create view", "drop view", "sql", "catalog"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A view stores a **query**, not rows. Reading it runs the query against current
data, so a view is never stale -- and never faster than the query it wraps. It
docs/web/sql/dml/copy.mdx
+5 -1
View File
@@ -6,7 +6,11 @@ icon: file-export
keywords: ["copy", "export", "parquet", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Warning>
Recent deployment versions refuse `COPY` over the SQL endpoint with "COPY is not
supported". On those, export a result by reading it through a Flight SQL client
or an SDK and writing the files from your application.
</Warning>
```sql
COPY (SELECT id, title FROM documents) TO 's3://bucket/export/' STORED AS PARQUET
docs/web/sql/dml/delete.mdx
+6 -2
View File
@@ -6,12 +6,16 @@ icon: trash
keywords: ["delete", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
```sql
DELETE FROM documents WHERE created_at < '2025-01-01'
DELETE FROM documents WHERE created_at < TIMESTAMP '2025-01-01 00:00:00'
```
Compare a timestamp column with a typed literal, as above, or with
`CAST('2025-01-01' AS TIMESTAMP)`. A bare string such as `'2025-01-01'` is
rejected in a `DELETE` predicate, even though a `SELECT` accepts it.
Deletion marks rows rather than erasing them, so the data remains recoverable
for as long as the version holding it survives
[retention](/maintenance/cleanup).
docs/web/sql/dml/insert.mdx
+1 -1
View File
@@ -6,7 +6,7 @@ icon: plus
keywords: ["insert", "insert into", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
```sql
INSERT INTO documents (id, title) VALUES (1, 'Merlin'), (2, 'Arthur')
docs/web/sql/dml/refresh.mdx
+14 -12
View File
@@ -6,7 +6,7 @@ icon: arrows-rotate
keywords: ["refresh column", "refresh materialized view", "backfill", "cascade"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
LanceDB extensions with no DataFusion equivalent. They are what makes
[feature engineering](/feature-engineering) reachable from SQL: a column
@@ -52,33 +52,35 @@ REFRESH COLUMN embedding ON documents
`num_workers` sets a count; the budget options size each worker. They are
alternatives rather than a pair.
{/*
Draft for the Functions release.
<Note>
`error_policy` and `num_workers` are refused for a column filled by a registered
[function](/management/function) with its own resource budget -- use the CPU,
memory and GPU options there.
</Note>
*/}
## REFRESH MATERIALIZED VIEW {#refresh-materialized-view}
```sql
REFRESH MATERIALIZED VIEW <name> [CASCADE] [WITH (<option> = <value>)]
REFRESH MATERIALIZED VIEW <name>
```
```sql
REFRESH MATERIALIZED VIEW chunks
REFRESH MATERIALIZED VIEW chunks WITH (full = true)
```
`full = true` asks for every row to be recomputed. Without it, whether a
refresh rebuilds the whole view -- as every local SDK refresh does -- or
reconciles only what changed in the source depends on the deployment's LanceDB
Enterprise version and configuration.
A refresh updates the view's stored result. Refresh views in dependency order,
starting with their source views.
<Warning>
`CASCADE` asks for the views this one is built on to be refreshed first, in
dependency order. Whether a deployment carries it out depends on its version;
where it does not, refresh each view yourself, its sources first.
</Warning>
<Note>
Some older server versions accepted `WITH (full = true)` and `CASCADE`.
Those options are not accepted by the SQL implementation described here.
Use the basic statement above; do not assume that the options accepted by a
local SDK's `refresh()` method are also SQL options.
</Note>
## Watching a refresh {#watching}
docs/web/sql/dml/update.mdx
+1 -1
View File
@@ -6,7 +6,7 @@ icon: pen
keywords: ["update", "sql"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
```sql
UPDATE documents SET title = upper(title) WHERE category = 'film'
docs/web/sql/dql/aggregation.mdx
+17 -15
View File
@@ -6,11 +6,10 @@ icon: layer-group
keywords: ["group by", "having", "count", "sum", "avg", "aggregate", "approximate"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
Aggregation is one of the two main reasons to reach for SQL rather than a query
builder -- the SDKs project, filter and search, but they do not group. The other
is [joins](/sql/dql/join).
Use SQL aggregates to summarize records, either across a whole table or
within groups. Common examples include counts by category, totals and averages.
## Grouping {#grouping}
@@ -22,9 +21,8 @@ HAVING count(*) > 10
ORDER BY n DESC
```
`WHERE` filters rows before grouping; `HAVING` filters groups after. Putting a
row-level predicate in `HAVING` still gives the right answer but does more work,
because rows that could have been discarded early are grouped first.
`WHERE` filters rows before grouping; `HAVING` filters groups after. Use `WHERE` for row-level conditions and `HAVING` for conditions on aggregate
results. The planner may push eligible predicates earlier in the query.
Grouping by several expressions produces one row per distinct combination:
@@ -83,8 +81,8 @@ array_agg string_agg
The approximate functions are the ones worth reaching for on a large table.
`approx_distinct` answers "roughly how many distinct values" for a fraction of
what `count(DISTINCT ...)` costs, which matters most where it is most expensive
— a high-cardinality column.
what `count(DISTINCT ...)` costs, which can help with high-cardinality columns. Measure the accuracy and runtime
for your workload.
`DISTINCT` works inside an aggregate: `count(DISTINCT category)`. `FILTER`
narrows one aggregate without narrowing the others:
@@ -107,12 +105,16 @@ FROM documents TABLESAMPLE (1)
GROUP BY category
```
Scale the result by the inverse of the sample fraction, and treat it as an
estimate -- a 1% sample of a skewed column will miss rare categories entirely.
For a uniform percentage sample, counts and sums can be scaled by the inverse
of the sample fraction to estimate whole-table values. Do not apply that scaling
to averages, minima or maxima. Small samples can miss rare categories; consider
the sampling method and uncertainty when interpreting the result.
## What aggregation does not do {#not}
Aggregates run over rows the query already produced. They cannot combine with
[vector search](/search/vector-search) in one statement, because similarity
search has no SQL form. Run the search in an SDK and aggregate its result, or
materialize the search output first.
Aggregates run over rows the query already produced. Whether a statement can
include [vector search](/search/vector-search)
is described under [SQL vector search](/basics/sql#no-vector-sql). If your server
does not support that query form, run the
search in an SDK and aggregate its result, or materialize the search output
first.
docs/web/sql/dql/explain.mdx
+6 -5
View File
@@ -6,7 +6,7 @@ icon: map
keywords: ["explain", "explain analyze", "explain verbose", "query plan"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
```sql
EXPLAIN SELECT * FROM documents WHERE identifier < 1000
@@ -23,9 +23,10 @@ The result is a small relation with `plan_type` and `plan` columns, read
through the same client you use for regular queries -- see
[Connecting](/sql#connecting).
The operators are the same ones the SDK's
[`explain_plan` and `analyze_plan`](/search/explain) methods produce, so the
reasoning about index coverage and filter pushdown carries across unchanged:
SQL plans and the SDK's
[`explain_plan` and `analyze_plan`](/search/explain) output share scan and index
operators. Use the same approach to inspect index coverage and filter pushdown;
SQL can also add joins, aggregates and other operators:
<CardGroup cols={2}>
<Card title="What it shows" icon="map" href="/search/explain#inspect-a-query-plan">
@@ -41,7 +42,7 @@ reasoning about index coverage and filter pushdown carries across unchanged:
| Sign | Means |
| --- | --- |
| `ScalarIndexQuery` | the predicate used a scalar index |
| `LanceScan` with high `bytes_read` | no index, or too many columns selected |
| `LanceScan` with high `bytes_read` | inspect the selected columns, filter pushdown and index use |
| `FilterExec` above a search | the filter ran after search rather than pushing into it |
| `ANNSubIndex` / `ANNIvfPartition` | the vector index was used |
docs/web/sql/dql/fts.mdx
+1 -1
View File
@@ -6,7 +6,7 @@ icon: magnifying-glass
keywords: ["fts", "sql", "full text search", "fuzzy", "phrase", "boolean", "boost", "multi-match"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
Full-text search is a **table function**, `fts(table, query)`, rather than a
predicate -- it produces a relevance score alongside the matching rows, and a
docs/web/sql/dql/join.mdx
+2 -3
View File
@@ -6,10 +6,9 @@ icon: link
keywords: ["join", "inner", "left", "outer", "cross", "namespace"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
Joining is the other thing the query builders cannot do. A question spanning two
tables is a SQL question.
Use SQL joins to combine rows from two or more tables.
## The join types {#types}
docs/web/sql/dql/sampling.mdx
+13 -12
View File
@@ -6,12 +6,11 @@ icon: dice
keywords: ["tablesample", "sampling", "bernoulli", "system", "repeatable", "random"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
`TABLESAMPLE` draws a random subset of a table. On a lakehouse table that is
often the difference between a question you can afford to ask and one you
cannot -- an aggregate over 1% of forty billion rows answers in a fraction of
the time, and for most estimates that is enough.
Use `TABLESAMPLE` to explore a subset of a table or estimate aggregate
statistics. The sampling method determines how much data is read and whether
the table's physical layout influences the sample.
```sql
SELECT * FROM documents TABLESAMPLE (10) -- 10 percent
@@ -41,8 +40,8 @@ only as uniform as the data's layout. If rows were written in an order that
correlates with what you are measuring, a fragment sample inherits that
correlation.
Reach for `SYSTEM` when you want a cheap look at the data and `BERNOULLI` when
the number you compute has to be defensible. Omitting the method gives
Use `SYSTEM` for a quick look at the data and `BERNOULLI` when you need
independent row sampling. Omitting the method gives
`BERNOULLI`.
## Repeatability {#repeatable}
@@ -54,9 +53,10 @@ different rows. `REPEATABLE` fixes the draw:
SELECT * FROM documents TABLESAMPLE (10) REPEATABLE (42)
```
This is what makes a sample usable as a **fixed evaluation set**: the same seed
against the same data gives the same rows, so a metric computed today is
comparable with one computed last week.
Use the same seed and table snapshot to reproduce a sampling run. For a fixed
evaluation set, save the selected row IDs or materialize the sample and retain
its version. A seed alone does not preserve membership after source changes
or changes to the physical layout.
## Where the clause can go {#placement}
@@ -64,7 +64,7 @@ It attaches to a table in `FROM`, including inside a subquery, a CTE, or either
side of a join:
```sql
SELECT * FROM (SELECT * FROM documents) d TABLESAMPLE (10)
SELECT * FROM (SELECT * FROM documents TABLESAMPLE (10)) d
SELECT * FROM a TABLESAMPLE (5 ROWS) JOIN b ON a.id = b.id
@@ -75,7 +75,7 @@ Each side of a join samples independently. Sampling both sides of a join
shrinks the *result* far more than either fraction suggests, because a row only
survives if both its sides were drawn.
## What is refused {#refused}
## Unsupported sampling forms {#refused}
Sampling is semantics, not a hint. A clause that were read and then not applied
would be a wrong answer rather than a missing feature, so anything that cannot
@@ -86,6 +86,7 @@ be honoured exactly is an error:
| `TABLESAMPLE (101)` | a percentage outside `0..=100` is refused, not clamped |
| `TABLESAMPLE (1.5 ROWS)` | a row count must be whole |
| `TABLESAMPLE (BUCKET 1 OUT OF 4)` | bucket sampling is not supported |
| `(SELECT ...) d TABLESAMPLE (10)` | a subquery's result cannot be sampled; sample the table inside it |
| the `SAMPLE` spelling | use `TABLESAMPLE` |
| the `OFFSET` modifier | not supported |
docs/web/sql/dql/select.mdx
+7 -6
View File
@@ -6,7 +6,7 @@ icon: table-columns
keywords: ["select", "where", "order by", "limit", "cte", "union", "distinct"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
```sql
[WITH cte AS (...)]
@@ -35,9 +35,10 @@ await tbl.query().select(["id", "title", "identifier"]).toArrow();
```
</CodeGroup>
Projection pushes into the scan in all three -- an unselected column is never
read off storage. On a table of images or video that is the difference between a
fast query and a slow one, so **prefer naming columns over `SELECT *`**.
Name the columns you need instead of using `SELECT *`, particularly for
tables containing images, video or large vectors. Projection lets the engine
avoid loading unnecessary columns, although columns used by filters, joins or
other expressions may still need to be read.
Row metadata is available as [pseudo-columns](/sql/data-types#pseudo-columns)
(`_rowid`, `_rowaddr`, `_rowoffset`) but only when named, so they stay out of
@@ -63,8 +64,8 @@ await tbl.query().where("identifier > 0 AND identifier < 1000000").toArrow();
</CodeGroup>
The SDK `where` takes the same expression language, so a predicate moves between
them unchanged. A predicate on an indexed column becomes a `ScalarIndexQuery`
rather than a scan -- confirm with [`explain_plan`](/search/explain). See
them unchanged. A supported predicate can use a scalar index; confirm the chosen plan with
[`explain_plan`](/search/explain). See
[Filtering](/search/filtering) for the operators and their pushdown behaviour.
## Ordering and limiting {#order-limit}
docs/web/sql/dql/subqueries.mdx
+11 -10
View File
@@ -6,7 +6,7 @@ icon: code-branch
keywords: ["subquery", "exists", "in", "correlated", "derived table", "lateral"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A subquery is allowed anywhere an expression or a relation is.
@@ -19,11 +19,11 @@ SELECT * FROM (
WHERE n > 100
```
A derived table needs an alias. It can be [sampled](/sql/dql/sampling) like a base
table:
A derived table needs an alias. Its result cannot be
[sampled](/sql/dql/sampling); sample the table inside it instead:
```sql
SELECT * FROM (SELECT * FROM documents) d TABLESAMPLE (10)
SELECT * FROM (SELECT * FROM documents TABLESAMPLE (10)) d
```
## As a scalar {#scalar}
@@ -33,8 +33,8 @@ SELECT title, (SELECT count(*) FROM revisions r WHERE r.doc_id = d.id) AS revisi
FROM documents AS d
```
A scalar subquery must return one row and one column. Returning more is an
error rather than a silently-taken first row.
A scalar subquery returns one column and at most one row. No rows produces
`NULL`; more than one row is an error.
## IN and EXISTS {#in-exists}
@@ -46,8 +46,9 @@ SELECT title FROM documents AS d
WHERE EXISTS (SELECT 1 FROM revisions r WHERE r.doc_id = d.id)
```
`EXISTS` stops at the first matching row, so it is the cheaper test when you
only care whether anything matches. `IN` builds the full set.
Use `EXISTS` to test whether a matching row exists and `IN` to test membership
in a set of values. The optimizer may rewrite either into a join; inspect
`EXPLAIN` before choosing one for performance reasons.
`NOT IN` and `NOT EXISTS` differ on nulls: `NOT IN` against a set containing a
null yields no rows, because the comparison is unknown rather than false. Prefer
@@ -55,8 +56,8 @@ null yields no rows, because the comparison is unknown rather than false. Prefer
## Correlated subqueries {#correlated}
A subquery referencing a column from the enclosing query — `d.id` above — is
correlated, and is conceptually evaluated per outer row. The planner usually
A subquery referencing a column from the enclosing query, such as `d.id` above,
is correlated, and is conceptually evaluated per outer row. The planner usually
rewrites it into a join.
<Note>
docs/web/sql/dql/window.mdx
+4 -4
View File
@@ -6,7 +6,7 @@ icon: chart-simple
keywords: ["window", "over", "partition by", "row_number", "rank", "lag", "lead"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A window function computes across a set of rows related to the current one and
returns a value **per row**. That is the difference from
@@ -33,8 +33,8 @@ OVER (
```
`PARTITION BY` splits the rows into independent groups. `ORDER BY` orders them
inside each partition -- required by the ranking functions, since a rank without
an order is meaningless. The frame clause narrows the window further, which is
inside each partition. Supply an ordering for useful ranks, and a stable
tie-breaker when selecting one row from several equal values. The frame clause narrows the window further, which is
how running totals and moving averages are expressed:
```sql
@@ -74,7 +74,7 @@ lag lead first_value last_value nth_value
**Aggregates**
Any [aggregate function](/sql/dql/aggregation#functions) used with `OVER` becomes a
window function — that is how running totals and moving averages are written.
window function. This is how running totals and moving averages are written.
## Deduplicating with a window {#dedup}
docs/web/sql/functions.mdx
+1 -1
View File
@@ -6,7 +6,7 @@ icon: function
keywords: ["functions", "datafusion", "scalar", "aggregate", "window", "fts", "table function"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
LanceDB executes SQL with [Apache DataFusion](https://datafusion.apache.org),
currently **version 54**. The function set is DataFusion's default registration
docs/web/sql/index.mdx
+29 -73
View File
@@ -1,94 +1,45 @@
---
title: "SQL reference"
sidebarTitle: "Overview"
description: "The SQL surface: dialect, connecting a client, and a page per construct."
description: "SQL statements, expressions and data types supported by LanceDB Enterprise."
icon: table-list
keywords: ["sql", "datafusion", "flight sql", "reference", "dialect"]
---
<Badge color="red">Enterprise</Badge> SQL runs on a deployment. A local database
has no engine to plan a statement, so `execute_query` raises there -- see
[SQL](/basics/sql#sql-is-remote).
This is a reference rather than a guide: one page per construct, syntax first.
The guides elsewhere in these docs show the same operations from the SDKs.
LanceDB Enterprise supports SQL through an Arrow Flight SQL endpoint. This
reference covers statement syntax, data types and expressions. To set up a
client and run your first query, see [SQL](/basics/sql).
## The dialect {#dialect}
LanceDB executes SQL with [Apache DataFusion](https://datafusion.apache.org),
currently **version 54**. The grammar, built-in functions and type system are
DataFusion's, so anything its reference documents for that version works here
unchanged and is not reproduced.
What these pages add is the part that is LanceDB's: how a query reaches the
engine, how Lance columns appear to it, and the statements a lakehouse needs
that DataFusion has no concept of.
LanceDB uses [Apache DataFusion](https://datafusion.apache.org) for SQL planning
and execution. These pages describe LanceDB's supported statements and
extensions. DataFusion's reference is useful for expression syntax, but does
not establish that every DataFusion statement or feature is exposed by
LanceDB.
## Connecting {#connecting}
SQL is served over an
[Arrow Flight SQL](https://arrow.apache.org/docs/format/FlightSql.html)
endpoint, so any Flight SQL client reaches it and results arrive as Arrow
record batches -- nothing is converted to pandas or Python objects unless you
ask.
From Python, the connection runs SQL directly, without a separate client:
Use Python's `execute_query` or a Flight SQL client after configuring the
[Enterprise connection](/basics/sql#sql-is-remote). TypeScript applications
use a Flight SQL client for SQL queries.
```python
reader = db.execute_query("SELECT id, text FROM my_docs LIMIT 10")
table = reader.read_all()
for batch in reader:
print(batch)
```
`execute_query` hands back a reader as soon as the first batch is ready rather
than when the query finishes, so a large result set does not have to fit in
memory before you start reading it. See [Queries](/basics/query).
The TypeScript SDK has no SQL method; reach for a Flight SQL client instead --
and in Python, reach for one too when the caller is outside the SDK's own
process, such as a BI tool or a notebook in another language:
```bash
pip install flightsql-dbapi # Python DB API 2 interface
npm install --save @lancedb/flightsql-client
```
<CodeGroup>
```python Python icon="python"
from flightsql import FlightSQLClient
client = FlightSQLClient(
host="your-enterprise-endpoint",
port=10025,
insecure=True,
token="DATABASE_TOKEN",
metadata={"database": "your-project-slug"},
features={"metadata-reflection": "true"},
)
info = client.execute("SELECT * FROM flights WHERE origin = 'SFO'")
ticket = info.endpoints[0].ticket
result = client.do_get(ticket).read_all()
```
```typescript TypeScript icon="square-js"
import { Client } from "@lancedb/flightsql-client";
const client = await Client.connect({
host: "your-enterprise-endpoint:10025",
username: "lancedb",
password: "password",
});
const result = await client.query("SELECT * FROM flights WHERE origin = 'SFO'");
```
</CodeGroup>
SQL results arrive as Arrow record batches. Iterating over the reader avoids
collecting the entire result in application memory. See
[Queries](/basics/query) for result handling.
## Usage {#usage}
The endpoint executes one statement per request and is meant for the queries
and DDL/DML pages below. Reach for the SDKs instead for table-management
operations that have no SQL form, such as building a vector index or a merge
insert -- see [What SQL cannot do](#cannot).
operations that have no SQL form, such as a merge insert -- see
[What SQL cannot do](#cannot).
## Reference {#reference}
@@ -101,9 +52,13 @@ insert -- see [What SQL cannot do](#cannot).
| [View](/sql/ddl/view) | `CREATE VIEW`, `DROP VIEW` |
| [Materialized view](/sql/ddl/materialized-view) | `CREATE MATERIALIZED VIEW`, `SHOW MATERIALIZED VIEWS` |
| [Index](/sql/ddl/index) | `CREATE INDEX`, `DROP INDEX` |
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
| [Job](/sql/ddl/job) | `SHOW JOBS`, `SHOW ERRORS`, `PAUSE`/`RESUME`/`CANCEL JOB` |
{/*
Draft for the Functions release. Add this row to the table above, after Index:
| [Function](/sql/ddl/function) | `CREATE FUNCTION`, `SHOW FUNCTIONS`, `DROP FUNCTION` |
*/}
### DML {#dml}
| | |
@@ -111,7 +66,7 @@ insert -- see [What SQL cannot do](#cannot).
| [Insert](/sql/dml/insert) | `INSERT INTO`, including `INSERT ... SELECT` |
| [Update](/sql/dml/update) | `UPDATE ... SET` |
| [Delete](/sql/dml/delete) | `DELETE FROM` |
| [Copy](/sql/dml/copy) | `COPY INTO` / `COPY TO` |
| [Copy](/sql/dml/copy) | `COPY` -- refused by recent deployment versions |
| [Refresh](/sql/dml/refresh) | filling computed columns and materialized views |
### DQL {#dql}
@@ -135,15 +90,16 @@ insert -- see [What SQL cannot do](#cannot).
| [Operators](/sql/operators) | comparison, logical, arithmetic, literals, precedence |
| [Functions](/sql/functions) | where to find each kind, and table functions |
| [Scalar functions](/sql/scalar-functions) | the per-row catalogue, and what Lance accelerates |
| [Prepared statements](/sql/prepared-statements) | planning once, executing many times |
| [Prepared statements](/sql/prepared-statements) | preparing through a Flight SQL client |
## What SQL cannot do {#cannot}
## SQL limitations and alternatives {#cannot}
| | |
| --- | --- |
| Vector search | no SQL form -- a query-builder operation |
| Vector search | See [SQL vector search](/basics/sql#no-vector-sql) for supported query forms and server-version requirements. The [SDK query builder](/search/vector-search) also supports vector search. |
| Merge insert | no SQL form -- needs a source relation and a per-match policy |
| `information_schema` | not enabled, so `SHOW TABLES` and `SHOW COLUMNS` fail |
| `COPY` | refused by recent deployment versions -- see [Copy](/sql/dml/copy) |
| `PREPARE` / `EXECUTE` | preparation is a [protocol feature](/sql/prepared-statements), not a statement |
| Run on a local database | no engine to plan the statement |
docs/web/sql/operators.mdx
+11 -7
View File
@@ -6,10 +6,11 @@ icon: plus-minus
keywords: ["operators", "literals", "precedence", "null", "cast", "interval"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
The operator set is DataFusion's. This page covers what is worth knowing when
the operands are Lance columns.
LanceDB uses DataFusion's SQL operators. This page describes comparison,
logical and arithmetic operators, literal values, type conversions and
operator precedence.
## Comparison {#comparison}
@@ -44,8 +45,8 @@ IN`, which is the usual place this bites. See
<Warning>
Arithmetic over a vector column is **list** arithmetic, not scalar arithmetic —
a vector is a `FixedSizeList<Float32>`. When the operation you want is a
similarity comparison, use [vector search](/search/vector-search); there is no
SQL form for it.
similarity comparison, use [vector search](/search/vector-search); whether SQL
can express it [depends on the Enterprise server version](/basics/sql#no-vector-sql).
</Warning>
## Literals {#literals}
@@ -66,10 +67,13 @@ or a name with mixed case: `SELECT "order" FROM "MyTable"`.
## Casting {#casting}
```sql
SELECT CAST(identifier AS VARCHAR), identifier::VARCHAR FROM documents
SELECT CAST(identifier AS VARCHAR) AS with_cast, identifier::VARCHAR AS with_colons
FROM documents
```
Both forms work. `TRY_CAST` returns null instead of erroring on a value that
Both forms work. Give each its own alias when one projection uses both: they
produce the same expression name, and a projection refuses two columns with
one name. `TRY_CAST` returns null instead of erroring on a value that
will not convert, which is the one to use over a column you do not control.
## Precedence {#precedence}
docs/web/sql/prepared-statements.mdx
+29 -33
View File
@@ -1,17 +1,18 @@
---
title: "Prepared statements"
sidebarTitle: "Prepared Statements"
description: "Preparing a statement once and executing it many times, over the Flight SQL protocol."
description: "What a deployment does when a Flight SQL client prepares a statement or binds parameter values."
icon: bolt
keywords: ["prepared statement", "flight sql", "parameters", "bind"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
A prepared statement is planned once and executed many times, which saves
re-planning a statement whose shape does not change between runs.
A Flight SQL client can prepare a statement before it runs it. A deployment
accepts that, but it keeps no plan between runs and does not accept parameter
values.
## It is a protocol feature, not a statement {#protocol}
## Prepare through a Flight SQL client {#protocol}
<Warning>
There is no SQL `PREPARE` / `EXECUTE` here. Preparation is part of the
@@ -20,46 +21,41 @@ by sending a statement.
</Warning>
That distinction matters when porting SQL from another engine: a script
containing `PREPARE name AS SELECT ...` will not run, while the same intent
expressed through a Flight SQL client will.
containing `PREPARE name AS SELECT ...` will not run.
## The flow {#flow}
## Prepare and execute {#flow}
Every Flight SQL client exposes the same three steps, whatever it calls them:
A Flight SQL client takes these steps, whatever it calls them:
1. **Prepare** — send the SQL, get back a handle and the result schema. The
1. **Prepare:** send the SQL and get back a handle and the result schema. The
statement is parsed and planned at this point, so a syntax or name error
surfaces here rather than at execution.
2. **Bind and execute** — supply parameter values for the handle and read the
result. Repeat as often as you like.
3. **Close** — release the handle. Servers do not hold them forever.
Knowing that the plan is built at step 1 is the practical part: if a statement
is slow to prepare and fast to execute, the cost is planning, and reusing the
handle is exactly the right fix.
2. **Execute:** run the handle and read the result. The deployment plans the
statement again each time it runs, so reusing a handle does not save
planning time.
3. **Close:** release the handle.
## Parameters {#parameters}
Parameters are placeholders in the SQL text, bound to an Arrow record batch at
execution. Binding rather than interpolating is worth doing for the usual two
reasons — the value cannot change the statement's meaning, and the plan is
reused across values.
A deployment does not accept parameter values for a prepared statement. A
client that binds values to placeholders such as `$1` and `$2` gets a "not
implemented" error when it executes the statement. A statement with fixed,
trusted values written into it runs as usual:
```sql
SELECT id, title FROM documents WHERE category = $1 AND score > $2
SELECT id, title FROM documents WHERE category = 'news' AND score > 0.5
```
The exact placeholder spelling and the binding call belong to your client; see
[connecting a client](/sql#connecting).
Do not put untrusted input, such as a value from a user or a request, directly
into the statement text. Quoting it does not protect the statement the way
binding a parameter would.
## When it helps {#when}
## When to use prepared statements {#when}
| | |
| --- | --- |
| The same query with different values, many times | prepare once, bind per execution |
| A complex plan — several joins, window functions | planning dominates; reuse it |
| A one-off analytical query | no benefit; just execute it |
Prepare a statement when your client requires it, or to get a statement's
result schema and catch syntax and name errors without running it. Preparing
does not make a repeated statement faster.
For a statement you want to keep rather than re-prepare, a
[view](/management/view) stores the query server-side, and a
[materialized view](/management/materialized-view) stores its rows.
To give a reusable query a name, create a [view](/management/view). To store
its result for later queries, create a
[materialized view](/management/materialized-view).
docs/web/sql/scalar-functions.mdx
+20 -12
View File
@@ -6,7 +6,7 @@ icon: calculator
keywords: ["scalar functions", "string functions", "math functions", "regex", "array functions", "json"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
Functions that take a value and return a value, one row at a time. The set is
based on DataFusion's registration plus Lance and deployment-specific
@@ -118,8 +118,7 @@ arrow_cast arrow_try_cast arrow_typeof arrow_field arrow_metadata
with_metadata version
```
`arrow_typeof` is the fastest way to answer "what is this column, really" when a
result does not have the shape you expected:
Use `arrow_typeof` to inspect the Arrow data type of a column or expression:
```sql
SELECT arrow_typeof(vector) FROM documents LIMIT 1
@@ -156,17 +155,26 @@ it over `LIKE` for keyword tests.
| `json_get_int(col, key)` | `Int64` |
| `json_get_float(col, key)` | `Float64` |
| `json_get_bool(col, key)` | `Boolean` |
| `json_extract(col, path)` | `Utf8`, by path |
| `json_extract_with_type(col, path, type)` | the requested type |
| `json_extract(col, path)` | `Utf8`: the value as JSON text, by path |
| `json_extract_with_type(col, path)` | a struct of the value as JSON and a type tag, by path |
| `json_exists(col, path)` | `Boolean` |
| `json_array_contains(col, value)` | `Boolean` |
| `json_array_length(col)` | length |
`json_get` returns JSON, so accessors nest:
`json_get` returns JSON, so a typed accessor can read a field inside it:
```sql
SELECT * FROM events
WHERE json_get_string(json_get(json_get(data, 'user'), 'profile'), 'name') = 'Alice'
WHERE json_get_string(json_get(data, 'user'), 'name') = 'Alice'
```
That works one level down. A `json_get` nested inside another `json_get` is
refused, so reach a deeper value with `json_extract` and a JSONPath instead. It
returns JSON text, so a string compares with its quotes:
```sql
SELECT * FROM events
WHERE json_extract(data, '$.user.profile.name') = '"Alice"'
```
**`json_get`, its typed variants, and `json_extract` are index-accelerated** by a
@@ -181,10 +189,10 @@ feature rather than an unknown-function error, and the spatial index is gated
the same way.
</Warning>
## Reaching a function not listed here {#escape-hatch}
## Additional scalar functions {#escape-hatch}
Nothing wraps or restricts the set — a query reaches any registered function by
writing it. When a name is not recognised, check it against
SQL queries can call functions registered with the server, including functions
not listed on this page. Consult
[DataFusion's scalar function reference](https://datafusion.apache.org/user-guide/sql/scalar_functions.html)
for version 54 before assuming it is unsupported; functions are added in most
DataFusion releases, and a function documented on `main` may not exist here yet.
for version 54. Functions documented for a newer DataFusion release may not be
available in your LanceDB Enterprise version.