mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-16 08:42:29 +00:00
Brendan Clement
06aa526ae1
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. The curation, SQL and operations pages contain outdated claims and examples that do not reliably work as written. This corrects those pages against the reviewed SDK and local server evidence, including prepared statements, background jobs, cleanup and monitoring. Integration guides are grouped with their examples in #4533. Function and Secret-object documentation stays unpublished until its release. The two Function pages are listed in `.mintignore` and left out of the navigation, and the sections about them on other pages are MDX comments. Stacked on #4533. Each MDX file has one PR owner. Rebase onto main after #4533 is squash-merged. Local assembly, Mint validation and link checks pass, and the assembled site shows no Function or Secret-object content. A local release rehearsal that uncomments the drafts and restores the navigation also passes. Hold publication until this PR and lancedb/sophon#8688 are on `main`.
101 lines
3.7 KiB
Plaintext
101 lines
3.7 KiB
Plaintext
---
|
|
title: "Functions"
|
|
sidebarTitle: "Functions"
|
|
description: "Where to find each kind of function, and the dialect version they come from."
|
|
icon: function
|
|
keywords: ["functions", "datafusion", "scalar", "aggregate", "window", "fts", "table function"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> [SQL runs in LanceDB Enterprise.](/sql)
|
|
|
|
LanceDB executes SQL with [Apache DataFusion](https://datafusion.apache.org),
|
|
currently **version 54**. The function set is DataFusion's default registration
|
|
plus what Lance adds.
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| [Scalar functions](/sql/scalar-functions) | per row — math, string, regex, date/time, array, struct, JSON |
|
|
| [Aggregate functions](/sql/dql/aggregation#functions) | per group — `count`, `sum`, statistical, approximate |
|
|
| [Window functions](/sql/dql/window#functions) | per window — ranking, offset, running aggregates |
|
|
| [Table functions](#table-functions) | produce a relation rather than a value |
|
|
|
|
<Note>
|
|
Function availability is version-specific. DataFusion adds functions in most
|
|
releases, so one documented on `main` may not exist in 54. When a name is not
|
|
recognised, check it against the reference for that version before assuming a
|
|
bug.
|
|
</Note>
|
|
|
|
## Table functions {#table-functions}
|
|
|
|
A table function appears in `FROM` and produces a relation, which is what lets
|
|
it return columns a scalar function has nowhere to put.
|
|
|
|
### `fts(table, query)` {#fts}
|
|
|
|
Full-text search is a table function rather than a predicate, because it
|
|
produces relevance alongside the matching rows. The following query assumes
|
|
`my_docs` has `id` and `text` columns and a full-text index on `text`; see the
|
|
[setup example](/sql/dql/fts#setup):
|
|
|
|
```sql
|
|
SELECT id, text, _score
|
|
FROM fts('my_docs', '{"match":{"column":"text","terms":"puppy"}}')
|
|
ORDER BY _score DESC
|
|
LIMIT 5
|
|
```
|
|
|
|
The second argument is a string literal containing a JSON query object. A bare
|
|
query string such as `'puppy'` is rejected. Match, fuzzy, phrase, boolean, boost
|
|
and multi-match queries use this JSON encoding.
|
|
|
|
Its schema contains the dataset's columns plus `_score`, so every stored
|
|
column is available in the result. Projections push into the scan.
|
|
Being a relation is also what makes it compose — join it, filter it, aggregate
|
|
over it. This join additionally requires `my_docs.category_id` and a
|
|
`categories` table with `id` and `name` columns:
|
|
|
|
```sql
|
|
SELECT c.name, count(*) AS hits
|
|
FROM fts('my_docs', '{"match":{"column":"text","terms":"puppy"}}') AS m
|
|
JOIN categories AS c ON c.id = m.category_id
|
|
GROUP BY c.name
|
|
ORDER BY hits DESC
|
|
```
|
|
|
|
For a keyword test that needs no score, the
|
|
[`contains_tokens`](/sql/scalar-functions#contains-tokens) predicate is
|
|
index-accelerated and usable directly in `WHERE`.
|
|
|
|
See [Full-Text Search](/sql/dql/fts) for the fuzzy, phrase, boolean, boost and
|
|
multi-match query forms.
|
|
|
|
### `_score` {#score}
|
|
|
|
A pseudo-column produced by `fts()`, carrying relevance for the row. It behaves
|
|
like any other column — order by it, filter on it, project it — and exists only
|
|
in a relation that came from `fts()`. See
|
|
[Data types](/sql/data-types#pseudo-columns).
|
|
|
|
### `generate_series` {#generate-series}
|
|
|
|
```sql
|
|
SELECT * FROM generate_series(1, 10)
|
|
```
|
|
|
|
Rows from a range, useful for building a dense axis to left-join sparse data
|
|
onto.
|
|
|
|
## What Lance accelerates {#accelerated}
|
|
|
|
Two function families use an index rather than scanning, which is the difference
|
|
between a fast predicate and a slow one:
|
|
|
|
| | Index used |
|
|
| --- | --- |
|
|
| [`contains_tokens`](/sql/scalar-functions#contains-tokens) | full-text index |
|
|
| [`json_get`, typed variants, `json_extract`](/sql/scalar-functions#json) | JSON scalar index |
|
|
|
|
Confirm with [`EXPLAIN`](/sql/dql/explain): an accelerated predicate shows an
|
|
index node rather than a `LanceScan` followed by a filter.
|