mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-12 23:02:28 +00:00
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).
This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.
Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.
**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.
**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.
Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.
**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.
The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.
---------
Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
191 lines
5.9 KiB
Plaintext
191 lines
5.9 KiB
Plaintext
---
|
|
title: "Scalar functions"
|
|
sidebarTitle: "Scalar Functions"
|
|
description: "The per-row function set: math, string, regex, date/time, array, struct, and what Lance adds."
|
|
icon: calculator
|
|
keywords: ["scalar functions", "string functions", "math functions", "regex", "array functions", "json"]
|
|
---
|
|
|
|
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
|
|
|
|
Functions that take a value and return a value, one row at a time. The set is
|
|
based on DataFusion's registration plus Lance and deployment-specific
|
|
functions. The names below are selected examples, not an exhaustive catalog
|
|
for every deployment. Availability depends on the deployed version and build.
|
|
|
|
For per-group functions see [Aggregation](/sql/dql/aggregation); for per-window,
|
|
[Window functions](/sql/dql/window).
|
|
|
|
## Math {#math}
|
|
|
|
```text
|
|
abs ceil floor round trunc signum power sqrt cbrt exp
|
|
ln log log2 log10 factorial gcd lcm pi random nanvl
|
|
isnan iszero cot degrees radians to_degrees to_radians
|
|
sin cos tan asin acos atan atan2
|
|
sinh cosh tanh asinh acosh atanh
|
|
```
|
|
|
|
## String {#string}
|
|
|
|
```text
|
|
ascii bit_length btrim chr concat concat_ws contains ends_with
|
|
starts_with levenshtein lower upper ltrim rtrim octet_length
|
|
repeat replace split_part to_hex uuid
|
|
```
|
|
|
|
Unicode-aware variants:
|
|
|
|
```text
|
|
character_length find_in_set initcap left right lpad rpad
|
|
reverse strpos substr substr_index substring translate
|
|
```
|
|
|
|
<Note>
|
|
For keyword matching on a text column, [`contains_tokens`](#contains-tokens) is
|
|
index-accelerated where `contains` and `LIKE` are not.
|
|
</Note>
|
|
|
|
## Regular expressions {#regex}
|
|
|
|
```text
|
|
regexp_count regexp_instr regexp_like regexp_match regexp_replace
|
|
```
|
|
|
|
## Date and time {#datetime}
|
|
|
|
```text
|
|
current_date current_time date_bin date_part date_trunc
|
|
from_unixtime to_unixtime make_date make_time to_char
|
|
to_date to_time to_local_time
|
|
```
|
|
|
|
`date_bin` is the one worth knowing for analytics: it buckets timestamps into
|
|
fixed-width intervals, which is how you build a time series without a
|
|
self-join.
|
|
|
|
```sql
|
|
SELECT date_bin(INTERVAL '1 hour', created_at) AS hour, count(*)
|
|
FROM events GROUP BY hour ORDER BY hour
|
|
```
|
|
|
|
## Arrays and maps {#arrays}
|
|
|
|
```text
|
|
make_array array_has array_filter array_transform array_any_match
|
|
array_compact array_normalize array_slice arrays_zip cardinality
|
|
flatten range generate_series
|
|
```
|
|
|
|
```text
|
|
map map_keys map_values map_entries map_extract
|
|
```
|
|
|
|
Distance functions over arrays:
|
|
|
|
```text
|
|
cosine_distance distance exact_distance inner_product
|
|
```
|
|
|
|
<Warning>
|
|
Distance functions can evaluate arrays per row. Some deployment versions also
|
|
recognize `ORDER BY distance(vector, query_vector) LIMIT k` over a table scan
|
|
and plan it as vector search, with indexed or flat search as appropriate.
|
|
Where available, `exact_distance` evaluates distance without that rewrite.
|
|
Confirm the
|
|
functions supported by your deployment and inspect SQL
|
|
[`EXPLAIN`](/sql/dql/explain); the function name alone does not guarantee an
|
|
index-backed plan. See [SQL vector search](/basics/sql#no-vector-sql).
|
|
</Warning>
|
|
|
|
## Structs and fields {#structs}
|
|
|
|
```text
|
|
named_struct get_field coalesce nullif nvl nvl2 greatest least
|
|
union_extract union_tag
|
|
```
|
|
|
|
## Crypto and encoding {#crypto}
|
|
|
|
```text
|
|
digest md5 sha224 sha256 sha384 sha512 encode decode
|
|
```
|
|
|
|
## Type functions {#type}
|
|
|
|
```text
|
|
arrow_cast arrow_try_cast arrow_typeof arrow_field arrow_metadata
|
|
with_metadata version
|
|
```
|
|
|
|
`arrow_typeof` is the fastest way to answer "what is this column, really" when a
|
|
result does not have the shape you expected:
|
|
|
|
```sql
|
|
SELECT arrow_typeof(vector) FROM documents LIMIT 1
|
|
-- FixedSizeList(Field { name: "item", data_type: Float32, .. }, 1536)
|
|
```
|
|
|
|
`arrow_cast` casts to an Arrow type by name, reaching types SQL has no spelling
|
|
for. See [Data types](/sql/data-types).
|
|
|
|
## What Lance adds {#lance}
|
|
|
|
These are registered by Lance on top of DataFusion's set.
|
|
|
|
### `contains_tokens(column, tokens)` {#contains-tokens}
|
|
|
|
```sql
|
|
SELECT * FROM my_docs WHERE contains_tokens(text, 'fox jumps dog')
|
|
```
|
|
|
|
True when the column contains **all** the tokens, split on whitespace and
|
|
punctuation. Equivalent to an `fts()` match query with fuzziness off, `AND`
|
|
semantics and the simple tokenizer.
|
|
|
|
**Index-accelerated.** When the column has a full-text index that suits the
|
|
query, the planner uses it instead of scanning — which is the reason to prefer
|
|
it over `LIKE` for keyword tests.
|
|
|
|
### JSON accessors {#json}
|
|
|
|
| Function | Returns |
|
|
| --- | --- |
|
|
| `json_get(col, key)` | the value at `key`, still JSON |
|
|
| `json_get_string(col, key)` | `Utf8` |
|
|
| `json_get_int(col, key)` | `Int64` |
|
|
| `json_get_float(col, key)` | `Float64` |
|
|
| `json_get_bool(col, key)` | `Boolean` |
|
|
| `json_extract(col, path)` | `Utf8`, by path |
|
|
| `json_extract_with_type(col, path, type)` | the requested type |
|
|
| `json_exists(col, path)` | `Boolean` |
|
|
| `json_array_contains(col, value)` | `Boolean` |
|
|
| `json_array_length(col)` | length |
|
|
|
|
`json_get` returns JSON, so accessors nest:
|
|
|
|
```sql
|
|
SELECT * FROM events
|
|
WHERE json_get_string(json_get(json_get(data, 'user'), 'profile'), 'name') = 'Alice'
|
|
```
|
|
|
|
**`json_get`, its typed variants, and `json_extract` are index-accelerated** by a
|
|
JSON scalar index on the column. The rest evaluate per row.
|
|
|
|
### Geospatial {#geo}
|
|
|
|
<Warning>
|
|
Lance defines `st_*` functions on GeoArrow, but they sit behind a build feature
|
|
**LanceDB does not ship**. Calling one returns an error naming the missing
|
|
feature rather than an unknown-function error, and the spatial index is gated
|
|
the same way.
|
|
</Warning>
|
|
|
|
## Reaching a function not listed here {#escape-hatch}
|
|
|
|
Nothing wraps or restricts the set — a query reaches any registered function by
|
|
writing it. When a name is not recognised, check it against
|
|
[DataFusion's scalar function reference](https://datafusion.apache.org/user-guide/sql/scalar_functions.html)
|
|
for version 54 before assuming it is unsupported; functions are added in most
|
|
DataFusion releases, and a function documented on `main` may not exist here yet.
|