Files
lancedb/docs/web/sql/dql/aggregation.mdx
T
Jack YeandBrendan Clement 47b7babc08 docs: bring the open-source documentation into this repository (#4122)
Move the public website pages into `docs/web` beside the SDK, with
shared navigation and executable example sources in `docs/web-tests`.
The combined website is assembled by
[lancedb/docs#355](https://github.com/lancedb/docs/pull/355), with six
Enterprise replacements from
[lancedb/sophon#7681](https://github.com/lancedb/sophon/pull/7681).

This includes Function guidance, runnable Quickstarts, regenerated
snippets, production contact/demo fixes, the legacy SQL route, and the
LeRobot redirect. LeRobot remains attributed to Geneva; all 34 Geneva
pages remain in the docs repository until Function officially launches.

Materialized-view guidance follows the full rebuild refresh introduced
by #4373: local refresh writes the complete result, has no `full`
argument, supports `source_version`, and no longer requires stable row
IDs. Deployment SQL refresh options and stable-ID setup are qualified by
version/configuration. Function input guidance distinguishes ordinary
stored columns from computed inputs supported by newer deployments.

**Validation:** independently accepted local checks include 95 Python
tests (real Enterprise excluded), eight TypeScript tests, Rust
Quickstart, snippet parity, typos, deterministic assembly, Mint
validation and links/anchors/redirects. The combined baseline retains
all 181 production routes and 17 legacy redirects. Examples ran locally
against source-built Python SDK 0.40.0b13; existing GitHub workflows do
not run `docs/web-tests`. New main at `069b2f7d` is 0.41.0b0 and adds
computed Function inputs; that SDK was not exercised by the accepted
tests. Latest prose corrections receive focused site validation.

**Merge/rollout:** merge this and #7681 before #355's combined CI, which
reads producer `main`. This is migration infrastructure: section
overlays, source-owned skills and automatic versioned paired releases
remain follow-ups. The current manual publisher is transitional. Hosted
validation, a supported SDK/Enterprise release pair and a real
Enterprise walkthrough remain pending.

Production Mintlify watches `lancedb/docs:deploy-freeze`, directory
`docs`; preserve that source through review. The separate SDK GitHub
Pages reference workflow rebuilds on main merges.

**Updated-head CI (9a50f8db):**
[Typos](https://github.com/lancedb/lancedb/actions/runs/37654185443/job/112904990558),
[PR
title](https://github.com/lancedb/lancedb/actions/runs/37654179507/job/112904974595)
and
[Aikido](https://app.aikido.dev/featurebranch/scan/215386145?groupId=74229)
pass. Critical findings in the example lockfiles were patched: all Sharp
copies use 0.35.5, protobufjs 7.6.6, tar 7.5.22, quinn-proto 0.11.19,
zerovec 0.11.8 and zerovec-derive 0.11.6. The SDK's product dependencies
are unchanged. The revised lockfiles pass eight TypeScript examples,
HuggingFace decode/resize/crop/rescale/normalize plus AVIF roundtrip,
and Rust Quickstart. Aikido passing does not mean every advisory is
cleared; npm still reports 29 high and five moderate advisories in the
example dependency graph. No scanner bypass or suppression was added.

The Function prose correction passes combined/standalone Mint,
deterministic assembly, routes and assets. The revised 325-file site
checksum is
`175bec06864abb400dd292e667ba93448c0479b8359fd7bf6146afabf9040647`. This
PR remains open for independent review; further merges are on hold.

---------

Co-authored-by: Brendan Clement <brendan.clement@lancedb.com>
2026-10-07 11:15:00 -07:00

119 lines
3.3 KiB
Plaintext

---
title: "Aggregation"
sidebarTitle: "Aggregation"
description: "GROUP BY, HAVING, and the aggregate function set."
icon: layer-group
keywords: ["group by", "having", "count", "sum", "avg", "aggregate", "approximate"]
---
<Badge color="red">Enterprise</Badge> [SQL runs on a deployment.](/sql)
Aggregation is one of the two main reasons to reach for SQL rather than a query
builder -- the SDKs project, filter and search, but they do not group. The other
is [joins](/sql/dql/join).
## Grouping {#grouping}
```sql
SELECT category, count(*) AS n, avg(score) AS mean_score
FROM documents
GROUP BY category
HAVING count(*) > 10
ORDER BY n DESC
```
`WHERE` filters rows before grouping; `HAVING` filters groups after. Putting a
row-level predicate in `HAVING` still gives the right answer but does more work,
because rows that could have been discarded early are grouped first.
Grouping by several expressions produces one row per distinct combination:
```sql
SELECT category, source, count(*) FROM documents GROUP BY category, source
```
## Aggregate without grouping {#whole-table}
Omitting `GROUP BY` aggregates the whole table into one row -- the usual way to
ask how big something is:
```sql
SELECT count(*), min(created_at), max(created_at) FROM documents
```
`count(*)` counts rows; `count(col)` counts non-null values, which is how you
measure how populated a [computed column](/feature-engineering/add-computed-column)
is before deciding to refresh it again.
## The function set {#functions}
Registered in DataFusion 54. Names below are what is actually available.
**General**
```text
count sum min max avg median
bit_and bit_or bit_xor bool_and bool_or
first_value last_value nth_value grouping
```
**Statistical**
```text
stddev stddev_pop stddev_samp
var var_pop var_samp
corr covar_pop covar_samp
regr_slope regr_intercept regr_count regr_r2
regr_avgx regr_avgy regr_sxx regr_syy regr_sxy
```
**Approximate**
```text
approx_distinct approx_median
approx_percentile_cont approx_percentile_cont_with_weight
percentile_cont
```
**Collecting**
```text
array_agg string_agg
```
The approximate functions are the ones worth reaching for on a large table.
`approx_distinct` answers "roughly how many distinct values" for a fraction of
what `count(DISTINCT ...)` costs, which matters most where it is most expensive
— a high-cardinality column.
`DISTINCT` works inside an aggregate: `count(DISTINCT category)`. `FILTER`
narrows one aggregate without narrowing the others:
```sql
SELECT
count(*) AS total,
count(*) FILTER (WHERE score > 0.9) AS confident
FROM documents
```
## Aggregating a sample {#sampling}
For an estimate over a very large table, aggregate a
[sample](/sql/dql/sampling) rather than the whole thing:
```sql
SELECT category, count(*) * 100 AS estimated
FROM documents TABLESAMPLE (1)
GROUP BY category
```
Scale the result by the inverse of the sample fraction, and treat it as an
estimate -- a 1% sample of a skewed column will miss rare categories entirely.
## What aggregation does not do {#not}
Aggregates run over rows the query already produced. They cannot combine with
[vector search](/search/vector-search) in one statement, because similarity
search has no SQL form. Run the search in an SDK and aggregate its result, or
materialize the search output first.