Files
lancedb/docs/web/sql/dql/select.mdx
T
Jack YeandClaude Sonnet 5 de95ed3f6d docs: overhaul curation, feature engineering, security and deployment guides (#4748)
A broad documentation pass across the areas LanceDB Enterprise and
Embedded
diverge on most:

**Data Curation** -- GPU Function execution, Slurm scheduling (and what
Slurm is, for readers new to it), blob late materialization as its own
page, joins, aggregation/profiling, sampling, near-duplicate detection,
chunking, distributed shuffle, and IVF-partition clustering.

**Feature Engineering** -- computed columns and materialized views,
restructured to lead with the Function-based path and show the SQL form
side by side with Python throughout; a single Auto Refresh page instead
of
duplicating scheduling details on every refresh page; adaptive
execution;
checkpointing; GPU and Slurm placement for refresh workers; a Training
Cache page for the PyTorch data loader's automatic cache benefit.

**Security & Governance** -- an Audit Log reference (schema, access,
grants), Secrets promoted to an ordinary catalog object with their own
SQL
DDL and a vending/rotation guide, and privilege-table fixes verified
against the authorization source.

**Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry
moved
into a real Deployment area; Automated Cleanup folded into Intelligent
Auto-Maintenance (every other maintenance decision already lived there);
RTE Compaction and Advanced Conflict Resolution documented in depth.

**Landing page** -- the Enterprise feature comparison table reorganized
(several redundant or stale rows removed, several new areas added), and
the
"why Enterprise" bullets replaced with ones tied to a concrete page
(Performance, Scale, Fault tolerance, Multicloud deployment, Simplify
operations) instead of vague promises.

Plus a full pass addressing lancedb-gatekeeper's review: a real
test-safety
bug (a quickstart test could silently overwrite and delete a
deployment's
own `frames` table), a fabricated SQL statement with no backing
implementation, an actual dedup-guarantee bug in `take_row_ids`, and
dozens
of documentation-accuracy fixes verified against source rather than
taken
on faith -- each either fixed, or left with an explicit note on why it
needs a feature or deployment owner's sign-off instead of being guessed
at.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-03 16:17:24 -08:00

154 lines
4.2 KiB
Plaintext

---
title: "SELECT"
sidebarTitle: "SELECT"
description: "Projection, filtering, ordering, and composing queries out of other queries."
icon: table-columns
keywords: ["select", "where", "order by", "limit", "cte", "union", "distinct"]
availability:
embedded: unavailable
enterprise: available
---
<CodeGroup>
```sql SQL icon="database"
[WITH cte AS (...)]
SELECT [ALL | DISTINCT] select_expr [, ...]
FROM from_item [[JOIN ...] ON ...]
[WHERE predicate]
[GROUP BY expr [, ...]] [HAVING predicate]
[WINDOW name AS (...)]
[ORDER BY expr [ASC | DESC] [NULLS FIRST | LAST] [, ...]]
[LIMIT n] [OFFSET n]
```
</CodeGroup>
## Projection {#projection}
<CodeGroup>
```sql SQL icon="database"
SELECT id, title, identifier FROM documents
```
```python Python icon="python"
tbl.search().select(["id", "title", "identifier"]).to_arrow()
```
```typescript TypeScript icon="square-js"
await tbl.query().select(["id", "title", "identifier"]).toArrow();
```
</CodeGroup>
Name the columns you need instead of using `SELECT *`, particularly for
tables containing images, video or large vectors. Projection lets the engine
avoid loading unnecessary columns, although columns used by filters, joins or
other expressions may still need to be read.
Row metadata is available as [pseudo-columns](/sql/data-types#pseudo-columns)
(`_rowid`, `_rowaddr`, `_rowoffset`) but only when named, so they stay out of
`SELECT *`.
`DISTINCT` deduplicates the projected rows.
## Filtering {#filtering}
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM documents
WHERE identifier > 0 AND identifier < 1000000
```
```python Python icon="python"
tbl.search().where("identifier > 0 AND identifier < 1000000").to_arrow()
```
```typescript TypeScript icon="square-js"
await tbl.query().where("identifier > 0 AND identifier < 1000000").toArrow();
```
</CodeGroup>
The SDK `where` takes the same expression language, so a predicate moves between
them unchanged. A supported predicate can use a scalar index; confirm the chosen plan with
[`explain_plan`](/basics/sql#query-plans). See
[Filtering](/search/filtering) for the operators and their pushdown behaviour.
## Ordering and limiting {#order-limit}
<CodeGroup>
```sql SQL icon="database"
SELECT id, title FROM documents
ORDER BY created_at DESC NULLS LAST
LIMIT 100 OFFSET 200
```
</CodeGroup>
`ORDER BY` on a large table is a sort over the whole result, so pair it with a
`LIMIT` unless you genuinely need every row ordered.
## Set operations {#set-operations}
`UNION`, `UNION ALL`, `INTERSECT` and `EXCEPT` combine whole queries:
<CodeGroup>
```sql SQL icon="database"
SELECT id FROM documents WHERE category = 'film'
UNION ALL
SELECT id FROM archived_documents WHERE category = 'film'
```
</CodeGroup>
`UNION` deduplicates and `UNION ALL` does not, which makes `UNION ALL` the
cheaper one when you already know the inputs are disjoint.
## Common table expressions {#ctes}
<CodeGroup>
```sql SQL icon="database"
WITH recent AS (
SELECT * FROM documents WHERE created_at > now() - INTERVAL '7 days'
)
SELECT category, count(*) FROM recent GROUP BY category
```
</CodeGroup>
A CTE is scoped to the statement. One you want to keep is a
[view](/management/view); one whose rows you want stored is a
[materialized view](/management/materialized-view).
## Subqueries {#subqueries}
Subqueries are allowed anywhere an expression or a relation is (in `FROM`, in
`WHERE` with `IN` or `EXISTS`, and as a scalar in the select list):
<CodeGroup>
```sql SQL icon="database"
SELECT title
FROM documents
WHERE category_id IN (SELECT id FROM categories WHERE active)
```
</CodeGroup>
## Namespaced tables {#namespaces}
A table inside a [namespace](/management/namespace) is written with its
schema -- the `$`-joined namespace path, not a dotted one -- before the
table name; see [Schema](/basics/catalog#schema) for why:
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM `prod$events`.clicks
```
</CodeGroup>
## Reading the plan {#plans}
`EXPLAIN` and `EXPLAIN ANALYZE` work as DataFusion defines them, and produce the
same operators the SDK methods do:
<CodeGroup>
```sql SQL icon="database"
EXPLAIN ANALYZE SELECT * FROM documents WHERE identifier < 1000
```
</CodeGroup>
See [Explain](/basics/sql#inspect-a-query-plan) and [Explain Analyze](/basics/sql#analyze-a-query-plan).