Files
lancedb/docs/web/sql/dql/sampling.mdx
T
Jack YeandClaude Sonnet 5 de95ed3f6d docs: overhaul curation, feature engineering, security and deployment guides (#4748)
A broad documentation pass across the areas LanceDB Enterprise and
Embedded
diverge on most:

**Data Curation** -- GPU Function execution, Slurm scheduling (and what
Slurm is, for readers new to it), blob late materialization as its own
page, joins, aggregation/profiling, sampling, near-duplicate detection,
chunking, distributed shuffle, and IVF-partition clustering.

**Feature Engineering** -- computed columns and materialized views,
restructured to lead with the Function-based path and show the SQL form
side by side with Python throughout; a single Auto Refresh page instead
of
duplicating scheduling details on every refresh page; adaptive
execution;
checkpointing; GPU and Slurm placement for refresh workers; a Training
Cache page for the PyTorch data loader's automatic cache benefit.

**Security & Governance** -- an Audit Log reference (schema, access,
grants), Secrets promoted to an ordinary catalog object with their own
SQL
DDL and a vending/rotation guide, and privilege-table fixes verified
against the authorization source.

**Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry
moved
into a real Deployment area; Automated Cleanup folded into Intelligent
Auto-Maintenance (every other maintenance decision already lived there);
RTE Compaction and Advanced Conflict Resolution documented in depth.

**Landing page** -- the Enterprise feature comparison table reorganized
(several redundant or stale rows removed, several new areas added), and
the
"why Enterprise" bullets replaced with ones tied to a concrete page
(Performance, Scale, Fault tolerance, Multicloud deployment, Simplify
operations) instead of vague promises.

Plus a full pass addressing lancedb-gatekeeper's review: a real
test-safety
bug (a quickstart test could silently overwrite and delete a
deployment's
own `frames` table), a fabricated SQL statement with no backing
implementation, an actual dedup-guarantee bug in `take_row_ids`, and
dozens
of documentation-accuracy fixes verified against source rather than
taken
on faith -- each either fixed, or left with an explicit note on why it
needs a feature or deployment owner's sign-off instead of being guessed
at.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-11-03 16:17:24 -08:00

112 lines
3.8 KiB
Plaintext

---
title: "Random sampling"
sidebarTitle: "Sampling"
description: "TABLESAMPLE: draw a random subset of a table, by rows or by fragments."
icon: dice
keywords: ["tablesample", "sampling", "bernoulli", "system", "repeatable", "random"]
availability:
embedded: unavailable
enterprise: available
---
Use `TABLESAMPLE` to explore a subset of a table or estimate aggregate
statistics. The sampling method determines how much data is read and whether
the table's physical layout influences the sample.
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM documents TABLESAMPLE (10) -- 10 percent
SELECT * FROM documents TABLESAMPLE (500 ROWS) -- 500 rows
```
</CodeGroup>
A bare number is a **percentage**. Add `ROWS` for a count instead.
## Rows or fragments {#methods}
The standard calls these two methods of one clause. They are different
operations, and the difference matters on Lance:
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM documents TABLESAMPLE BERNOULLI (10)
SELECT * FROM documents TABLESAMPLE SYSTEM (10)
```
</CodeGroup>
| Method | Aliases | What it picks |
| --- | --- | --- |
| `BERNOULLI` | `ROW` | rows, each considered independently |
| `SYSTEM` | `BLOCK` | **Lance fragments**, and takes whatever rows they hold |
`BERNOULLI` gives a genuinely uniform sample and reads across the whole table to
get it. `SYSTEM` reads far less (it skips entire fragments), but the sample is
only as uniform as the data's layout. If rows were written in an order that
correlates with what you are measuring, a fragment sample inherits that
correlation.
Use `SYSTEM` for a quick look at the data and `BERNOULLI` when you need
independent row sampling. Omitting the method gives
`BERNOULLI`.
## Repeatability {#repeatable}
Without a seed, every execution resamples: running the same query twice gives
different rows. `REPEATABLE` fixes the draw:
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM documents TABLESAMPLE (10) REPEATABLE (42)
```
</CodeGroup>
Use the same seed and table snapshot to reproduce a sampling run. For a fixed
evaluation set, save the selected row IDs or materialize the sample and retain
its version. A seed alone does not preserve membership after source changes
or changes to the physical layout.
## Where the clause can go {#placement}
It attaches to a table in `FROM`, including inside a subquery, a CTE, or either
side of a join:
<CodeGroup>
```sql SQL icon="database"
SELECT * FROM (SELECT * FROM documents TABLESAMPLE (10)) d
SELECT * FROM a TABLESAMPLE (5 ROWS) JOIN b ON a.id = b.id
SELECT * FROM t a TABLESAMPLE (10) JOIN t b TABLESAMPLE (10) ON a.id = b.id
```
</CodeGroup>
Each side of a join samples independently. Sampling both sides of a join
shrinks the *result* far more than either fraction suggests, because a row only
survives if both its sides were drawn.
## Unsupported sampling forms {#refused}
Sampling is semantics, not a hint. A clause that were read and then not applied
would be a wrong answer rather than a missing feature, so anything that cannot
be honoured exactly is an error:
| | |
| --- | --- |
| `TABLESAMPLE (101)` | a percentage outside `0..=100` is refused, not clamped |
| `TABLESAMPLE (1.5 ROWS)` | a row count must be whole |
| `TABLESAMPLE (BUCKET 1 OUT OF 4)` | bucket sampling is not supported |
| `(SELECT ...) d TABLESAMPLE (10)` | a subquery's result cannot be sampled; sample the table inside it |
| the `SAMPLE` spelling | use `TABLESAMPLE` |
| the `OFFSET` modifier | not supported |
<Note>
`TABLESAMPLE` is also refused on a WAL-backed table, where the rows a sample
would draw from are not all in the fragments yet.
</Note>
## Sampling outside SQL {#sdk}
For a training loop rather than a query, the SDKs read a table in a shuffled
order without going through SQL at all. See [Training](/training/data-loading), where a
permutation drives the data loader.