mirror of
https://github.com/lancedb/lancedb.git
synced 2026-11-13 22:26:53 +00:00
Jack YeandClaude Sonnet 5
de95ed3f6d
A broad documentation pass across the areas LanceDB Enterprise and Embedded diverge on most: **Data Curation** -- GPU Function execution, Slurm scheduling (and what Slurm is, for readers new to it), blob late materialization as its own page, joins, aggregation/profiling, sampling, near-duplicate detection, chunking, distributed shuffle, and IVF-partition clustering. **Feature Engineering** -- computed columns and materialized views, restructured to lead with the Function-based path and show the SQL form side by side with Python throughout; a single Auto Refresh page instead of duplicating scheduling details on every refresh page; adaptive execution; checkpointing; GPU and Slurm placement for refresh workers; a Training Cache page for the PyTorch data loader's automatic cache benefit. **Security & Governance** -- an Audit Log reference (schema, access, grants), Secrets promoted to an ordinary catalog object with their own SQL DDL and a vending/rotation guide, and privilege-table fixes verified against the authorization source. **Deployment & Maintenance** -- Horizontal Scaling and OpenTelemetry moved into a real Deployment area; Automated Cleanup folded into Intelligent Auto-Maintenance (every other maintenance decision already lived there); RTE Compaction and Advanced Conflict Resolution documented in depth. **Landing page** -- the Enterprise feature comparison table reorganized (several redundant or stale rows removed, several new areas added), and the "why Enterprise" bullets replaced with ones tied to a concrete page (Performance, Scale, Fault tolerance, Multicloud deployment, Simplify operations) instead of vague promises. Plus a full pass addressing lancedb-gatekeeper's review: a real test-safety bug (a quickstart test could silently overwrite and delete a deployment's own `frames` table), a fabricated SQL statement with no backing implementation, an actual dedup-guarantee bug in `take_row_ids`, and dozens of documentation-accuracy fixes verified against source rather than taken on faith -- each either fixed, or left with an explicit note on why it needs a feature or deployment owner's sign-off instead of being guessed at. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_018PXMCbRH6i5w47TsjeEUMT --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
72 lines
3.2 KiB
Plaintext
72 lines
3.2 KiB
Plaintext
---
|
|
title: "Limits"
|
|
sidebarTitle: "Limits"
|
|
description: "Find fixed constraints, configurable bounds and what happens when you reach them."
|
|
icon: ruler
|
|
keywords: ["limits", "maximum", "quota", "constraints", "bounds"]
|
|
---
|
|
|
|
<span id="scope" />
|
|
|
|
This page lists limits for job events, SQL error reports, sampling and
|
|
full-text indexing. Each table identifies the affected API and what happens
|
|
when a request reaches its limit. SQL and server job limits apply to Enterprise.
|
|
|
|
<span id="names" />
|
|
|
|
[Naming Rules](/basics/naming) is the reference for name lengths, characters and
|
|
namespace depth. Client and server versions can differ in which rules they
|
|
enforce.
|
|
|
|
<span id="jobs" />
|
|
<span id="sql" />
|
|
|
|
## Fixed API limits {#fixed-limits}
|
|
|
|
| Operation | Scope | Limit and behavior |
|
|
| --- | --- | --- |
|
|
| Job event retrieval | Enterprise | At most 10,000 rows per response. Results can be truncated without a truncation indicator. |
|
|
| `SHOW ERRORS ... LIMIT` | Enterprise SQL | Accepts 1 through 100,000. Values outside the range are rejected. |
|
|
| `SHOW ERRORS` without `FOR JOB` | Enterprise SQL | Inspects up to the 32 most recent failed jobs. Use `FOR JOB` to inspect a specific job. |
|
|
| Percentage sampling | Enterprise SQL | `TABLESAMPLE` accepts percentages from 0 through 100. Values outside the range are rejected. |
|
|
| Row-count sampling | Enterprise SQL | `TABLESAMPLE (... ROWS)` requires a whole number. |
|
|
|
|
See [SQL job commands](/sql/ddl/job#errors) and
|
|
[Sampling](/sql/dql/sampling) for syntax and restrictions on supported table types.
|
|
|
|
<span id="schema" />
|
|
|
|
Data-type and index constraints belong to the operation you choose. Check
|
|
[Data types](/sql/data-types) and [Vector indexes](/indexing/vector-index)
|
|
before choosing a schema or index. For materialized views, confirm your
|
|
source table meets your deployment's requirements with your administrator
|
|
before creating one over it.
|
|
|
|
## Configurable limits {#not-quotas}
|
|
|
|
| Setting | What it controls |
|
|
| --- | --- |
|
|
| Job event `limit` | Requests more or fewer event rows. The default is 1,000; the 10,000-row maximum still applies. |
|
|
| FTS `max_token_length` | The maximum indexed token length. The default is 40; longer tokens are excluded from the index. |
|
|
| Enterprise capacity and quotas | Storage, request rate and concurrency are governed by your deployment's resources and configuration. Confirm these with your administrator or LanceDB support. |
|
|
|
|
## Handling rejected or incomplete results {#behavior}
|
|
|
|
- **Rejected requests:** correct the input before retrying, such as a sampling
|
|
percentage outside the accepted range.
|
|
- **Truncated responses:** do not assume that a returned event list is the full
|
|
history. Increasing `limit` cannot exceed the maximum; use filters to narrow
|
|
the events you inspect.
|
|
- **Excluded data:** an FTS token-length setting can affect which words are
|
|
searchable. Choose it for your text and indexing requirements.
|
|
|
|
<span id="availability" />
|
|
|
|
## Learn more {#next}
|
|
|
|
- [Naming Rules](/basics/naming): choose portable names and paths.
|
|
- [Job Framework](/basics/jobs): track work and inspect events.
|
|
- [Full-text indexes](/indexing/fts-index): configure tokenization.
|
|
- [SQL](/basics/sql) and [SDKs](/basics/sdk): check interface support.
|
|
- [Enterprise support](/deployment/support): discuss deployment capacity.
|