mirror of
https://github.com/lancedb/lancedb.git
synced 2026-10-17 09:12:28 +00:00
Brendan Clement
486f6c66bb
This PR is part of the documentation revamp. Once the full set of PRs is merged, we’ll publish the combined site to staging for review before updating docs.lancedb.com. The introductory pages are inconsistent, and the old PR split asked reviewers to review some pages twice. This brings the final quickstart, connection guide, architecture and Concepts pages together. It clarifies OSS and Enterprise, simplifies naming rules and includes the navigation and diagram fixes from #4489. Function and Secret details on the Concepts pages are MDX comments until their release, and CONTRIBUTING.md explains how to write docs before a release. Includes #4532. Ingestion, search and integrations are in #4533; data workflows, SQL and operations are in #4534. Each MDX file has one PR owner. Local assembly, Mint validation and link checks pass. The Function pages stay published from `main` until #4534 hides them, so hold publication until the complete stack is integrated.
91 lines
4.0 KiB
Plaintext
91 lines
4.0 KiB
Plaintext
---
|
|
title: "Consistency"
|
|
sidebarTitle: "Consistency"
|
|
description: "Understand when readers see changes and choose the freshness your application needs."
|
|
icon: "water"
|
|
availability:
|
|
oss: available
|
|
enterprise: available
|
|
summary: >-
|
|
Configure read freshness in OSS and Enterprise. OSS checks for table updates;
|
|
Enterprise uses the connection setting to tighten its configured freshness bound.
|
|
---
|
|
|
|
import { PyReadConsistency } from '/snippets/read_consistency.mdx';
|
|
|
|
## Table snapshots and freshness {#consistency}
|
|
|
|
Each table read uses one committed version of the data. **Freshness** describes
|
|
whether that version includes recent writes. For example, after another
|
|
application appends records, an already-open table handle can still read the
|
|
version from before the append.
|
|
|
|
For example, a dashboard may need the latest updates, while an experiment may
|
|
need to keep reading a specific table version. Use read freshness settings for the dashboard and an explicit table version
|
|
for the experiment.
|
|
|
|
## Visibility of writes {#visibility}
|
|
|
|
In **OSS**, a table handle sees its own committed writes. By default, it does
|
|
not automatically check for changes made through other handles or processes.
|
|
Use a read consistency interval or `checkout_latest()` to pick up those changes.
|
|
|
|
In **Enterprise**, readers use the service's configured freshness policy.
|
|
A connection can request fresher data. When a read must include a completed
|
|
write, use a zero interval or call `checkout_latest()` after the write, with a
|
|
client and server that support these freshness controls.
|
|
|
|
## Configure read freshness {#configure-consistency-parameters}
|
|
|
|
Set `read_consistency_interval` when connecting. This controls when reads
|
|
pick up newer table versions; it does not change how writes commit.
|
|
|
|
| Setting | OSS | Enterprise |
|
|
| --- | --- | --- |
|
|
| **Unset** | Keep the open table version until explicitly refreshed or changed by that handle. | Use the service's configured freshness policy. |
|
|
| **Zero** | Check for the latest committed version on every read. | Request the latest committed version on every read. |
|
|
| **Non-zero interval** | Check for updates on a read once the interval since the last check has elapsed. | Request data no older than the interval allows; the service may return fresher data. |
|
|
|
|
`checkout_latest()` explicitly refreshes an OSS table handle. On Enterprise it
|
|
asks subsequent reads on that handle to reflect data at least as recent as the
|
|
refresh request. To read a fixed historical version instead, use
|
|
[table versioning](/management/table#versioning).
|
|
|
|
<Accordion title="Complete Python example: compare two readers">
|
|
|
|
Install the Python SDK with `pip install lancedb`, then save this as
|
|
`consistency.py` and run `python consistency.py`. It creates a temporary local
|
|
database and removes it when finished. No Enterprise connection is needed.
|
|
|
|
<CodeBlock filename="consistency.py" language="python" icon="python">
|
|
{PyReadConsistency}
|
|
</CodeBlock>
|
|
|
|
The default reader sees one row until refreshed. The reader with a zero
|
|
interval sees the new row on its next read.
|
|
|
|
</Accordion>
|
|
|
|
## Choose a freshness interval {#choosing}
|
|
|
|
- **Reads that must include recent writes:** use a zero interval. Checking more
|
|
frequently can increase read latency and storage requests.
|
|
- **Dashboards with an acceptable delay:** choose an interval within that delay
|
|
budget, then measure the effect on latency and cost.
|
|
- **Reproducible experiments:** read an explicit table version and retain it for
|
|
as long as the experiment needs it.
|
|
|
|
Choose the interval based on the delay your application can tolerate between
|
|
a write completing and a reader seeing it.
|
|
|
|
## Learn more {#next}
|
|
|
|
- [Connect](/basics/connect): configure your client.
|
|
- [Table versioning](/management/table#versioning): read and restore retained versions.
|
|
- [Guarantees](/basics/guarantees): understand commit and failure boundaries.
|
|
|
|
<span id="handle-bad-vectors" />
|
|
|
|
For invalid vector handling during ingestion, see
|
|
[Create a table](/ingestion/create-table#from-a-custom-schema).
|