feat(remote): send read freshness headers for remote table consistency (#3439)

Closes client side work of #3370 

### Summary
- Plumbs `read_consistency_interval` from `ConnectBuilder` through
`RestfulLanceDbClient` so remote reads attach an
`x-lancedb-min-timestamp` freshness header. None = no header (default),
zero = "now", positive = `now - interval`.
- Adds per-table `FreshnessState` on `RemoteTable`: write responses
(`update`, `delete`, `merge_insert`, `add_columns`, `alter_columns`,
`drop_columns`) track the committed version, and the next read sends
`x-lancedb-min-version` so the server's cache honors read-your-write.
- `checkout(v)` / `checkout_tag(t)` / `checkout_latest()` / `restore()`
reset the freshness state appropriately; the validating `/describe/` and
tag-resolve requests are sent without freshness headers so they don't
carry stale state.
- Updates Rust, Python, and Node docstrings and calls out that stronger
consistency raises per-read latency and cost.

### Testing
- Unit tests cover default behavior, interval=0, positive interval,
checkout_latest baseline, min_version-after-write, checkout clears
state, and the two no-stale-header invariants on `checkout(v)` and
`checkout_tag(t)`.
- Ran smoke tests against local remote table to verify functionality
This commit is contained in:
Brendan Clement
2026-05-26 13:38:07 -07:00
committed by GitHub
parent df2b6d3dd4
commit 15e75804c4
9 changed files with 621 additions and 101 deletions

View File

@@ -94,7 +94,6 @@ def connect(
host_override: str, optional
The override url for LanceDB Cloud.
read_consistency_interval: timedelta, default None
(For LanceDB OSS only)
The interval at which to check for updates to the table from other
processes. If None, then consistency is not checked. For performance
reasons, this is the default. For strong consistency, set this to
@@ -104,6 +103,10 @@ def connect(
the last check, then the table will be checked for updates. Note: this
consistency only applies to read operations. Write operations are
always consistent.
Stronger consistency is not free. The smaller the interval, the more
often each read pays the cost of checking for updates against object
storage, raising per-read latency and cost.
client_config: ClientConfig or dict, optional
Configuration options for the LanceDB Cloud HTTP client. If a dict, then
the keys are the attributes of the ClientConfig class. If None, then the
@@ -217,6 +220,7 @@ def connect(
request_thread_pool=request_thread_pool,
client_config=client_config,
storage_options=storage_options,
read_consistency_interval=read_consistency_interval,
**kwargs,
)
_check_s3_bucket_with_dots(str(uri), storage_options)
@@ -343,7 +347,6 @@ async def connect_async(
host_override: str, optional
The override url for LanceDB Cloud.
read_consistency_interval: timedelta, default None
(For LanceDB OSS only)
The interval at which to check for updates to the table from other
processes. If None, then consistency is not checked. For performance
reasons, this is the default. For strong consistency, set this to
@@ -353,6 +356,10 @@ async def connect_async(
the last check, then the table will be checked for updates. Note: this
consistency only applies to read operations. Write operations are
always consistent.
Stronger consistency is not free. The smaller the interval, the more
often each read pays the cost of checking for updates against object
storage, raising per-read latency and cost.
client_config: ClientConfig or dict, optional
Configuration options for the LanceDB Cloud HTTP client. If a dict, then
the keys are the attributes of the ClientConfig class. If None, then the

View File

@@ -50,6 +50,7 @@ class RemoteDBConnection(DBConnection):
connection_timeout: Optional[float] = None,
read_timeout: Optional[float] = None,
storage_options: Optional[Dict[str, str]] = None,
read_consistency_interval: Optional[timedelta] = None,
):
"""Connect to a remote LanceDB database."""
if isinstance(client_config, dict):
@@ -103,6 +104,7 @@ class RemoteDBConnection(DBConnection):
host_override=host_override,
client_config=client_config,
storage_options=storage_options,
read_consistency_interval=read_consistency_interval,
)
)