docs: state what cross-process refresh actually leaves behind

The caveat said two refreshes "would each append the same rows", which
reads as a warning about wasted work. Both appends are durable: lance
rebases the loser's append instead of rejecting it, so the check that
follows the write reports an abort after the rows have landed, and its
"unrecorded" wording describes the watermark rather than the data. Say
that, and say what closing it needs -- one transaction carrying both the
fragments and the watermark, which no current lance operation expresses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Wyatt Alt
2026-08-15 07:12:00 -07:00
co-authored by Claude Fable 5
parent 14e73f7e44
commit 7f892aaed6
+16 -3
View File
@@ -22,9 +22,22 @@
//! and the next refresh rebuilds rather than trusting any of it.
//!
//! Refreshes of one view are serialized within a process by a per-view lock.
//! Across processes nothing serializes them, and two incremental refreshes
//! planned at the same watermark would each append the same rows; run one
//! process's refreshes against a view at a time.
//! Across processes nothing serializes them: two incremental refreshes
//! planned at the same watermark each append the same rows, and both
//! appends become durable. The version check that follows a write cannot
//! prevent this -- lance rebases a conflicting append rather than rejecting
//! it, so the loser's rows land and only then does it report an abort. Its
//! error says the refresh is "unrecorded", which is true of the watermark
//! and false of the rows: the view serves duplicates until something
//! rebuilds it.
//!
//! Closing this needs the data and the watermark to commit as one
//! transaction, which no current lance operation expresses (`Append` and
//! `Update` carry fragments, `UpdateConfig` carries schema metadata, and
//! `Overwrite` -- which carries config -- rebases over concurrent metadata
//! commits). Until then, run one process's refreshes against a view at a
//! time. `differential::concurrent_refreshes_hold_each_row_once` is the
//! reproducer.
use std::collections::{HashMap, HashSet};
use std::sync::atomic::{AtomicU64, Ordering};