mirror of
https://github.com/lancedb/lancedb.git
synced 2026-09-11 07:42:26 +00:00
feat: recompute computed column rows whose inputs changed
refresh_column fills nulls, so once a row has a value nothing revisits it: an update to one of its inputs, or a definition change, leaves the computed value stale for good. This stamps the column's field metadata with the definition it was computed under and a per-fragment signature of the input storage it was read from (input data files and overlays; not the deletion file, since a delete changes no surviving value). A refresh recomputes every live row of a fragment whose stamp disagrees with the manifest, then records what it computed from in a second commit after the fill. A compacted fragment inherits freshness through the Rewrite lineage when every fragment it was built from was signed, or was appended since the stamp, never had an input moved, and left its rows of the product unfilled (a raw append may supply a value; the product's data is the evidence, and the null fill covers those rows); otherwise it recomputes. A column declared before the stamps existed keeps the null-fill contract on its first refresh, which enrolls it as it stood. The stamp is a metadata-only commit on the computed columns, so a materialized view's drift check treats it like the fill. The core lives in `table::freshness` so a remote refresh can share the contract.
This commit is contained in:
Generated
+1
@@ -5553,6 +5553,7 @@ dependencies = [
|
||||
"serde_json",
|
||||
"serde_with",
|
||||
"serial_test",
|
||||
"sha2 0.10.9",
|
||||
"snafu 0.8.9",
|
||||
"tempfile",
|
||||
"test-log",
|
||||
|
||||
Reference in New Issue
Block a user