docs: state the concurrent-ingest interleaving honestly

`sole_dbt_producer`'s doc claimed the concurrent-deploy race only ever resolves
toward refusing. It does when the uncommitted producer is native; when it is the
dbt ingest, the check sees an empty producer set and accepts, and if that ingest
then commits and runs its warning query before the subscriber's trigger row
lands, neither side reports the dormant edge.

Not serialized: the two would have to share a per-relation lock, and the ingest
takes `script … FOR UPDATE` before its own advisory lock, so a deploy holding
relation locks first inverts that order into a cross-subsystem deadlock — a worse
failure than the cosmetic edge. Recorded beside the other orphaning the deploy
cannot catch, with the bound both share: the next deploy of that project warns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
2026-09-05 03:54:15 +02:00
co-authored by Claude Opus 5
parent f7f79d45cf
commit dff574d3d9
2 changed files with 24 additions and 6 deletions
+10 -2
View File
@@ -267,8 +267,16 @@ pub fn derive_pipeline_asset_trigger_refs(
/// them is free of the opposite error, because a script never wakes its own
/// subscription — the dispatcher skips that as a self-loop.
///
/// A producer another deploy is committing concurrently is still invisible, so
/// that race resolves toward refusing with a message the user can retry past.
/// A producer another deploy is committing concurrently is invisible either way,
/// and the outcome depends on which side it is. An uncommitted NATIVE producer
/// leaves a dbt-only set and refuses, with a message the user can retry past. An
/// uncommitted DBT one leaves an empty set and accepts — and if that ingest then
/// commits and runs [`dormant_dbt_subscriptions`] before this deploy's trigger
/// row lands, neither side reports the edge it left dormant. Serializing the two
/// is not worth it: they would have to share a per-relation lock, and the ingest
/// takes `script … FOR UPDATE` before its own advisory lock, so a deploy holding
/// relation locks first inverts that order into a deadlock across the two
/// subsystems. The next deploy of that project warns (docs/dbt-runtime.md).
pub async fn sole_dbt_producer<'e>(
executor: impl PgExecutor<'e>,
workspace_id: &str,
+14 -4
View File
@@ -734,10 +734,20 @@ What that leaves is a subscription accepted while it was live and later orphaned
A dbt project deployed afterwards that claims the relation names those edges in
its own log rather than leaving them silently dormant — the same "an edge that can
never fire is worse than saying so" the refusal is for, at the other point where
it is knowable. The remaining case, a native producer that drops its
`// materialize` and leaves dbt alone on the relation, is reported nowhere: the
deploy that causes it does not touch the subscriber, and the canvas is where it
shows.
it is knowable.
Two orphanings are reported nowhere, and both are accepted rather than overlooked.
A native producer that drops its `// materialize` and leaves dbt alone on the
relation: the deploy that causes it does not touch the subscriber. And the
interleaving where a dbt ingest commits between a subscriber's producer check and
its own commit — the check sees no producer and accepts, the ingest's warning
query sees no trigger and says nothing. Closing the second means a per-relation
lock shared by the deploy path and the ingest, and the ingest takes
`script … FOR UPDATE` before its own advisory lock, so a deploy holding relation
locks first inverts that order into a deadlock across two subsystems — a worse
failure than the cosmetic edge it would prevent. Both are bounded the same way:
the next deploy of that project warns, and the canvas is where they show
meanwhile.
A plain READ still renders the consumer beside the model, which is what makes
the lineage one graph — but it is written in the script's own code, not in a