Ruben Fiszel 52ce805f61 fix(pipelines): dedup guard for keyed merge + deploy-time SCD2 validation (#9936) ...

Two correctness/validation improvements to managed materialization:

1. A keyed `merge` (`key=<col>`) is delete-by-key + insert-all and does NOT
   deduplicate its source, so two incoming rows sharing a key both landed
   under that key — silently breaking the one-row-per-key contract. Codegen
   now emits an in-transaction guard (same `error(...)` shape as the schema
   -drift guard) that fails the run when the SELECT returns more than one row
   for a non-NULL key, naming the key. Authors deduplicate in the SELECT or
   switch to `append`. NULL keys are exempt, matching the delete's `IN (...)`
   scope.

2. The two SCD2 misconfigurations that were only caught at run time — `history`
   without `key=`, and `history` + `// partitioned` — now fail fast at deploy
   via a shared `MaterializeSpec::validate`, called from `create_script_internal`.
   The DuckDB executor keeps the same check as a safety net for preview/test
   runs that never deploy (shared message, no drift).

Adds unit tests for the merge guard codegen and for `validate` (all four
cases), and updates docs/ducklake-materialization.md and docs/pipelines-vs-dbt.md.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-05 22:24:41 +02:00

..

src

fix(pipelines): dedup guard for keyed merge + deploy-time SCD2 validation (#9936)

2026-07-05 22:24:41 +02:00

Cargo.toml

feat: ducklake materialization for data pipelines (#9689)

2026-06-20 15:42:03 +02:00