Files
windmill/docs
Ruben FiszelandClaude Opus 5 39fa2b10d6 feat: column-level lineage for dbt from the engine's parquet index
`manifest.json` carries no column-to-column edges, which is why decision 14
recorded column lineage as unavailable. The edges live in a different artifact:
`dbt compile --static-analysis strict --write-index` writes `target/index/`,
whose `dbt.column_lineage.parquet` holds them and whose
`dbt.node_columns.parquet` holds every column of every node, typed and ordered
rather than only the ones an author documented.

Strict analysis rejects SQL the default accepts, so this is a separate compile
with its own `--target-path`, opt-in per project via `column_lineage: true`, and
best-effort throughout: a project it cannot analyze keeps exactly the graph it
had, with the engine's own diagnostics in the job log.

Storage mirrors `dbt_edge`: `dbt_column_edge` keyed by (path, version, job) with
the same composite FK to `script` and the same sweeps. The typed column list
lands in `dbt_node.column_schema`, beside `columns` rather than merged into it,
so `columns` stays what the author declared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PRtsPQ3Ck69Fu9DNr7bMcJ
2026-09-04 17:06:54 +02:00
..