* feat(pipelines): auto-derive cascade trigger edges from ducklake/s3 reads
Within a `// pipeline`, a read of a ducklake table or s3 object now
auto-wires its cascade trigger edge straight from the FROM clause, so
`// on <asset>` is only needed for edges inference can't see (dynamic SQL)
or to carry per-edge opts. Two opt-outs: `// mute <asset>` suppresses a
single derived edge (a lookup / SCD input read every run but not cascaded
on), and `// mute all` opts the script out of derivation entirely (back to
explicit-`// on`-only). Explicit `// on` still wins the dedup.
Scoped to ducklake + s3 reads; resource/datatable/volume stay explicit.
Read-write (RW) and write inputs are excluded so a self-referential
merge can't loop-trigger itself; ambiguous (None) access is skipped.
- parser: `mute` / `mute_all` in PipelineAnnotations (Rust + TS mirror)
- deploy: derive_pipeline_asset_trigger_refs → script_trigger rows
- frontend: resolveGraph mirrors derivation for the live edit-mode canvas
- tests: shared parity corpus + derive-helper units + resolveGraph overlays
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): mark auto-derived cascade edges with a persisted derived flag + "auto" badge
Persist script_trigger.derived (deploy: true for ducklake/s3-read derivation,
false for explicit // on) and return it from the asset-graph endpoint so the
canvas renders a Sparkles "auto" badge on auto-wired edges — the inference is
now visible on both the deployed graph and the live edit canvas, not just
implied. Dispatch (fetch_subscribers) ignores the flag, so a derived edge fires
identically to an explicit // on. Also copy derived in the workspace-clone
trigger copy, and backfill muteAssets/muteAll into two empty PipelineAnnotations
literals the base commit left stale (check:fast).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): derive cascade edge from effective (alt-fallback) asset access
derive_pipeline_asset_trigger_refs gated on the raw parser access_type, but the
persisted asset.usage_access_type and the frontend canvas both use
access_type.or(alt_access_type). An ambiguous parse with a manual read override
was persisted/drawn as a read yet derived no edge, so the auto edge silently
vanished on deploy. Gate on the effective access type for parity.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(pipelines): badge muted reads instead of auto-derived edges
Auto-derivation is the default now, so badging every derived cascade edge is
noise. Drop the "auto" badge and the persisted `script_trigger.derived` flag
(migration + insert param + graph field + clone copy) that only powered it, and
instead badge the exception: a ducklake/s3 asset a script reads but does NOT
cascade — `// mute <asset>` / `// mute all`. `computeMutedReadKeys` marks a
read-only ('r') supported read with no cascade trigger and no self-write; the
canvas renders a bell-off "muted" badge on that read edge.
Also fixes two review parity nits:
- TS `// on` parser now strips trailing `key=value` opts (e.g. `debounce=60s`)
like the Rust `split_trailing_kv_opts`, so the ref dedups against inference.
- A `// materialize` producer reading its own target is upgraded to `rw`
(deploy) / excluded via the materialize write refs (canvas), so it neither
self-cascades nor shows as a muted read.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): drop redundant // on for auto-derived reads; gate muted badge to pipeline scripts
- Templates no longer scaffold `// on <asset>` for a ducklake/s3 input the body
reads — the read auto-wires the cascade now that derivation is the default.
Kept for datatable/resource (not auto-derived) and native triggers. The
discoverability hint now mentions `// mute` (the newly relevant annotation).
- computeMutedReadKeys only badges reads by `// pipeline` scripts. A plain
script or flow reading a ducklake/s3 asset never had an auto trigger to
suppress, so it must render as ordinary lineage, not "muted" (Codex review).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(pipelines): only drop template // on when the body actually reads the input
The redundant-`// on` removal assumed the generated body reads the ducklake/s3
input, but postgres/bash/generic bodies (and `data_upload`, which reads the
picker file) ignore `input` — dropping `// on` there left the asset-created
script with no cascade at all. Gate the drop on READS_INPUT_LANGS
(bun/deno/python/duckdb) so non-reading templates keep the explicit trigger.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ruben FiszelandClaude Opus 4.8
3dcd3949a1