Files
Ruben Fiszel 248540ac4d feat: bounded-cascade selective execution for pipelines (UI + CLI) (#9695)
* feat: bounded-cascade selective execution for pipelines (UI + CLI)

Run a prefix of a pipeline cascade: from a schedule/manual root, fan
downstream but stop at chosen end node(s) — the path-between set over the
asset-graph lineage DAG. Exposed as a canvas 'Run downstream up to…' pick
mode and a 'wmill pipeline run <folder> --to' CLI command. No backend or
parser changes; reads the existing graph, tags, and triggers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: surface bounded-run on the run caret, trigger-node kebab, and Test button

Move 'Run downstream up to…' from the runnable kebab onto the play-button
caret popover (Edit mode, next to Run / Run + trigger N downstream); add it
to the trigger-node kebab so schedule/data_upload entrypoints expose it on
the View page; and to the ScriptEditor Test split caret for the open script.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: address CI review on bounded-cascade (cubic)

- Port CLI engine test from Deno to bun:test under cli/test/ (won't run under bun test otherwise).
- closure() now excludes the start node on a cycle back to it (descendants/ancestors contract); regression tests both engines.
- CLI 'pipeline run --to' rejects unresolved/ambiguous end tokens instead of silently running a different subset.
- Sort a copy in the runSelection order test so the launch-order assertions aren't invalidated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: address standing review nits on bounded-cascade

Resolves the four recurring P1/P2 findings from the codex/pi/claude
reviews:

- UI gate (P1): the canvas/trigger-node "Run downstream up to…"
  affordance was gated on the subscriber-only downstream map, so a valid
  start whose only downstream is a pure reader had a non-empty bounded
  set but no menu entry. Gate on the read-aware lineage downstream
  (buildLineageDownstreamMap), matching the bounded engine.
- waitJob (CLI): a completed job without explicit success:true now
  counts as a failure, mirroring the frontend waitJobTerminal — the
  cascade only advances on a confirmed success.
- Comment fix (CLI): the unbounded `run` path uses the read-aware
  lineage DAG (pure readers included); dropped the false "parity with
  the canvas cascade" (subscriber-only) claim.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: expose bounded-run caret for pure-reader-only starts (codex P1)

The canvas wiring from the prior commit passed `onStartBoundedRun` from
the read-aware lineage map, but the leaf components still hid the popover
that holds the "Run downstream up to…" action behind a subscriber-only
gate:

- RunnableNode rendered the Run-button caret only when
  `hasCascade = downstreamCount > 0` (subscriber-only). A valid start
  whose only downstream is a pure reader got `onStartBoundedRun` but no
  visible action. Now the caret opens when there's a cascade OR a
  bounded-run start (`hasCaret`), and the "Run + trigger N downstream"
  item is gated on `hasCascade` so it never reads "trigger 0".
- ScriptEditor's Test split button activated only when
  `downstreamSubscribers > 0`, falling through to a plain Test button
  (no caret) otherwise. Now it also activates when `onBoundedRun` is
  set, with the "Test + trigger N" item gated on the count.

For a manual root (no trigger-node kebab fallback) with a pure-reader
downstream this was the only UI entry point, so it was previously
unreachable. Verified in-browser: a manual-root script writing an asset
read-only downstream now exposes "Run downstream up to…" on the
ScriptEditor Test caret with the cascade item hidden.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: gate ScriptEditor bounded-run on read-aware downstream; fix CLI asset-end warning (codex P2)

- Details-pane (ScriptEditor) bounded-run entry was gated only on
  `validStartPaths`, broader than the canvas which also requires
  read-aware downstream (`hasLineageDownstream`). An isolated start could
  thus expose "Run downstream up to…" and enter pick mode with no
  selectable end. Now gated on `lineageDownstreamPaths` (script paths with
  a downstream in `buildLineageDownstreamMap`), matching the canvas.
- CLI dropped-end warning called `scriptPathOf(d)` unconditionally, which
  slices `script:`-length chars off an asset id too — `datatable:main/raw`
  printed as `le:main/raw`. Now prefix-checks like the JSON output.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: correct --from error to exclude only row-backed event triggers (codex P2)

The bounded-start validation message listed `kafka/webhook/…` as event
triggers that can't start a bounded run, but webhook/data_upload are
rowless and read as manual roots (valid starts). Only the row-backed
native kinds (kafka/mqtt/nats/postgres/sqs/gcp/email — EVENT_TRIGGER_KINDS)
are excluded; the message now names those.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: surface dropped ends in CLI JSON; disambiguate shared-trigger bounded start (codex P2)

- CLI `run --json` silenced the dropped-end warning, and the JSON payload
  echoed the originally-resolved `--to` list with no reachable/dropped
  split — a resolved-but-unreachable end looked like a clean plan that
  silently runs only the start. JSON now includes `reachableEnds` and
  `droppedEnds` (shared `idLabel` helper, asset-id safe).
- Trigger nodes dedupe per (kind, ref), so a schedule shared across
  scripts collapses to one node, but `recordSourceTrigger` kept only the
  first target path — the bounded-run action then rooted at an arbitrary
  script (or hid when only that first script lacked downstream). Now all
  target paths are tracked and the action is offered only when exactly one
  is a valid start with downstream; multi-eligible nodes suppress it
  rather than guess.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: don't run hidden drafts in View-mode bounded cascade (codex P1)

launchCascadeScript unconditionally preferred drafts.get(path) over the
deployed script. In View mode with drafts hidden (displayGraph is
deployed-only), a bounded run started from a trigger-node kebab would
execute preview jobs from hidden local draft content instead of the
deployed scripts the user is looking at.

Gate draft execution on `mode === 'edit' || includeDrafts` — the exact
condition under which displayGraph includes drafts — so execution always
matches the displayed graph. No-op for scripts without a draft; the
edit-mode "Run + trigger N downstream" cascade is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-25 12:08:05 +02:00
..

System Prompts

This directory contains the single source of truth for AI system prompts used by both the frontend copilot and CLI guidance.

Structure

system_prompts/
├── base/              # Core instruction templates (manually written)
│   ├── flow-base.md   # Shared OpenFlow structure guidance
│   └── flow-cli.md    # CLI/local-agent workflow guidance for write-flow skill
├── languages/         # Language-specific instructions (manually written)
└── auto-generated/    # Auto-generated files (DO NOT EDIT)
    ├── sdks/          # SDK documentation
    ├── cli/           # CLI command documentation
    ├── prompts.ts     # TypeScript exports
    └── index.ts       # Helper functions

Usage

Regenerating Prompts

When SDK methods or the OpenFlow schema change, run:

python system_prompts/generate.py

To also refresh the standalone skills in a Claude plugin checkout:

python system_prompts/generate.py --plugin-dir ~/windmill-claude-plugin

--plugin-dir accepts:

  • the windmill-claude-plugin repo root
  • a plugin root such as plugins/windmill
  • a direct skills/ directory

To regenerate the public docs repo (consumed by context7):

python system_prompts/generate.py --context7-dir ~/windmill-cli-docs

--context7-dir writes a fully-rendered snapshot (AGENTS.md, cli-commands.md, skills/<name>/SKILL.md, README.md, manifest.json with the Windmill version) with all template placeholders resolved — suitable for ingestion by docs aggregators. In CI this runs from .github/workflows/publish-cli-docs.yml on every release tag. The generator refuses to wipe the target directory unless it's empty or has a context7 marker (context7.json, manifest.json, or a windmill-cli-docs git remote), so a typo can't delete unrelated files.

This will:

  1. Parse TypeScript and Python SDK files to extract function signatures
  2. Parse the OpenFlow YAML schema
  3. Parse the CLI commands
  4. Assemble complete prompts from markdown files
  5. Generate TypeScript exports in auto-generated/
  6. Optionally refresh plugin-ready standalone SKILL.md files in the target directory

Scope

These system prompts contain ONLY:

  • How to write Windmill scripts (language syntax, conventions, SDK usage)
  • How to structure Windmill flows (OpenFlow schema, module types, data flow)
  • Resource type handling, S3 operations

They DO NOT contain:

  • Tool usage instructions (edit_code, set_flow_json, etc.)
  • IDE/editor specific commands
  • Testing tool invocations

Tool instructions are added separately by the frontend and CLI.

CLI-only workflow instructions live in base/flow-cli.md and are included in the generated write-flow skill for wmill init. They are intentionally excluded from the frontend flow chat prompt.

Integration

Frontend

Uses Vite path alias $system_prompts pointing to auto-generated/:

import { FLOW_GUIDANCE } from "$system_prompts/flow";
import { getLangContext } from "$system_prompts/languages";

CLI

Generates /cli/src/guidance/skills.gen.ts with embedded skill content for wmill init.

Editing Guidelines

  • Edit markdown files in base/, languages/
  • Never edit files in auto-generated/ directly
  • After editing, run generate.py to update exports