* feat(pipelines): mid-DAG selective execution (dbt `model+`) for pipeline runs
Relax the root-only constraint on bounded-cascade starts so `--from` can name
any node in a pipeline — not just a schedule/manual root. A mid-DAG start runs
that node plus its transitive downstream and never re-runs upstream, giving
dbt's most common gesture (`dbt run --select model+`) a direct form:
wmill pipeline run f/orders --from fct_orders_daily
Previously this errored with "Starts must be schedule-triggered or manual
roots". The bounded-run engine already computed downstream/path-between sets
generically; only the eligibility gate was root-only.
- Shared engine (`boundedCascade.ts`, CLI + frontend mirror): add
`validFromStarts` — every autorun-able script (roots AND mid-DAG asset
subscribers / pure readers), excluding only event/input-only handlers
(kafka/mqtt/…/webhook/data_upload) that can't run with empty args.
- CLI: `--from` accepts any `validFromStarts` node; asset `--from` and
non-autorun handlers still rejected (the latter runnable via `--upload`). An
explicit mid-DAG start is protected from the barrier cut. Help text + regenerated
system_prompts describe the new surface.
- Frontend graph UI parity: any node with downstream now offers "Run + downstream…"
(was roots-only). With no end picked the bounded-run bar runs the full downstream
closure (`model+`); picking end(s) still bounds the path-between set.
- Unit tests for the new selection semantics in both engines.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): address CI review — scheduled-root --from regression + pick-mode barrier parity
Codex review findings on #9945:
- P1: explicit `--from` rejected a scheduled root that also carries a secondary
non-autorun trigger (e.g. `// on schedule` + `// on data_upload`), even though
it stays a valid IMPLICIT start. `validFromStarts` excluded anything in
`nonAutorunTriggerScripts`; now it unions in `validStarts` (which lets the
schedule identity win over the secondary trigger), so a scheduled root is
`--from`-eligible in both CLI and the graph UI. Regression tests added in both
engines.
- P2: bounded-pick mode built `eligible` (pickable end bounds) from raw
`descendants`, so an event handler — or a node only reachable through one —
could be clicked as an end yet be silently dropped from the barrier-cut run.
`eligible` is now the barrier-cut closure, so those nodes are dimmed and
non-pickable. The highlighted `bounded` ring now also reflects the actual
(barrier-cut) run set, including the no-ends "Run + downstream" case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend barrier set must exclude all valid roots, not just the picked start
Codex review follow-up: the frontend `boundReachable` barrier set only protected
the picked start (`id !== boundPickStart`), while the CLI protects every valid
root (`!starts.has(id)`). So a scheduled root that also carries an event trigger,
reached downstream from another start, was wrongly treated as a barrier — the UI
dimmed/skipped it and its downstream, diverging from the CLI run set.
Exclude `validStarts` from the barrier set too (a scheduled/manual root runs on
its own identity even with a secondary event trigger). Regression test asserts a
scheduled-event root and its downstream stay reachable from an upstream start,
and that the naive (start-only) barrier set would have dropped them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend must exclude webhook/data_upload as mid-DAG autorun starts
Codex review follow-up: the frontend `validFromStarts` only excluded
`EVENT_TRIGGER_KINDS`, so a mid-DAG `webhook`/`data_upload` subscriber was added
by the new eligibility loop — the UI would offer "Run + downstream" and launch it
with empty args (no uploaded S3Object / webhook body). The CLI mirror already
excludes these input-only kinds.
Add a frontend `NON_AUTORUN_TRIGGER_KINDS` (event kinds + webhook + data_upload),
mirroring the CLI, and use it in both `validFromStarts` (exclude such mid-DAG
handlers from starts) and `nonAutorunTriggerScripts` (cut them as barriers).
When the marker is visible (editor overlay / draft) these are now handled
exactly as the CLI does; the deployed-graph blind spot (no webhook/data_upload
rows) remains the documented pre-existing `validStarts` limitation.
Regression test: a `data_upload`/`webhook` mid-DAG subscriber is not an eligible
start and is barrier-cut (with its exclusive downstream) when running from an
upstream root.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
System Prompts
This directory contains the single source of truth for AI system prompts used by both the frontend copilot and CLI guidance.
Structure
system_prompts/
├── base/ # Core instruction templates (manually written)
│ ├── flow-base.md # Shared OpenFlow structure guidance
│ └── flow-cli.md # CLI/local-agent workflow guidance for write-flow skill
├── languages/ # Language-specific instructions (manually written)
└── auto-generated/ # Auto-generated files (DO NOT EDIT)
├── sdks/ # SDK documentation
├── cli/ # CLI command documentation
├── prompts.ts # TypeScript exports
└── index.ts # Helper functions
Usage
Regenerating Prompts
When SDK methods or the OpenFlow schema change, run:
python system_prompts/generate.py
To also refresh the standalone skills in a Claude plugin checkout:
python system_prompts/generate.py --plugin-dir ~/windmill-claude-plugin
--plugin-dir accepts:
- the
windmill-claude-pluginrepo root - a plugin root such as
plugins/windmill - a direct
skills/directory
To regenerate the public docs repo (consumed by context7):
python system_prompts/generate.py --context7-dir ~/windmill-cli-docs
--context7-dir writes a fully-rendered snapshot (AGENTS.md,
cli-commands.md, skills/<name>/SKILL.md, README.md, manifest.json
with the Windmill version) with all template placeholders resolved —
suitable for ingestion by docs aggregators. In CI this runs from
.github/workflows/publish-cli-docs.yml on every release tag. The
generator refuses to wipe the target directory unless it's empty or has
a context7 marker (context7.json, manifest.json, or a
windmill-cli-docs git remote), so a typo can't delete unrelated files.
This will:
- Parse TypeScript and Python SDK files to extract function signatures
- Parse the OpenFlow YAML schema
- Parse the CLI commands
- Assemble complete prompts from markdown files
- Generate TypeScript exports in
auto-generated/ - Optionally refresh plugin-ready standalone
SKILL.mdfiles in the target directory
Scope
These system prompts contain ONLY:
- How to write Windmill scripts (language syntax, conventions, SDK usage)
- How to structure Windmill flows (OpenFlow schema, module types, data flow)
- Resource type handling, S3 operations
They DO NOT contain:
- Tool usage instructions (edit_code, set_flow_json, etc.)
- IDE/editor specific commands
- Testing tool invocations
Tool instructions are added separately by the frontend and CLI.
CLI-only workflow instructions live in base/flow-cli.md and are included in the
generated write-flow skill for wmill init. They are intentionally excluded
from the frontend flow chat prompt.
Integration
Frontend
Uses Vite path alias $system_prompts pointing to auto-generated/:
import { FLOW_GUIDANCE } from "$system_prompts/flow";
import { getLangContext } from "$system_prompts/languages";
CLI
Generates /cli/src/guidance/skills.gen.ts with embedded skill content for wmill init.
Editing Guidelines
- Edit markdown files in
base/,languages/ - Never edit files in
auto-generated/directly - After editing, run
generate.pyto update exports