mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-22 00:01:34 +00:00
sql-s3-streaming-issue
51 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 hugocasa
|
98e6cca75d |
feat(cli): add --tag override to script and flow run/preview (#10079)
* feat: custom tags on CLI runs, show previews in default runs view Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * revert: don't include previews in default runs view Deferring the runs-view UX change; keeping only the CLI --tag work. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): forward --tag for codebase/bundle script previews The bundled-preview branch posts a multipart payload to /jobs/run/preview_bundle; --tag was only wired into the non-bundled runScriptPreview call, so codebase previews silently used the default tag. Include tag in the preview payload (backend reads preview.tag). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszel
|
735f2b20c4 |
docs(cli): clarify workspace fork naming and parent-workspace context (Fixes WIN-2148) (#10012)
* docs(cli): clarify workspace fork naming and parent-workspace context Expand `wmill workspace fork`'s help and interactive prompts so the two positional arguments are self-explanatory: - Command description now explains that the fork is created from the currently active (parent) workspace, that `workspace_name` is a friendly display name that may contain spaces (quote it), and that `workspace_id` is a bare slug auto-prefixed with `wm-fork-` which also determines the git branch name. - Interactive name/id prompts reworded to match. Regenerated system_prompts CLI guidance to reflect the new description. Fixes WIN-2148 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): default fork name to "<parent>'s fork", make it optional The fork's display name is no longer effectively required — it now defaults to "<parent workspace name>'s fork" (fetched via get_workspace_name, falling back to the local profile name / id) and stays fully overridable via the positional argument or interactive prompt. To produce this default, `setClient` and the parent-name lookup are moved ahead of the name/id resolution. The id default is decoupled from the possessive display name: when auto-naming, the id/branch slug is derived from "<parent>-fork" (e.g. wm-fork-acme-fork) rather than the awkward "<parent>-s-fork". Branch-rename forks keep their branch-derived id. Regenerated system_prompts CLI guidance. Fixes WIN-2148 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): fix workspace fork help — parent is branch-resolved, not active profile Addresses the codex/pi review: the fork help said the parent is the "currently active" workspace and told users to `wmill workspace switch`, but createWorkspaceFork resolves the parent from the current git branch's wmill.yaml mapping (tryResolveBranchWorkspace) and ignores the active profile. Reword to describe the actual branch-based resolution. Regenerated system_prompts CLI guidance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): note branch-derived fork id default in rename workflows Addresses the codex review: the `[workspace_id]` help and the interactive prompt said the default id is derived from the name, but rename workflows (non-base branch / --from-branch) keep the branch-derived default (`branchDefaultId ?? branchToForkId(idBasis)`) to keep the id/branch aligned with the branch being converted. Document that special case rather than changing the intentional behavior. Regenerated system_prompts CLI guidance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): cap auto fork name at 50 chars for long parent names Addresses the codex review: the "<parent>'s fork" default appended "'s fork" to a parent name that can itself be up to 50 chars (varchar(50)), so a parent name over 43 chars produced a default exceeding the limit and tripped the effectiveName.length > 50 guard — failing `wmill workspace fork --yes` (or accepting the interactive default) for a valid parent. Truncate the parent portion so the generated default stays within 50. Verified end-to-end: a 48-char parent name now yields a 49-char default ("... Team's fork") and the fork is created successfully. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Diego Imbert
|
e47aedac0a |
feat: add SQL migrations for data tables (#9693)
* feat: add datatable_migrations table * feat: add route to run datatable migrations * feat: sync datatable migrations as .up.sql/.down.sql files * feat: add datatable migrate up/down commands and post-push run prompt * feat: add datatable migrate new command to scaffold migrations * feat: add datatable migrations management UI * feat: prompt to create migration on DDL in datatable SQL editors * feat: support running a single specific datatable migration * feat: view migration content, run single migration, fix stacked modal * feat: per-row revert button with out-of-order warning * fix: avoid migrations list flicker on refresh after an action * feat: generate initial datatable migration via pg_dump * fix: surface datatable migration API error details in toasts * fix: revert created migration if create-and-run fails to run * fix: include postgres error detail in migration run/rollback failures * feat: sync datatable migrations as files via the workspace export * refactor: move datatable migrations to migrations/datatable/ path * fix: drop redundant datatable_migration label in sync output * fix: exclude datatable migration sql files from script metadata generation * feat: run datatable migrations as user-permissioned labeled jobs * feat: reject invalid datatable migrations on sync push * feat: datatable migrate up/down default to all datatables, --datatable to target one * fix: surface postgres error detail when datatable migrations fail to run * chore: regenerate CLI docs for datatable migrate commands * feat: default new datatable migration to a BEGIN/END transaction template * fix: validate datatable migration name and datatable at the API boundary * fix: ensure detected DDL ends with semicolon when wrapped in transaction * fix: re-prompt instead of stripping DDL when new-migration modal is cancelled * feat: refresh datatable schema after running a migration from the SQL REPL * feat: record db manager DDL on data tables as migrations * feat: make datatable migrations opt-in per data table * fix: make migration view editor read-only so its code can scroll * fix: don't re-prompt DDL guard when creating a migration without running * feat: generate down migrations for db manager DDL (postgres) * fix: correct down migration for db manager alters (no double-wrap, serial) * feat: explain migrations purpose with a tooltip in the migrations modal * compare paeg * feat: add datatable_migration kind to workspace diff pipeline * chore: point ee-repo-ref at datatable_migration git-sync companion * fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests * feat: deploy and run datatable migrations on workspace merge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Refactor + handle datatable setting delete/rename * refactor: move datatable migration rename/delete cascade into module Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(db-manager): add Migrations button to top bar, make Refresh icon-only Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * BEGIN/END placeholder in down migration * feat: autofocus migration name input and flag it red when empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * border nits * refresh db manager schema on migrations * BEGIN/END scaffold in CLI * feat(cli): push local datatable migrations before running on migrate up Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: flag invalid migration name with red border, not just empty Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop random slug from auto-generated migration names Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: offer revert-and-delete when deleting an installed migration Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: record fork merge as a migration when target datatable opts in * nit * clone migrations on fork * windmill-utils-internal * fix(datatable-migrations): serialize run/rollback with a per-db advisory lock Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(db-manager): fail closed when migrations-status check errors on DDL apply Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: fix generate_initial migration ordering comment to match code * chore(datatable-migrations): remove unused update_datatable_migrations endpoint Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: run DDL migration guard on the script editor Test button Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * split * ee-repo-ref * chore(frontend): sync package-lock with package.json (@emnapi deps) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(datatable-migrations): never resolve instance credentials into migration job args datatable_database_arg eagerly resolved instance data-table credentials (including the shared instance-wide Postgres password) and passed them as the migration job's plaintext `database` arg, landing in v2_job.args. Since the run route has no admin gate, a non-admin could run a migration and read args.database to recover the password, granting cross-workspace psql access to all instance data-table DBs. Pass a `datatable://<name>` reference for both resource-backed and instance data tables instead; the pg executor already resolves it to real credentials server-side at run time, so nothing sensitive is ever stored in the job args. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * fix: handle dollar-quoting and comments when splitting SQL statements * feat: deploy datatable migrations on merge with explicit opt-in error * fix(frontend): sync package-lock with npm 11 peer-dep resolution npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly 1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree needs both versions; the committed lock only had 1.10.0. Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit npm publish * fix: fail closed on migrations-status error in fork schema merge * nit CI emnapi/core version * prevent initial_datatable_migration if migrations already exist * fix(datatable-migrations): validate persisted data table names as path segments edit_datatable_config only validated rename segments, not the actual settings.datatables keys, so a data table could be saved directly under a name like '..' or one containing '/'. Since new tables default to migrations_enabled = true, generate_initial_datatable_migration would then insert a migration row and the sync export would build migrations/datatable/<name>/... paths from that name, producing malformed or directory-escaping export paths. Validate every persisted data table name in edit_datatable_config (alongside the existing rename checks) and add validate_datatable_path_segment to generate_initial_datatable_migration for defense in depth. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope datatable _wm_migrations by data table and cascade renames/deletes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(system_prompts): resolve nested local command groups in CLI docs generator The CLI docs generator anchored on the first `new Command()` in a file and never resolved locally-defined command groups passed as `.command("name", localCmd)`. For datatable this flattened the nested `migrate` group: it emitted `datatable new/up/down` plus a bare `datatable migrate`, and mislabeled the datatable command with the migrate group's description. jobs was broken the same way (its description was pull's, and pull/push rendered empty). Anchor block extraction on the `export default`ed command, recurse into locally-defined `const x = new Command()` groups mounted as subcommands, and render nested sub-subcommands. Regenerated docs now show `datatable migrate new/up/down` and `jobs pull/push` with their real options. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: drop unreleased _wm_migrations legacy-upgrade handling Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: return datatable migration SQL from getItemValue for the diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * nit * nit * fix: handle datatable migration renames on push and dedupe timestamps * fix: reject rewriting an already-applied datatable migration on upsert Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved lockfile entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): datatable migrate up/down default to main datatable, not all Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fail closed when applied status unreadable on datatable migration rewrite Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: surface full error detail in Database Manager DDL/query errors * "See migration" button in the toast * feat: add Enter shortcut to Create-a-migration in the DDL guard * fix(frontend): warn before running a newly-created datatable migration out of order The row-level Run action warns when earlier migrations are still pending, but the create-and-run paths ran a just-created migration with `only` directly, applying it ahead of older pending migrations without that confirmation. Reuse the same "Run migration out of order" confirmation across all create-and-run paths via a shared helper (datatableMigrationUtils): - NewDataTableMigrationModal "Create and run" (and the DDL guard path) - DatatableSchemaDiff fork→parent merge - dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a silent cancel Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep renamed datatable migrations visible in compare view * fix: record per-migration deployment on datatable migrations disable * fix(cli): run deployed datatable migrations after workspace merge The merge command upserted datatable_migration definitions into the target workspace and reported the item as successfully deployed, but never ran the migrations. For forked datatables backed by separate databases, this left the target schema unchanged until someone manually ran `wmill datatable migrate up`, while the CLI reported a successful merge. Collect the datatable migrations deployed (not deleted) into the target and, after the deploy loop, offer to run them via the existing offerToRunNewMigrations helper — the same post-deploy run prompt the push/sync path uses (interactive only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export parseDatatableMigrationDeployPath so the merge path can parse the deployed items. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): serialize datatable migration edits/deletes with the run lock A migration run snapshots a migration's code_up from datatable_migrations and only records its version in the data table's _wm_migrations after the job succeeds. upsert_datatable_migration checked _wm_migrations before allowing an edit but took no lock, so a concurrent edit could read "not applied yet", rewrite code_up/code_down, and then the in-flight run would record the version for the old SQL — leaving _wm_migrations pointing at SQL that was never applied (migrate up then skips it; rollback runs a down that doesn't match). Serialize definition rewrites and deletes with the same per-database advisory lock the run/rollback paths use: - Factor the connect+advisory-lock into lock_datatable_migration_runs and the applied-versions read into read_applied_versions_on_client. - run_datatable_migrations now snapshots the definitions AFTER taking the lock, so code_up can't change between snapshot and version-record. - upsert (when changing an existing def) and delete take the lock across the applied-check and the write; delete now rejects deleting an already-applied migration (would orphan its _wm_migrations record), symmetric with upsert. Both fail closed if the data table database is unreachable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): stack the out-of-order migration confirm above the DB editor preview Creating a table on a migrations-enabled data table opened the DB table editor's "Confirm running the following" preview modal, whose confirm triggers applyDdl, which then asks for out-of-order confirmation. Both are ConfirmationModals with a hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before the editor), so it rendered behind the still-open preview modal. Add an optional zIndexClass prop to ConfirmationModal (default z-[9999], backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it stacks on top. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private. Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4 New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
2d3a773441 |
feat(pipelines): mid-DAG selective execution (dbt model+) for pipeline runs (#9945)
* feat(pipelines): mid-DAG selective execution (dbt `model+`) for pipeline runs
Relax the root-only constraint on bounded-cascade starts so `--from` can name
any node in a pipeline — not just a schedule/manual root. A mid-DAG start runs
that node plus its transitive downstream and never re-runs upstream, giving
dbt's most common gesture (`dbt run --select model+`) a direct form:
wmill pipeline run f/orders --from fct_orders_daily
Previously this errored with "Starts must be schedule-triggered or manual
roots". The bounded-run engine already computed downstream/path-between sets
generically; only the eligibility gate was root-only.
- Shared engine (`boundedCascade.ts`, CLI + frontend mirror): add
`validFromStarts` — every autorun-able script (roots AND mid-DAG asset
subscribers / pure readers), excluding only event/input-only handlers
(kafka/mqtt/…/webhook/data_upload) that can't run with empty args.
- CLI: `--from` accepts any `validFromStarts` node; asset `--from` and
non-autorun handlers still rejected (the latter runnable via `--upload`). An
explicit mid-DAG start is protected from the barrier cut. Help text + regenerated
system_prompts describe the new surface.
- Frontend graph UI parity: any node with downstream now offers "Run + downstream…"
(was roots-only). With no end picked the bounded-run bar runs the full downstream
closure (`model+`); picking end(s) still bounds the path-between set.
- Unit tests for the new selection semantics in both engines.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): address CI review — scheduled-root --from regression + pick-mode barrier parity
Codex review findings on #9945:
- P1: explicit `--from` rejected a scheduled root that also carries a secondary
non-autorun trigger (e.g. `// on schedule` + `// on data_upload`), even though
it stays a valid IMPLICIT start. `validFromStarts` excluded anything in
`nonAutorunTriggerScripts`; now it unions in `validStarts` (which lets the
schedule identity win over the secondary trigger), so a scheduled root is
`--from`-eligible in both CLI and the graph UI. Regression tests added in both
engines.
- P2: bounded-pick mode built `eligible` (pickable end bounds) from raw
`descendants`, so an event handler — or a node only reachable through one —
could be clicked as an end yet be silently dropped from the barrier-cut run.
`eligible` is now the barrier-cut closure, so those nodes are dimmed and
non-pickable. The highlighted `bounded` ring now also reflects the actual
(barrier-cut) run set, including the no-ends "Run + downstream" case.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend barrier set must exclude all valid roots, not just the picked start
Codex review follow-up: the frontend `boundReachable` barrier set only protected
the picked start (`id !== boundPickStart`), while the CLI protects every valid
root (`!starts.has(id)`). So a scheduled root that also carries an event trigger,
reached downstream from another start, was wrongly treated as a barrier — the UI
dimmed/skipped it and its downstream, diverging from the CLI run set.
Exclude `validStarts` from the barrier set too (a scheduled/manual root runs on
its own identity even with a secondary event trigger). Regression test asserts a
scheduled-event root and its downstream stay reachable from an upstream start,
and that the naive (start-only) barrier set would have dropped them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(pipelines): frontend must exclude webhook/data_upload as mid-DAG autorun starts
Codex review follow-up: the frontend `validFromStarts` only excluded
`EVENT_TRIGGER_KINDS`, so a mid-DAG `webhook`/`data_upload` subscriber was added
by the new eligibility loop — the UI would offer "Run + downstream" and launch it
with empty args (no uploaded S3Object / webhook body). The CLI mirror already
excludes these input-only kinds.
Add a frontend `NON_AUTORUN_TRIGGER_KINDS` (event kinds + webhook + data_upload),
mirroring the CLI, and use it in both `validFromStarts` (exclude such mid-DAG
handlers from starts) and `nonAutorunTriggerScripts` (cut them as barriers).
When the marker is visible (editor overlay / draft) these are now handled
exactly as the CLI does; the deployed-graph blind spot (no webhook/data_upload
rows) remains the documented pre-existing `validStarts` limitation.
Regression test: a `data_upload`/`webhook` mid-DAG subscriber is not an eligible
start and is barrier-cut (with its exclusive downstream) when running from an
upstream root.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
d65f58c388 |
fix: pipeline dogfooding fixes — SCD2 data-test scope, --partition, s3object upload binding (#9875)
* fix: scope SCD2 built-in data tests to current rows Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: add --partition to pipeline run and fix duckdb s3object upload binding Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: note filesystem storage type is dev-only in storage settings Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use ISO week for weekly partition default in pipeline run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
b883adbc00 |
fix(duckdb): auto-declare partition arg for // partitioned scripts (#9878)
* fix(duckdb): auto-declare the partition arg for // partitioned scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): pipeline run --arg to pass plain run args to cascade scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
hugocasa
|
a73b14d902 |
fix(cli): correct misleading delete-fork command description (#9870)
* fix(cli): correct misleading delete-fork command description The `wmill workspace delete-fork` description claimed it deletes "a forked workspace and git branch", but the implementation only deletes the Windmill workspace via the backend API and removes the local workspace profile. No git operations are performed, so the remote branch is left untouched. Drop the "and git branch" clause and regenerate the derived guidance/system-prompt files. Fixes WIN-2120 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(cli): permanently delete temp workspaces in folder test cleanup The isolated-workspace test helper archived each temp workspace on teardown. After #9865 added a CE cap of 1 archived workspace, the second archive-cleanup is refused, so temp workspaces leak into the active set and hit the 2-workspace CE cap — failing every subsequent create/fork across the shared test backend. Permanently delete the workspace instead (DELETE /api/workspaces/delete), which frees the slot without occupying the archived quota. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
74f579e6d9 |
feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) (#9840)
* feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) Add the local edit→preview→run loop for data pipelines (folders of `// pipeline` scripts), the analog of `wmill dev` / `wmill app dev`, usable from a code editor or an agentic loop — without deploying. No backend changes: full body inference comes from the same wasm the frontend uses (windmill-parser-wasm-asset), which returns assets + pipeline annotations in one call; local runs reuse runScriptPreview with _wmill_skip_asset_dispatch. - localGraph.ts: wasm-backed working-tree → asset-graph builder (the enabler) - pipeline show/run --local; new pipeline docs (PIPELINE.md/AGENTS.md) subcommand - pipeline dev watcher + /pipeline_dev page (PipelineDevView) rendering the same PipelineGraphEditor from the pushed local graph, run via preview - cascadeRun.ts: reusable run primitives extracted from the route page - regenerated CLI agent docs See docs/pipeline-local-dev.md for the full design, test steps, and handoff TODOs. The live `pipeline dev` browser preview is implemented but not yet stack-verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): improve local dev preview (run, activity, responsive) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): dev-preview args, multi-root run, ws auto-reconnect Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): connect managed-materialize producer in local dev graph The CLI pinned windmill-parser-wasm-asset ^1.728.1, which predates managed-materialize support (added in 1.733.1); the frontend already pins 1.740.0. The CLI's wasm therefore never emitted `// materialize`, so the producer had no output edge and showed disconnected from its `// on` consumers. Bump the CLI to 1.740.0 (matching the frontend) and translate the parsed materialize target into the producer's write edge + materialize_target, mirroring frontend resolveGraph.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): harden local-dev CLI (bare-.sql crash, defaultTs, docs clobber) Review fixes, complementary to the dev-preview/materialize/multi-root work already on the branch (none overlap those commits): - localGraph: a bare `.sql` (no dialect) made inferContentTypeFromFilePath throw and abort the whole graph build — and wedge `pipeline dev` at startup. Skip the unclassifiable file instead. Also map `bunnative` → parse_assets_ts and add ruby/rlang/nu/powershell to the `#`-comment fallback. - show/run/docs/dev: thread the resolved `wmill.yaml` defaultTs into the graph builder so `.ts` infers under the workspace's runtime (bun vs deno) instead of always bun — `opts.defaultTs` was always undefined (no such CLI flag). - dev: wrap the startup graph build so a half-written file can't abort the watcher. - docs: don't clobber a user-authored AGENTS.md/CLAUDE.md — only (over)write the pointer when absent or already a generated `@PIPELINE.md` pointer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): bind dev WS to loopback + local-graph regression tests - pipeline dev WS broadcast the folder's full script source (scripts[].content + temp_script_refs) unauthenticated on 0.0.0.0:3201 — bind 127.0.0.1 so it's not LAN-reachable (webview localhost + SSH/devbox port-forward still work). - Add regression tests for the just-landed local-graph fixes: bare .sql is skipped (was a build/dev-startup crash), defaultTs threads into .ts runtime inference (bun vs deno), and #-comment languages (ruby) use the # annotation fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --frontend flag for pipeline dev page origin wmill pipeline dev opens <remote>/pipeline_dev, but that route only exists in this build's frontend, so it 404s against a remote whose deployed frontend predates it. --frontend <origin> points the page at a locally-run frontend (REMOTE=<remote> npm run dev) while the API/token still target the remote — enabling the live preview against a real backend before the PR is deployed. No behavior change when omitted. Regenerated CLI agent docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): WS session token + details-pane live-reload refresh Addresses CI review (Codex/Pi/Claude): - dev WS: a browser tab could open ws://localhost:<port>/ws and receive the folder's full source (browsers don't enforce same-origin on WS, loopback bind alone doesn't help). Gate the upgrade on an unguessable per-session token carried in the dev-page URL (verifyClient → 401 without it). Verified: no-token/bad-token connections get 401 with no bundle. - details pane: scriptRes keyed on [workspace, selection, draftScript] didn't re-run on a pipeline dev live-reload (same selection), so the open pane showed stale source. Thread a localScriptsVersion (the pushed bundle) into the key. Verified: editing a selected node's file updates the pane source without reselect. - docs/pipeline-local-dev.md: refresh the stale 'not yet exercised' status + done TODOs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): emit volume: annotation assets in local dev graph Addresses CI review (Codex P1 / Pi P1): the wasm body parser doesn't surface `// volume: <name>` annotations — the frontend (infer.ts:parseVolumeAnnotations) and backend (asset_inference.rs) parse them separately and merge as rw volume assets. localGraph didn't, so a `# volume: cache` producer had no write edge and showed disconnected from its `// on volume://cache` consumer (and pipeline run --local wouldn't schedule downstream). Mirror the leading-comment-block scan (SQL excluded, matching both reference parsers) and merge into inferScriptAssets. Regression test added; verified producer -> volume://cache -> consumer connects. Also (Codex P2): docs/pipeline-local-dev.md manual browser URL omitted the new ws_token param — without it the WS upgrade is rejected and the page sits disconnected. Doc now says to copy the URL the CLI prints (carries wm_token + ws_token) and recommends --frontend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): runAll excludes event roots + review polish Addresses CI review (Codex P1, Claude P2/P3): - pipeline run runAll: derive the whole-pipeline selection from validStarts + descendants instead of all runnables, so an unqualified 'pipeline run <folder>' no longer fires event-trigger roots (kafka/mqtt/…) with empty args/side effects. Verified: a kafka root is excluded from the plan. - cascadeRun.ts runBoundedCascade: use buildLineageDownstreamMap (read-aware) so a pure-reader runs after its producer, and return cyclic — parity with the route page's bounded run (the file is meant to be THE shared correct primitive). - PipelineGraphEditor: storedRightPaneSize starts at 0 so the orientation-aware default (55% stacked / 40% side-by-side) actually applies on first open. - localGraph fallbackParse (go/bash): scan only the leading comment header (no body-comment phantom triggers) and strip key=value options from the asset URI; regression test added. - docs: reject '..' in the folder arg (it writes files under f/<folder>). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): route local previews to the // tag worker Addresses CI review P1: the local graph/bundle dropped the parsed `// tag`, so a node annotated `// tag gpu` ran on the default worker in both `pipeline run --local` and `/pipeline_dev`, while the deployed pipeline routes it to that worker tag. Carry the tag through LocalScript / the pushed bundle / LocalScriptContent and pass it to runScriptPreview at all three launch sites. Verified: a duckdb node tagged `bash` produces a job tagged `bash`; regression test added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): add asset partitions/schemas routes to OpenAPI, use generated client The ducklake asset panels (PartitionStatusGrid, SchemaHistoryPanel) hit /assets/partitions and /assets/asset_schemas via raw fetch with cookie-only auth, because those backend routes were never added to openapi.yaml so the generated client had no methods for them. On /pipeline_dev (token-via-URL, no session cookie) the raw fetches 401'd. Add both GET routes + MaterializedPartition/AssetSchemaVersion schemas to openapi.yaml and call them through AssetService, which injects the bearer token, types, and cancellation automatically. Verified: Partitions + Schema tabs load in /pipeline_dev. (backfill stays a raw fetch — it's an EE-only route not in the OSS spec — with the token added inline.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): regenerate bun.lock for windmill-parser-wasm-asset package.json / package-lock.json carry windmill-parser-wasm-asset@1.740.0 but the tracked bun.lock (the CLI installs/builds/tests via bun) was stale, so fresh bun installs would resolve a different graph than the committed lock. Regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): show asset producer + its runs in the dev-preview panel Selecting a ducklake/asset node in /pipeline_dev showed 'No producer for this asset' because selectionProducers wasn't passed (it's derived from the deployed graph on the route page, absent here). Compute it from the local graph's w/rw write-edges (incl. the // materialize target) and pass it through, mirroring the route page — so the panel shows the producing script and its (preview) runs, including data-test failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): carry annotation metadata onto local-graph runnables The local graph emitted only path/usage_kind/in_pipeline/materialize_target per runnable, so /pipeline_dev and pipeline show --local weren't the same surface as the deployed graph for annotated scripts — missing the badges/lineage the shared canvas renders. Map the wasm-parsed partition_kind, freshness, tag, retry, data_tests, column_lineage, and materialize_strategy (derived append/merge/replace) onto each runnable, mirroring the deployed AssetGraphRunnableNode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude event handlers that are lineage descendants from runAll The runAll guarantee ('never fires an event handler with empty args') only held for event ROOTS — validStarts excludes them, but runAll then unions in descendants(dag, start), so a kafka/mqtt/... handler that also reads an upstream pipeline asset (a lineage descendant of a valid start) still landed in the plan. Add eventTriggerScripts() and subtract it from the selection after the descendant union. +unit test. Also: docs/pipeline-local-dev.md recipe used 'pipeline docs demo_pipeline' without --local (default queries the deployed graph → hits the empty hint); add --local. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): whole-pipeline run cuts at event handlers (drop their downstream too) The prior runAll fix subtracted event handlers from the selection but left their downstream: for manual_root → asset_x → kafka_handler → asset_y → consumer, deleting only kafka_handler left consumer selected, and topoOrder then ran it as a root with missing/stale event-derived inputs. Replace the descendant-union+delete with reachableCutting(dag, validStarts, eventHandlers): traverse from valid starts but treat event handlers as cut points, so a node reachable ONLY through an event handler is dropped while one reachable via a non-event path stays. +unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): recover // tag in the go/bash annotation fallback The wasm path carries out.tag, but the go/bash fallback (and the wasm-error degradation path) only recovered pipeline + on, so a // tag gpu on a bash/go node — or a temporarily-unparseable ts/py/sql node — silently routed the local preview to the default worker while the deployed pipeline routes to the tag. Scan for // tag in fallbackParse too. +test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): extract shared assetProducers helper The 'who writes this asset' write-edge derivation was copied verbatim in PipelineDevView and the pipeline route page — two copies that would drift. Extract assetProducers(graph, selection) into graphTraversal.ts and use it from both, keeping the dev view and route page in lockstep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): only overwrite AGENTS.md/CLAUDE.md when it's the exact generated pointer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): wire local-dev runs into the selected-node runs pane Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude data_upload/webhook entrypoints from auto CLI runs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --upload binds an object to a data_upload/webhook entry point Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): add "Run + downstream" to the dev preview detail form Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): cut non-autorun triggers on all run paths; multi-binding --upload Address CI review: apply the data_upload/webhook/event barrier cut to the single-root and bounded (--from/--to) paths, not just whole-pipeline; accumulate repeatable --upload bindings per script (were overwritten); scope dev upload keys by script+param to avoid basename clobbering; drop <script> from help text. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reseed dev run form when a local edit changes the script's args The read-only pane is keyed on script.path only, so in /pipeline_dev the selected node re-resolves on every WS bundle without remounting; PipelineScriptView cloned script.schema once, so adding/removing args left the run form on a stale schema (could run with missing inputs). Extract PipelineRunForm (owns the SchemaForm clone) and key it on the serialized schema: a real arg change reseeds the form, an unchanged re-resolve keeps in-progress input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): don't cut a scheduled/manual root that also has a non-autorun trigger Address Codex P1: the barrier set subtracted only --upload-bound scripts, so a script with both `// on schedule` and `// on data_upload` resolved as the start yet was also a barrier — reachableCutting skipped it, giving an empty run plan. Subtract all valid starts (schedule/manual roots + bound handlers) from barriers: a legitimately-scheduled root runs on its schedule path even if it also carries a caller-input trigger; pure input-only roots stay cut. Adds a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): deployed non-autorun enrichment, s3:// storage, --to cut accounting, tag regex Address CI review (Codex P1/P1/P2, Pi P2): - Deployed `pipeline run` recovers marker-only data_upload/webhook/email triggers from script bodies (like the `show` path) so input-only entrypoints are cut instead of auto-run empty on the deployed graph. - `--upload s3://<storage>/<key>` keeps the named storage (authority) instead of folding it into the key, matching the S3Object round-trip convention. - Bounded `--to` targets cut by a barrier are reported in droppedEnds (+warning), not reachableEnds. - fallbackParse `// tag` matches a single token (\S+), rejecting multi-word prose. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): header-only deployed marker scan, fail-closed enrichment, default-storage s3 keys Address CI review (Codex P2, cubic P1/P1/P2): - Deployed marker recovery scans the LEADING comment header only (shared recoverHeaderMarkers helper, reused by the show enrichment too) so a body comment `// on data_upload` can't inject a phantom trigger and over-cut. - Deployed run enrichment fails CLOSED: a script-body fetch error aborts the run instead of silently letting an input-only entrypoint run with empty args. - Revert `--upload s3://` to default-storage whole-path keys (matching pipeline `s3://` asset-URI semantics); named-storage authority-splitting broke nested default keys like `s3://raw/2026/events.csv`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reject trailing content on fallback native markers; trim s3:/// key Address CI review (Codex P2, cubic P3): - fallbackParse now requires a native marker (`// on data_upload`) to stand alone; a line with trailing content (`// on data_upload f/foo`, `# on kafka topic`) is rejected, matching the canonical parser and keeping local/deployed parity. - s3UriKey trims a leading slash so the canonical empty-authority default form `s3:///key` doesn't leak a leading slash into the object key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): persist dev WS token per-port so reconnect survives a CLI restart Address Codex P2: the /pipeline_dev auto-reconnect reuses the ws_token from the page URL, but `pipeline dev` minted a fresh random token each start, so a restart on the same port left the open page rejected by verifyClient forever. Persist the token per-port under the user-private config dir (0600) and reuse it on restart, so an already-open page reconnects — matching the reconnect behavior's intent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): scope persisted dev WS token by workspace+folder+port Address cubic P2: keying the persisted token by port alone let a stale browser tab from a previous folder's session on the same port reconnect and receive a different folder's source. Scope the token file by workspace+folder+port so a same-session restart still reconnects, but a different folder on the same port gets a distinct token that rejects stale cross-folder tabs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): caller args can't override skip-dispatch guard; hash the dev token key Address CI review (Codex P1, cubic P2): - makeLaunch / CLI run build args with `_wmill_skip_asset_dispatch` LAST (and drop any caller-supplied copy) so a run-form/`--upload` arg can't re-enable backend asset dispatch while the client orchestrates the cascade (double-run / running deployed subscribers from a local preview). Adds a cascadeRun guard test. - Dev WS token file key is a sha256 of NUL-delimited workspace+folder+port, so different folders (`a/b` vs `a_b`) can't collide onto the same token file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): canonical s3://storage/key --upload parsing; scope dev token by remote+root Address Codex P1/P1: - Restore canonical S3Object URI parsing for `--upload` s3 sources, matching the frontend's `parseS3Object` (`s3://<storage>/<key>`, empty authority ⇒ default, `s3:///key`/`s3:///nested/key` for the default store). `s3://secondary/k.csv` → `{ s3: "k.csv", storage: "secondary" }` so a named-storage object is read from the right store. (This is the canonical convention; the default-storage nested key is served by the `s3:///` form.) - Scope the persisted dev WS token by remote+workspace+root+folder+port (was workspace+folder+port), so two profiles on different remotes (or local checkouts) with the same workspace/folder/port don't share a token — a stale tab can't reconnect across a workspace/remote boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
248540ac4d |
feat: bounded-cascade selective execution for pipelines (UI + CLI) (#9695)
* feat: bounded-cascade selective execution for pipelines (UI + CLI) Run a prefix of a pipeline cascade: from a schedule/manual root, fan downstream but stop at chosen end node(s) — the path-between set over the asset-graph lineage DAG. Exposed as a canvas 'Run downstream up to…' pick mode and a 'wmill pipeline run <folder> --to' CLI command. No backend or parser changes; reads the existing graph, tags, and triggers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: surface bounded-run on the run caret, trigger-node kebab, and Test button Move 'Run downstream up to…' from the runnable kebab onto the play-button caret popover (Edit mode, next to Run / Run + trigger N downstream); add it to the trigger-node kebab so schedule/data_upload entrypoints expose it on the View page; and to the ScriptEditor Test split caret for the open script. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address CI review on bounded-cascade (cubic) - Port CLI engine test from Deno to bun:test under cli/test/ (won't run under bun test otherwise). - closure() now excludes the start node on a cycle back to it (descendants/ancestors contract); regression tests both engines. - CLI 'pipeline run --to' rejects unresolved/ambiguous end tokens instead of silently running a different subset. - Sort a copy in the runSelection order test so the launch-order assertions aren't invalidated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address standing review nits on bounded-cascade Resolves the four recurring P1/P2 findings from the codex/pi/claude reviews: - UI gate (P1): the canvas/trigger-node "Run downstream up to…" affordance was gated on the subscriber-only downstream map, so a valid start whose only downstream is a pure reader had a non-empty bounded set but no menu entry. Gate on the read-aware lineage downstream (buildLineageDownstreamMap), matching the bounded engine. - waitJob (CLI): a completed job without explicit success:true now counts as a failure, mirroring the frontend waitJobTerminal — the cascade only advances on a confirmed success. - Comment fix (CLI): the unbounded `run` path uses the read-aware lineage DAG (pure readers included); dropped the false "parity with the canvas cascade" (subscriber-only) claim. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: expose bounded-run caret for pure-reader-only starts (codex P1) The canvas wiring from the prior commit passed `onStartBoundedRun` from the read-aware lineage map, but the leaf components still hid the popover that holds the "Run downstream up to…" action behind a subscriber-only gate: - RunnableNode rendered the Run-button caret only when `hasCascade = downstreamCount > 0` (subscriber-only). A valid start whose only downstream is a pure reader got `onStartBoundedRun` but no visible action. Now the caret opens when there's a cascade OR a bounded-run start (`hasCaret`), and the "Run + trigger N downstream" item is gated on `hasCascade` so it never reads "trigger 0". - ScriptEditor's Test split button activated only when `downstreamSubscribers > 0`, falling through to a plain Test button (no caret) otherwise. Now it also activates when `onBoundedRun` is set, with the "Test + trigger N" item gated on the count. For a manual root (no trigger-node kebab fallback) with a pure-reader downstream this was the only UI entry point, so it was previously unreachable. Verified in-browser: a manual-root script writing an asset read-only downstream now exposes "Run downstream up to…" on the ScriptEditor Test caret with the cascade item hidden. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: gate ScriptEditor bounded-run on read-aware downstream; fix CLI asset-end warning (codex P2) - Details-pane (ScriptEditor) bounded-run entry was gated only on `validStartPaths`, broader than the canvas which also requires read-aware downstream (`hasLineageDownstream`). An isolated start could thus expose "Run downstream up to…" and enter pick mode with no selectable end. Now gated on `lineageDownstreamPaths` (script paths with a downstream in `buildLineageDownstreamMap`), matching the canvas. - CLI dropped-end warning called `scriptPathOf(d)` unconditionally, which slices `script:`-length chars off an asset id too — `datatable:main/raw` printed as `le:main/raw`. Now prefix-checks like the JSON output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: correct --from error to exclude only row-backed event triggers (codex P2) The bounded-start validation message listed `kafka/webhook/…` as event triggers that can't start a bounded run, but webhook/data_upload are rowless and read as manual roots (valid starts). Only the row-backed native kinds (kafka/mqtt/nats/postgres/sqs/gcp/email — EVENT_TRIGGER_KINDS) are excluded; the message now names those. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: surface dropped ends in CLI JSON; disambiguate shared-trigger bounded start (codex P2) - CLI `run --json` silenced the dropped-end warning, and the JSON payload echoed the originally-resolved `--to` list with no reachable/dropped split — a resolved-but-unreachable end looked like a clean plan that silently runs only the start. JSON now includes `reachableEnds` and `droppedEnds` (shared `idLabel` helper, asset-id safe). - Trigger nodes dedupe per (kind, ref), so a schedule shared across scripts collapses to one node, but `recordSourceTrigger` kept only the first target path — the bounded-run action then rooted at an arbitrary script (or hid when only that first script lacked downstream). Now all target paths are tracked and the action is offered only when exactly one is a valid start with downstream; multi-eligible nodes suppress it rather than guess. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: don't run hidden drafts in View-mode bounded cascade (codex P1) launchCascadeScript unconditionally preferred drafts.get(path) over the deployed script. In View mode with drafts hidden (displayGraph is deployed-only), a bounded run started from a trigger-node kebab would execute preview jobs from hidden local draft content instead of the deployed scripts the user is looking at. Gate draft execution on `mode === 'edit' || includeDrafts` — the exact condition under which displayGraph includes drafts — so execution always matches the displayed graph. No-op for scripts without a draft; the edit-mode "Run + trigger N downstream" cascade is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
7155a0bb96 |
feat: Data Pipelines alpha (#9193)
* feat: add workspace asset graph view Workspace-wide canvas of assets and their producer/consumer scripts, reachable from the assets page. Left-to-right layered layout via d3-dag sugiyama, rendered with @xyflow/svelte (same stack as the flow editor). GET /w/:ws/assets/graph returns deduped nodes + edges. Follow-ups: filters (kind/folder/search), node detail drawer, inline script edit from a clicked node. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * all * all * update * all * all * all * feat(pipeline): output-kind picker and per-(lang, output) templates Add a third stage to PipelineInsertMenu that asks what kind of asset the new script will produce (datatable / ducklake / s3 parquet / s3 object / none). The picked kind drives a real wmill SDK skeleton — typed datatable inserts, ducklake CREATE+INSERT, s3 parquet COPY, etc. — with the upstream asset auto-wired as the input source when added from an asset node. Reorder languages to bun → duckdb → python → sql so data-shaped languages surface first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * chore(main): release 1.693.4 (#8994) * chore(main): release 1.693.4 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit (#8997) * feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: include .yaml variants in collections/roles requirements lookup Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries (#9000) * fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries PR #8940 stopped lowercasing in sanitizeForFilesystem to fix #8939, where a raw-app runnableId like CamelCaseTSRunnable produced a CamelCase YAML metadata file but a lowercased code file, making them desync and register as duplicate runnables on push. That fix overshot. sanitizeForFilesystem is also reached by newPathAssigner, which serves normal apps and flows where the input is the script's human summary ("Get Users Data") rather than an identifier. There the on-disk filename is the only artifact — there's no companion YAML to keep in sync — so lowercasing was the right behavior. Removing it changed both the on-disk filename and the !inline reference in app.yaml / flow.yaml from get_users_data.inline_script.ts to Get_Users_Data.inline_script.ts on the next pull, surfacing as unwanted case churn for users updating to 1.693.x. Add a preserveCase option to sanitizeForFilesystem (default false → lowercase). newRawAppPathAssigner opts in; newPathAssigner stays on the default. Update unit tests accordingly and add an end-to-end raw-app round-trip in raw_app_sync.test.ts that pushes a CamelCase backend runnable, pulls it back, and asserts both YAML and code file preserve case with no lowercase orphan. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(cli): use readdir for exact-case orphan check on Windows The CamelCase round-trip test used fileExists("camelcasetsrunnable.ts") to assert no lowercase orphan was produced, which false-positives on Windows since the filesystem is case-insensitive and resolves the lookup to the existing CamelCaseTSRunnable.ts. Switch to readdir + toContain so the exact on-disk casing is compared identically on Linux and Windows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): wmill-lock.yaml auto-fill + --rehash-only + path-prefix dedup (#8978) * fix(cli): canonical lockfile hashes + lock upgrade migration to v3 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): use __app_hash subpath in rehash missing-entry check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): run sync pull lockfile auto-fill regardless of changes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: regenerate system prompts for new lock and rehash-only commands Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on lock upgrade Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop v3 marker; always run fallback; fail-fast on unknown lockfile version Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop yaml-round-trip legacy hash variant; recover via --rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): include legacy hash in script push staleness warning check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * revert(cli): drop canonical hash formula; keep raw-bytes hashing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf(cli): reuse change-tracker map for sync pull lockfile auto-fill Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): pin lockfile hash + yaml format and cover regression cases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): byte-stable snapshot tests for flow.yaml format Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): add app and script-metadata yaml snapshot fixtures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude review on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): factorize script-path to remote-path derivation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude + cubic review (dry-run mutation, rehash short-circuit) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): make rehash a subcommand and factorize fs walks Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): normalize line endings in yaml snapshot tests for windows ci Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash + auto-fill - Flat-layout scripts now clearGlobalLock before rehash write so legacy ./-prefixed duplicates get cleaned up (matches flow/app behavior). - Add MalformedLockfileError; sync pull auto-fill re-throws it alongside UnknownLockVersionError instead of silently warning + continuing. - Document the legacy step-removal false-negative in isFlowDirectlyStale / isAppDirectlyStale and the categorizeLocalFiles ignore-filter invariant. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix: use otel.status_message for OTLP Status.message on failed jobs (#8995) tracing-opentelemetry only recognizes otel.status_code and otel.status_message as fields that map to the OTLP Status proto. The previously-used otel.status_description fell through to the generic attribute recorder, leaving Status.message unset and preventing OTLP consumers from filtering spans on error status. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: route email trigger path through standard info channel (#8996) * docs(skill): document email triggers and S3 attachments Add an "Email triggers" section to the triggers skill covering the local-part config, the parsed_email/raw_email/email_extra_args payload, the URL-style extras convention, where to find trigger_path (only with a preprocessor, at event.trigger_path), and — most importantly — that binary attachments are uploaded to the workspace S3 bucket and surface as `{ s3: "windmill_emails/<job_id>/attachments/<filename>" }`. Scripts must use wmill.loadS3File / wmill.load_s3_file to read them. Also pulls EmailTrigger into the schema mappings so a real `email_trigger.schema.yaml` is generated, and adds Email/Azure to the trigger kinds list in the CLI agent guidance. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref for email trigger path fix Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 26184ab7a4aadfc529dcedf038aa08d36c7ad381 This commit updates the EE repository reference after PR #553 was merged in windmill-ee-private. Previous ee-repo-ref: 318a46897a605dc9be3817901f35ba5a99a0a525 New ee-repo-ref: 26184ab7a4aadfc529dcedf038aa08d36c7ad381 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * update git sync version to 1.693.5 * fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999) * fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(pg): wrap encoder errors with arg context, add fallback test Followups on #8999 review: - Wrap rust-postgres "error serializing parameter N" failures with the arg name, JSON value kind, and asserted Postgres type plus a hint about an explicit cast — so users see actionable context instead of an opaque WrongType. - Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree on the Type for every recognised arg_t when the JSON value matches its natural Rust kind. Catches future drift if either side changes. - Integration test for the prepare + query_raw fallback path: confirms unrecognised arg_t (custom enum) is routed through prepare and the server-resolved type appears in the failure surface — flips into a test failure if a regression accidentally routes unrecognised types through query_typed_raw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add otyp_inferred flag + regex-based placeholder renumbering Two follow-ups from the review of #8999: 1. **Issue #1 (Number/Bool + explicit text decl in WHERE)** Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets it `true` only at the "no info → fall back to text" site (bare `$N`, no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep it `false`. In `convert_val` this flag distinguishes: - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works (`text = text` operator). Pre-#8988 behaviour, restored. - parser-default text (bare `$N`) — bind the value's natural Rust type so the regression case (`Value::Bool` against a real `bool` column via `CAST AS bool`) keeps working. `Arg` is in `windmill-parser`; the new field has `#[serde(default)]` so persisted signatures stay backward-compatible. 2. **Issue #4 ($5/$50 substring rewrite collision)** Replace the per-index `String::replace` chain (which turned `$50` into `$10` when oidx=5 was processed first) with a single regex pass. `\d+` is greedy, so `$5` and `$50` match as distinct units; indices outside the mapping are left intact. 3. Tests: - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline- cast/decl/mixed shapes. - executor unit: `convert_val_bool_against_every_arg_t` and `convert_val_*_number_*` split each text-like target into explicit vs inferred expectations. - executor unit: `renumber_sparse_placeholders_no_collision`. - integration: `test_postgresql_arg_type_combinations` adds 4 cases covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and sparse positional args ($5/$50). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality Backend: 1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s `ToSql for String` / `FromSql for String` reject `Kind::Enum` and `Kind::Domain` even though the wire format is plain UTF-8. The wrapper accepts those kinds in both directions. End result: explicit `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col` results come back as JSON strings instead of erroring at the FromSql layer. 2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text / non-temporal arg_t fell through to `Box<String> + TEXT`, which then failed at the server (no implicit cast text→numeric in expression context). Now strings are parsed into the matching native type with clear error messages on parse failure. 3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering (which fixed the `$5/$50` substring collision but still walked through string literals and comments, mangling `'price: $5'` etc.) with a walk over `parse_pg_statement_arg_positions` — the same string/comment/dollar-quote-aware tokenizer used for index discovery. Adds `parse_pg_statement_arg_positions` to the parser's public API. SDK: 4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now stringifies bigints before serialisation; the executor accepts numeric strings into BIGINT arg slots via the existing `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split so `BigInt` always resolves to `BIGINT` (was reaching `Number.isInteger(BigInt)` which returns false → wrong default). 5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers primitive types only (number / bigint / string / boolean); mixed or nested arrays still fall back to JSON. Mixed int/float widens to `DOUBLE PRECISION[]`. 6. **`.query()` positional bug**: previously the `.query()` method abused the template-tag builder, which appended `$N::TYPE` after the user's literal SQL string instead of binding by position (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()` builds the executor-shaped content directly: a `-- $N argN (TYPE)` declaration block followed by the user's SQL verbatim. Tests: - Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments` asserts string literals, comments, and dollar-quoted blocks don't produce positions (so renumbering doesn't mangle them). - Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling` uses the new position-aware path and includes string-literal + comment + `$$…$$` cases. Existing convert_val tests grow to cover new String→numeric/real/double/oid/bool arms. - Integration: `test_postgresql_arg_type_combinations` adds 13 cases (enum round-trip both directions, string→numeric/real/double/bool/oid, string-literal `$N` non-mangling). The prepare-fallback test now asserts SUCCESS (not failure) for enum encoding via AnyTextValue. - SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests) exhaustively covering inferSqlType primitives + arrays, parseTypeAnnotation, datatable() template tag (with all the new shapes — BigInt, homogeneous arrays, RawSql, schema preamble), datatable().query() positional, and ducklake() shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache) Found while exhaustively probing custom-type DX: every cached-connection reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL` deallocates *all* prepared statements server-side — including the typeinfo statements that tokio_postgres caches per-Client to resolve custom enum / domain Oids. tokio_postgres still held `Statement` objects whose names the server had forgotten, so the next custom-type query failed with intermittent "prepared statement \"sN\" does not exist" errors. The failure was easy to reproduce: any sequence that forced typeinfo lookup for two different custom-type kinds on the same cached connection (e.g. enum followed by domain) would hit it. Replace `DISCARD ALL` with a curated reset that explicitly targets the state we actually care about, *without* touching prepared statements: RESET ALL — GUC parameters (search_path, application _name, statement_timeout, …) RESET SESSION AUTHORIZATION — undoes both `SET SESSION AUTHORIZATION` and `SET ROLE` (RESET ALL does NOT — these aren't GUC parameters, so without this an elevated role from a previous job would silently leak) UNLISTEN * — drops LISTEN registrations CLOSE ALL — closes open cursors Trade-off: temp tables, advisory locks (session-scoped), and user-created PREPARE statements may persist across cached-connection reuse — rare in datatable / PG-script workloads. tokio_postgres's typeinfo cache survives intact, so custom enum / domain queries are fast on subsequent reuse. Tests: - `test_postgresql_custom_types_on_cached_connection` — runs 10× alternating enum + domain queries on a cached connection. Pre-fix this failed with `prepared statement "sN" does not exist` after the first reuse; post-fix passes. - `test_postgresql_set_role_does_not_leak_across_cached_connection` — switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres role, then runs a follow-up job and asserts current_user/session_user are restored. Specifically catches the case where someone might switch back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION AUTHORIZATION) and silently introduce a permission-leak vector. - All existing session-isolation tests (`test_postgresql_cached_connection_resets_session`, `test_postgresql_single_worker_session_isolation`, `test_postgresql_100_jobs_cached`) continue to pass. Found via end-to-end probing of datatable / PG-script DX, not previously covered: the existing isolation tests only did `SET ROLE postgres`, the connecting user, so the leak was invisible. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): address PR #8999 review (cubic + claude) cubic (P1, real bug): - `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but chrono `NaiveTime` only encodes for TIME (same caveat as the scalar arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz` assignment cast at the column site. Add an explicit unit test. claude (#1, silent failure → explicit error): - `Bool` + explicit `(char)` / `(character)` decl previously silently bound BOOL, hoping the server would cast at the use site — but PG has no implicit `bool→char` and the resulting error ("operator does not exist: bool = char") was opaque. Now error at bind time with an actionable hint to use `bool` decl or pass the value as a "t"/"f" string. claude (#2, asymmetry doc): - Object/Array still coerce to text on `matches!(typ, Typ::Str(_))` (covers both explicit AND inferred-default text), unlike Bool/Number which key on `explicit_text_target`. The asymmetry is intentional (no implicit `jsonb → text` cast in expression context vs PG having implicit `bool/int → text` casts) — added a body comment so future maintainers don't try to "align" them. claude (#3, perf): - `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions` walked the SQL tokenizer twice. Fold into a single pass that derives the index set from the position list. claude (#4, fmt drift): - `cargo fmt` over the parser crates I touched with perl scripts in the earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py, rust,graphql,yaml,r}). Net cosmetic. claude (#5, parseTypeAnnotation): - One-line caveat in the SDK's `parseTypeAnnotation` that the returned string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a real PG type, but the only consumer just checks `!== undefined`). While here — discovered + fixed independently while exhaustively probing DX: - **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing intermittent `prepared statement "sN" does not exist` errors on custom-type queries after cached-conn reuse. New regression tests: `test_postgresql_custom_types_on_cached_connection` and `test_postgresql_set_role_does_not_leak_across_cached_connection` (the latter catches the case where someone might switch back to `RESET ALL` alone and silently introduce a permission-leak vector — RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION). - **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00") and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") — neither parseable by `date-fns parseISO`, JavaScript `new Date()` is lenient enough to handle them but several frontend `App*Input.svelte` components use parseISO and fail silently. Switched to ISO-8601 with `T` separator and `+00:00` offset; arg-parsing path still accepts the legacy " UTC" suffix for back-compat. Test coverage: - 17/17 unit (`pg_executor::tests`) - 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`) - 27/27 parser (`windmill-parser-sql`) - 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling Found while probing PG-script DX with millions of numeric cells: 1. **Numeric precision-loss warning**: `numeric` results are still serialised as JSON Number (back-compat — switching to JSON String would silently break user code doing arithmetic on results), but we now detect `Decimal -> f64 -> Decimal` round-trip failure and emit a single job-log warning recommending a `::text` cast in the SQL. Bounded by `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic load + one fetch_sub on the hot path; first lossy value short-circuits to a single load thereafter). Worst-case overhead on a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic loads (vs. ~100ms unbounded). 2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"` (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is what the apps `App*Input.svelte` components use, so timestamp values silently failed to round-trip into date pickers. Switch to ISO-8601 (`T` separator + `+00:00` offset) on the result side; arg-parser continues to accept the legacy `" UTC"`-suffixed format for back-compat. 3. **Float NaN / Infinity results**: `Number::from_f64` returns None for NaN / ±Inf, which `pg_cell_to_json_value` was raising as "invalid json-float" — failing the *entire* query if any cell held one of these special values. Now serialise them as JSON strings ("NaN", "Infinity", "-Infinity") and let the rest of the row come through. Arg-side: `s.parse::<f64>()` already accepts the same strings. Tests: - `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit cases for the precision-loss predicate. - `precision_check_budget_caps_per_query_overhead` — locks in the budget cap and the loss-flag short-circuit. - All 9 PG integration tests + 17 unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults While probing PG-script DX further found three more frictions: 1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;` meant session-scoped advisory locks (`pg_advisory_lock`) leaked across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()` to the chain — `DISCARD ALL` covered this implicitly via `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it in the switch. 2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g. `-- $1 amount (numeric)`) but not provided in the args object was bound as NULL with no error / warning. Misspelling the key in the args object silently produced a row of NULLs — a notorious DX debugging trap. Now: collect the names of declared-but-missing args during dispatch and emit a single one-shot warning to the job logs at end-of-query naming each one. Bound NULL is preserved for back-compat. 3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries `arg.default = Some(Number(5))`, but the dispatch fell straight to NULL when the arg was missing. Now: respect the default — user-supplied value > declaration default > NULL. Also fixes the warning logic above (only warn for args that *don't* have a default). Tests: existing 19 unit + 9 integration pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values Two more frictions found while probing SDK end-to-end against a real datatable resource: 1. **Multi-word array types lose the [] suffix in the parser**. `transform_types_with_spaces` recognises aliases for "double precision", "character varying", "timestamp with time zone", etc. but its return type was `&'a str` — only the bare alias, never with a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at the first space, so the regex's own `(?:\[\])?` array-suffix branch sees only `"double"` (not `"double precision[]"`); the `[]` was silently lost. Result: `$1::double precision[]` (which the SDK now emits for homogeneous float arrays via the new auto-tag) routed through `Value::Array → Type::JSONB` and the server failed with "cannot cast type jsonb to double precision[]". Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>` and re-check the trailing bytes after a multi-word match. If they start with `[]`, return `format!("{alias}[]")` — Owned. Single-word types and the no-match path keep returning Borrowed slices, so no allocation in the hot path. 2. **Array arms in `convert_vec_val` rejected stringified values for numeric / int* / bool / oid / real / double**. The scalar `convert_val` already parses strings into the matching native type for these arg_ts, but the array variant only accepted JSON-native counterparts. Sending `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with "Mixed types in array". Now the array arms mirror the scalar ones — `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes round-trip cleanly. Tests: 19 unit + 9 integration pass; existing parser tests cover the multi-word array forms (the regex-cap behaviour didn't break for single-word types, and Cow plumbing is transparent to all callers). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files CI failures: the perl-driven sweep that added `otyp_inferred: false` to every `Arg { ... }` literal when I introduced the field in the parser schema covered `src/lib.rs` files but missed: - parsers/windmill-parser-bash/src/lib.rs (mass-edited but a later format pass un-applied a few sites) - parsers/windmill-parser-go/src/lib.rs (same) - parsers/windmill-parser-graphql/src/lib.rs (same) - parsers/windmill-parser-nu/tests/tests.rs (test file — not swept the first time) - parsers/windmill-parser-ts/tests/tests.rs (test file — same) Also tightened the regex to handle `oidx: None` without the trailing comma (some test files had the field as the last initialiser line). `cargo build --features <CI feature combo> --workspace --all-targets` is clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string Two more frictions found while running the actual SDK end-to-end against a live datatable resource: 1. **JS `Date`** fell into the typeof "object" branch and was tagged `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's `json → text → timestamptz` implicit cast chain, but `${date}` against a `timestamptz` column without a user-supplied cast bound the value as a JSON string and the comparison `timestamptz = json` failed. Now: `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`; `serializeArgValue` emits `Date.toISOString()` so the executor's `Value::String → TIMESTAMPTZ` arm parses it cleanly. 2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)` returns `"null"` per the JS spec, so the value reached the executor as JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL double. Fix: detect non-finite numbers in `serializeArgValue` and stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals directly, and the result-side already renders the values as JSON strings (matching round-trip). SDK unit tests grow from 42 → 44 passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(pg): integration coverage for multi-word arrays + stringified array elements Locks in the two array fixes from the previous commit (`fix(pg): multi-word PG types with [] suffix lost the array-ness`) with end-to-end cases in `test_postgresql_arg_type_combinations`: - `double precision[]`, `character varying[]`, `timestamp without time zone[]` — verifies the parser keeps the `[]` suffix after multi-word alias resolution. - `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies the array arms of `convert_vec_val` apply the same string-coercion the scalar arms do. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: fix indentation drift on otyp_inferred lines cargo fmt cleanup of leftover indentation where the perl-driven sweep that introduced the otyp_inferred field landed at the wrong column. No behaviour change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat: support assigning a worker tag to app inline scripts (#9002) * feat: support assigning a worker tag to app/raw-app inline scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: omit empty tag field from inline script raw_code payload Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: shrink tag popover width --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * feat(pipeline): 2-col picker, draft path edit, save-all + leave guard Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * all * update * fix(cli): forward HEADERS env var on every backend fetch call (#9075) Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped the `HEADERS` env var, causing requests to fail behind auth gateways like Cloudflare Access (same shape as #6421): - `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw `fetch` for the `skip_if_noop` query param. - Script preview `/jobs/run/preview_bundle`. - App dev `/jobs_u/getupdate_sse` SSE stream. - `wmill docs` `/api/inkeep`. All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()` so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque JSON parse failure. Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that 403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and otherwise reverse-proxies to the test backend, then runs `wmill sync push` of a fresh script through the proxy. Negative case (no `HEADERS` env) verifies the proxy actually gates; positive case asserts every request including `/scripts/create` reaches the backend with the headers attached. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): add --parallel flag to generate-metadata (#9074) * feat(cli): add --parallel flag to generate-metadata * fix(cli): validate --parallel input and harden flush ordering * perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078) * fix(cli-tests): stabilize flow lock-gen race + Windows path (#9080) * fix(cli-tests): stabilize flow lock-gen race + Windows path Three CLI test failures on the latest main, all flaky on CI: 1. `Mixed Case Paths: pull and push flow with capitalized folder` and `Integration: Mixed scripts and flows with nonDottedPaths are idempotent`: flow create/update queues an async FlowDependencies job that fills inline-script lockfiles and rewrites flow.value. The tests pulled/pushed before the worker finished, so dry-run idempotency saw phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a `waitForFlowDependencyJob` helper that polls `/flows/get` for the latest `dependency_job` and `/jobs_u/completed/get` until it lands, and called it after each API/CLI flow write in both tests. 2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only, added in #9075): the new test built the CLI entrypoint via `new URL("..", import.meta.url).pathname`, which yields `/C:/...` on Windows and `Bun.spawn` rejected before reaching the proxy, leaving `rejectedRequests.length` at 0. Switched to `fileURLToPath` + `node:path.join` to match `cargo_backend.ts`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli-tests): use /flows/deployment_status to actually wait for dep job CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob` as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`, but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60) do not include that field. The helper exited on the first iteration without polling. Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns `{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID written into `deployment_metadata` in the same tx as the dep-job push (backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by the time the create/update API call returns, the response carries the latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to ~32s, confirming the helper now actually waits instead of returning immediately. The 404 short-circuit in `sync_pull_push.test.ts` still works — `get_deployment_status` returns 404 when the flow is absent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(flows): cache resolved flow_env per flow execution (#9079) * perf(flows): cache resolved flow_env per flow execution * perf(flows): tighten flow_env cache cap to 1024 and clarify memory note * perf(flows): don't cache transient flow_env resolution failures * chore(main): release 1.698.0 (#9076) * chore(main): release 1.698.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix: reject root-rooted paths in ansible playbook validator on windows (#9081) * fix(native-triggers): serialize Google channel renewal across replicas (#9060) * fix(native-triggers): serialize Google channel renewal across replicas `sync_all_triggers` runs every 5 minutes on every windmill-app replica with no leader election. Multiple replicas were each rotating the webhook token, creating a new Google watch channel, and racing the trigger UPDATE — leaving the loser's new token (in `token`) and channel (in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week without the silent best-effort `delete_token_by_hash` ever logging a warning. Wrap each per-trigger renewal in a transaction and acquire the row with `SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row instead of duplicating the work. The lock spans `rotate_webhook_token` → Google API call → `update_native_trigger_service_config` and is only released on commit. Re-checks `should_renew_channel` after acquiring the lock so a replica that committed seconds earlier doesn't trigger a duplicate renewal. The pattern matches existing batch-cleanup paths in `monitor.rs` (job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites. Also logs at `debug!` when `delete_token_by_hash` finds no matching row, so future investigations can distinguish "deleted" from "not found" without changing the `Ok(false)` contract. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address claude review: - #5: per-skip log info -> debug (expected outcome under SKIP LOCKED) - #2: warn moved out of delete_token_by_hash to the call site that knows the expected state (try_renew_channel_locked); other callers are race-prone and shouldn't warn - #3: NULL service_config now warns (anomalous case) - #4: post-Google-API DB-update + commit failures log distinctly so the channel-orphan case is grep-able Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration, mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the existing 'ephemeral-' filter excludes them from user-token email/critical-alert paths (no filter changes in 3 places). Orphans now self-clean via the existing expiry sweep in monitor.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Address second-round review: - Claude #1 (P2): username_override_from_label now strips the 'ephemeral-' prefix for ephemeral-webhook-* labels, so created_by stays webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-... (preserves audit/job-list filter compatibility) - Codex (P2): updated renew_channel doc — labels are no longer copied; rotate mints fresh ephemeral-webhook-google-{rd5} with 14d expiration - Claude #3 (optional): test_rotate_webhook_token now asserts the rotated Google token has an ephemeral-webhook-google-* label and a populated expiration Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fixup! fix(native-triggers): serialize Google channel renewal across replicas fixup! fix(native-triggers): serialize Google channel renewal across replicas Reconsider the previous fixup: stripping the 'ephemeral-' prefix made created_by no longer match token.label exactly, defeating the linking purpose. Just allowlist 'ephemeral-webhook-' alongside the other recognized webhook/email/ws prefixes — created_by becomes ephemeral-webhook-google-XXXXX, matching token.label exactly. The 'ephemeral-' substring also informs operators that this is a system-managed auto-expiring token vs a user-managed webhook trigger. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): bump svelte version in `wmill app new` template (#9084) * fix(cli): bump svelte version in `wmill app new` template The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte compiler bundled in `wmill app dev` emits `$.delegated('click', ...)` calls. The `delegated` export was added later, so 5.45.2 doesn't have it — esbuild warns `Import "delegated" will always be undefined`, replaces the call with `void 0`, and the page crashes at first event-handler bind (white screen). Bump to `^5.55.5` so the compiler and runtime stay in sync. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): bump svelte version in raw_apps UI template Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a package.json with `svelte: "5.45.2"`. That works today only because the bundled rolldown worker also pins 5.45.2 — when the worker is upgraded past 5.51.1, the compiler will emit `$.delegated()` and the runtime won't have it, producing the same white-page crash that hit the CLI. 5.55.5 still exports `event` (used by the current bundled compiler), so this is forward-compatible: it works with the 5.45.2 compiler now and won't break when the worker is upgraded. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085) * feat: parse windmill_failure field to tag run as failure (#9073) * feat: parse windmill_failure field in job result to tag run as failure * feat: preserve top-level fields when windmill_failure tags a run as failure * fix: address review findings on windmill_manual_failure * refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases * fix: prefer injected ManualFailure error over sibling name/message in OTel * fix: hide _ENTRYPOINT_OVERRIDE jobs from script/flow history panel (#9088) * fix(flows): populate error handler input args from failure picker (#9087) * fix(flows): populate error handler input args from failure picker * style(flows): fix indentation in failure-step branch * fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090) The Python per-package dependency cache could persist an incomplete wheel extraction with `.valid.windmill` set, then propagate that broken artifact to every worker through the object store. Customer hit this on argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on botocore/httpx (truncated tars). Symptom is a runtime ImportError that looks like a missing dependency declaration rather than a Windmill bug. Three changes that together stop the propagation: 1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and confirm every listed path exists on disk before writing `.valid.windmill`. On failure, wipe the directory and fall through to a fresh local install — the next install also self-heals the broken object-store entry by pushing a fresh tar. 2. After `uv pip install` succeeds, run the same RECORD check before queuing the piptar upload or writing `.valid.windmill`. A bad install never becomes the source of a broken tar in the object store. 3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes for upload, so we never push an unfinalized archive (no end-of-archive marker) to the object store. Verified with a 60-package end-to-end integration test (first-fill → clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar → detect-and-self-heal). All 27 packages on the live test pulled cleanly, and the deliberately corrupted argon2-cffi tar was caught with the exact expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py") and replaced with a fresh tar. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(main): release 1.699.0 (#9082) * chore(main): release 1.699.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat(cli): auto-infer args for `wmill app push` (#9091) Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`) with no args. The local path defaults to CWD, and the remote path is derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/ `__app`/`__raw_app` suffixes stripped. Either, both, or neither positional argument can be passed. Also resolves `file_path` against the user's original CWD before `resolveWorkspace` may chdir to the wmill.yaml root, so a relative `file_path` argument is interpreted from where the user invoked the command (previously it could resolve against the wrong directory). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * all * fix(pipeline): live-update graph for annotations and body assets * fix(pipeline): persist draft body edits across node switches * fix(pipeline): persist live writes per draft to keep output node fresh after switch * feat(pipeline): animate graph edges only while a runnable is executing * feat(pipeline): add run button on script nodes + recomputing hint on preview * feat(pipeline): compact preview layout, two-way Test/Run sync * fix(pipeline): test button cross-browser placement (no overflow trick) * style(log-viewer): replace took/mem-peak labels with timer/cpu icons * style(log-viewer): hyphenate Auto-scroll label and prevent wrapping * style(log-viewer): lowercase auto-scroll label, force vertical scrollbar * style(log-viewer): force horizontal scrollbar instead of vertical * fix(log-viewer): scope overflow-x to top bar so pre doesn't drive panel width * fix(pipeline): overlay live body-asset writes for persisted scripts too * fix(pipeline): persist inferred body assets at save so edges survive page reload * fix(pipeline): snapshot live draft writes at persist time so they survive reload * fix(pipeline): keep inferred body writes on the canvas across selection changes * fix(pipeline): untrack inferredWrites cache mutation to break effect loop * fix(pipeline): refetch asset graph after persisted-script save * feat(pipeline): optional AI prompt when creating a pipeline script * all * all * test: cover asset-trigger dispatch end-to-end through worker * feat(pipeline): split-button Test with optional downstream cascade * feat(pipeline): cascade option on graph Run + match button heights * style(pipeline): match caret bg/text to Test button's accent-secondary * feat(pipeline): split Run pill on graph node exposes cascade option * feat: live run activity + status badges in pipeline asset graph - folder-scoped queue poll lights up the downstream asset-trigger cascade (not just the launched script); zero requests at rest, catch-up for fast hops, auto-disarm when idle - per-runnable node badge: last-run status + session run count - animate unsaved/live-parsed edges (was unconditionally suppressed) - background-pane click no longer clears selection - run-bridge guarded so node selection/save no longer triggers a test Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: live activity log, optimistic badges, node-avoiding graph edges - collapsible folder activity log (PipelineEventLog): live job feed, polls only while open/active, slow idle cadence, capped + pruned - composable: observe mode + events list + run-count anchored to graph-open time (pre-existing history excluded) - optimistic node badge: launched script shows running instantly via the zero-latency activeRunnable hint, keeps the polled run count - activity pane height capped (min(18rem,40vh)) then scrolls - route asset-graph edges through sugiyama-computed waypoints so they go around nodes instead of under them; bezier fallback for adjacent-layer / draft-overlay edges Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: prefetch all folder script assets so graph is stable on load On pipeline load, eagerly infer body assets for every persisted folder script and seed the existing inferredWritesByPath overlay, instead of only filling it when a node is selected. Scripts whose persisted asset rows are missing (e.g. object-form writeS3File) now have their edges from first paint, so clicking a node no longer re-layouts the graph. One-shot per (workspace, base-graph) load, untracked map reads, generation-cancelled, pool-capped fetches. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf: guard no-op poll re-layout; dedupe write-asset extraction - skip reactive ids/states/events reassignment when unchanged, so an idle poll tick no longer re-runs the full sugiyama layout every 3-6s - bound countedJobIds (rebuilt from eventsById in lockstep with prune) - extract shared extractWrites() helper, replacing 4 copy-pasted write-asset filter/map blocks in the pipeline page - compute activeRunnable node-id once, reuse for the active-edge set and the optimistic badge (flattened ternary); trim narrating docs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: live read-lineage overlay for inferred body assets Renaming e.g. duckdb read_parquet('s3://...') / loadS3File now updates the asset->reader edge live instead of only after Save re-derives the persisted asset rows. - extractReads() (+ shared refsByAccess) mirroring extractWrites - inferredReadsByPath sticky cache, filled by handleAssetsChange and the load prefetch alongside writes - replace the write-only overlay loop with one overlayLineage(map, access) helper invoked for both 'w' and 'r' (net DRY) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: detect S3 assets passed as SDK object arg in ts parser Mirrors merged PR #9181 so feat/asset-graph-view is self-contained (local origin/main is stale and lacks it). Object/{ s3, storage } form of writeS3File/loadS3File is now detected, not only the bare s3:// string literal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate wasm Cargo.lock + frontend package-lock Lockfile churn from local wasm-pack (asset target) + npm operations during the asset-graph work. No source/dependency-intent change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: revert to bezier graph edges; add parsing-assets hint The sugiyama-waypoint routing looked worse than the original; revert AssetGraphEdge/assetGraphLayout to the pre-routing bezier logic (same as the flow editor's BaseEdge) and drop the now-unused route plumbing from the canvas. Add a small 'Parsing assets…' hint shown while the load-time prefetch sweep is still inferring folder scripts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: extract pure resolveGraph merge + unit tests Move the ~230-line graphWithDraft precedence/merge (base < session- inferred < draft-seeded < open-script-live, +read/write/annotation overlays, +dedup) out of the 1648-line route into a pure, testable resolveGraph() module; the route's graphWithDraft is now a thin $derived. Behaviour extracted verbatim. 10 unit tests cover the precedence matrix. Phase 1 of the state/render split. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: graph controls top-right, lift minimap, hide Save when unchanged Controls -> top-right horizontal, no lock toggle; MiniMap !mb-10 so it clears the activity bar; hide the per-script Save button when the script is already at its latest save point (drafts still show Create). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: scope runtime-asset prune by id to spare static lineage rows prune_runtime_assets deleted by (workspace_id, path, kind) tuple, so trimming surplus usage_kind='job' rows for an s3 path also wiped the static usage_kind='script'/'flow' producer rows for the same path — silently breaking the asset-trigger cascade (fetch_producer_writes found no writes; downstream never dispatched; required band-aid re-syncs). Delete the surplus job rows by id instead; the inner query is already scoped to usage_kind='job'. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: don't re-pulse already-running jobs after they finish The catch-up pulse re-added a completed job to the active set if its start was within the (lagging) lookback window — even one we'd already animated the whole time it ran — keeping its edges lit ~a poll interval past completion (~5s after a 3.5s test). Track job ids seen in-flight and skip the pulse for them; it still fires for hops whose whole lifetime fell between two polls. Bound the set in lockstep with eventsById; cleared on dispose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: don't catch-up-pulse the runnable launched from the graph If the poll never sampled a launched run's in-flight window, the catch-up pulse re-flashed its edges one tick after it correctly stopped (the page already animated it zero-latency via activeRunnable). arm(launchedId) records the launched runnable id; catch-up skips it. Cascade hops (other ids) still pulse. launchedIds cleared on stop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style: nudge graph controls left to clear panel toggle Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: partition value resolver + asset-cascade propagation windmill-common/partition: pure resolver — time kinds (tz/format/start anchor) + dynamic $.a.b JSONPath; 9 unit tests. asset_dispatch: read the producer's resolved partition and thread it into every cascaded subscriber's args + trigger.partition, so a chain resolves once at the top. No migration (cascade needs no spec lookup). Stage 1+3 of pipeline partition runtime; run-start resolution is Stage 2. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: show args form in compact pipeline preview when script has inputs AssetGraphDetailsPane keeps the compact (hideArgs) preview but, via a new previewPanel.argsAboveLogs flag, renders a compact SchemaForm between the floating Test button and the logs/result panel when the script declares inputs (e.g. a partitioned script needing a `partition` arg). The preview pane also grows ~18pts so the args form doesn't shrink logs/result. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: parser join-mode (`// trigger all`) + script_trigger.join_all Stage A: JoinMode{Any(default),All} + `// trigger any|all` directive in parse_pipeline_annotations; TriggerSpec::is_partition_bearing() (path contains {partition}); join_mode threaded through all 4 asset-parser crates (ts/py/sql/yaml). Stage B: reversible migration adds script_trigger.join_all; insert_script_trigger writes it; deploy path sets it from the parsed annotation. No reader yet (AND-join dispatch is the next stage) so runtime behaviour is unchanged. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: resolve pipeline partition at job execution time Stage C: in handle_code_execution_job, once the script content is loaded, parse the // partitioned annotation (free here) and resolve the concrete partition once — schedule fire-time (scheduled_for anchor, not wall-clock) for time kinds, triggering payload for dynamic. The value is injected into the in-memory args the body sees (via a shadowed job clone) and persisted back to v2_job.args so dispatch_asset_triggers propagates the same value down the cascade. Already-set (explicit/backfill/cascade) partitions are never re-resolved (run identity immutable); unresolvable partitioned runs fail with a clear error. Integration test exercises the full worker loop + cascade propagation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: AND-join barrier for partitioned pipeline subscribers Stage D: a // trigger all subscriber no longer fires on any input. New join_pending_inputs slot table keyed (workspace, subscriber, partition); fetch_subscribers now returns join_all and the dispatch loop records each partition-bearing input arrival, pushing the subscriber once only when every partition-bearing input it declares is present for that partition. Per-partition slots, cleared on fire (re-accumulate, no double-fire), skew-immune (unlike debounce). Case-3 guard: an unpartitioned producer or a reference (non-{partition}) input never fires a partitioned join. Integration test covers wait/fire/isolation/no-double-fire. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: opt-in // debounce for asset-cascade subscribers (parser + schema) Stage E1+E2. Parser: script-level // debounce <dur> + per-// on debounce=<dur> override (edge wins, else script default, else none = fan-out, unchanged); TriggerSpec::Asset carries the per-edge override; split_trailing_kv_opts separates the ref from trailing key=val opts. Schema/deploy: reversible migration adds script_trigger.debounce_s; parse_duration_secs (bare int or <n>s|m|h|d, fail-safe on garbage) resolves the effective per-edge window at deploy and writes it per row. No reader yet (dispatch wiring is E3) so runtime is unchanged. New unit tests for the parser directive and duration parsing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: apply opt-in debounce to asset-cascade subscriber dispatch Stage E3. fetch_subscribers now also returns debounce_s; push_subscriber builds real DebouncingSettings (delay + a (subscriber, partition) key, so distinct partitions never collapse and latest-in-window falls out) instead of ::default() when the edge opted in. Default stays no-debounce (fan-out — the prior deliberate behaviour, now overridable rather than reversed). Wiring test asserts the dispatched job carries the configured window/key and an undebounced edge carries none. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: atomic AND-join gate + preserve resolved partition; drop scratch artifacts Addresses local-review findings before PR: - P1: record_and_check_join_slot was a non-atomic check-then-act on a pooled connection; concurrent completion of a subscriber's last two partition-bearing inputs on different workers could double-dispatch. Now one transaction guarded by a tx-scoped advisory lock keyed on (workspace, subscriber, partition) so the gate fires exactly once. - P2: the preprocessed-args overwrite in result_processor replaced args wholesale, dropping a partition resolved by resolve_partition_for_job; the UPDATE now preserves an existing persisted partition key. - P2: gate resolve_partition_for_job on a cheap code.contains check so non-pipeline script jobs skip the annotation scan on the hot path. - P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a local scheduler lock accidentally committed; gitignore the lock. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: AND-join fires once under concurrent upstream completion Regression for the check-then-act race fixed by the advisory-locked transactional gate: releases N producer dispatches simultaneously via a barrier and asserts the AND subscriber is pushed exactly once and the slot is cleared. The invariant holds for the correct gate regardless of interleaving; a non-atomic regression fails it. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * test: fuller partitioned join + multi-hop pipeline coverage Exercises a complex pipeline combining options end to end: two partitioned producers fanning into a // trigger all join, then a multi-hop downstream chain. Asserts the resolved partition propagates unchanged at every hop, chain depth increments per hop, the AND barrier fires exactly once, and a second partition opens an independent slot with no cross-partition bleed across the whole graph. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * refactor: simplify pipeline code per review (dedup, single-parse, constant) - ParseAssetsOutput::new() collapses the 6-line annotation copy-paste across the 4 asset-parser crates to one call site. - asset_dispatch: parse the cascade trigger object once and pass it to the depth/partition readers instead of deserializing it twice; add a TRIGGER_ARG constant for the previously stringly-typed key (3 sites). - scripts deploy: drop a redundant debounce_default clone. No behavior change; 29 parser + 6 dispatch integration tests green. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat: reap abandoned AND-join slots after a TTL (default 60d, per-slot) join_pending_inputs slots are normally cleared when the join fires; partial slots whose inputs never all arrive (upstream removed/renamed, one-off dynamic partition key, permanent skew) would otherwise leak. windmill_queue::asset_dispatch::reap_stale_join_slots, called from the monitor's delete_expired_items loop, deletes a (workspace, subscriber, partition) slot only when its MOST RECENT row is older than JOIN_SLOT_TTL_SECS (60d) — per-slot, never per-row, so a legitimately slow join is not corrupted mid-accumulation. Conservative default; per-join configurable TTL via the annotation is a planned follow-up. Test covers stale-reaped / fresh-kept / mixed-slot-kept. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * update * feat: path-less native trigger markers + missing-trigger placeholder * feat: pipeline // tag and // retry annotations + dispatch_event log * fix: derive test-pane min from split-axis dimension (height in bottom layout) * feat: show last run logs/result when a script node is selected * fix: backfill asset rows from script.assets for pre-feature scripts * feat: job-id link + dispatch popover above script log/result * style: drop 'dispatched' label, keep just the check icon * fix: drop tag picker from pipeline script editor (set via // tag annotation) * Nicer UI * refactor: move google ai proxy handling to windmill-ai (#9260) * refactor: add ai proxy execution mode * refactor: move google ai proxy handling * refactor: share google ai request building * fix: early return should consider failure_module result (#9241) * fix(flows): flag noLogs jobs and lazily resolve them in log panel (#9099) * fix(flows): flag noLogs jobs and lazily resolve them in log panel * fix appending to flag * fix: preserve WM_LOGS_SKIPPED sentinel on SSE/replay completion pickMoreCompleteLogs resolved both sentinel and undefined to '', so the SSE completion event (whose job field is fetched .without_logs()) would clobber the sentinel placed by flagSkippedLogs. The module log panel then saw '' instead of the sentinel, defeating the lazy-resolve path. Also wire onLogsResolved on the OutputPickerInner inline LogViewer so a lazy resolve writes back to flowStateStore.previewLogs, matching ModulePreviewResultViewer and avoiding repeated fetches on remount. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(main): release 1.705.0 (#9229) * chore(main): release 1.705.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * chore: add playwright mcp for frontend verification (#9269) * feat: CLI datatable serve / psql (#9267) * feat(cli): add datatable list and run commands * feat(cli): render datatable query results as a table * feat(cli): serve datatables as a postgres-wire endpoint * feat(cli): add 'datatable psql' to launch psql against the proxy * feat(cli): route datatable serve by client-supplied database name * override database list + password option * fix: support extended queries in datatable serve * fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma * refactor: extract raw_output envelope encoding into pg_raw_output module --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> * oom_adj nit * feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting (#9271) * feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting Allows operators to point `uv python install` at a private mirror of the python-build-standalone releases. Configurable via the `UV_PYTHON_INSTALL_MIRROR` env var or the `uv_python_install_mirror` instance setting, with the env var as the boot fallback and the instance setting taking precedence at reload. Fixes WIN-1966 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: hoist uv_python_install_mirror binding above sandboxing branch The non-sandboxed uv pip install branch referenced a binding that was only declared inside the sandboxed branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: neutral placeholder for uv_python_install_mirror The previous placeholder was the default public URL the setting is meant to redirect away from. A neutral example mirror URL is clearer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(indexer): tell admins when ingress routes search to wrong pod (#9274) * [ee] fix(indexer): tell admins when ingress routes search to wrong pod When the IndexReader is absent on the pod handling a search request but another pod is actively holding the indexer lock, the EE handler now returns a tailored error pointing at the ingress/load-balancer configuration instead of the generic "indexer not running" message. The indexer status endpoint reads the DB lock so it reports "running" from any pod, but search endpoints need the in-memory IndexReader that only exists on the lock holder. In multi-replica deployments this looks like the indexer is healthy but every search 404s. Companion: windmill-labs/windmill-ee-private#TBD Fixes WIN-1968. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817 This commit updates the EE repository reference after PR #586 was merged in windmill-ee-private. Previous ee-repo-ref: 7dd43d1850813071cc18ba49ba090583e7321f4b New ee-repo-ref: eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * feat(cli): add `wmill init prompts` and custom override slot (#9266) * feat(cli): add `wmill init prompts` and custom override slot Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): replace init prompts with refresh prompts + AGENTS.md/AGENTS.cli.md split Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): dedupe claude skills via @-includes and add prompts freshness check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): drop migration-choice flags from `refresh prompts` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(cli): add 'Running and previewing local changes' section to AGENTS.cli.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): write full skill content to .claude/, drop @-include wrapper Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): reconcile CLAUDE.md the same way as AGENTS.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): address PR review nits — argv parsing, lazy import, comment detection, error propagation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add yolo mode for ai chat tools (#9258) * feat: add yolo mode for ai chat tools * nit * fix: align chat footer controls * feat: add ai chat autonomy modes * feat: add autonomy mode dropdown * fix: highlight yolo autonomy icon * fix: auto accept flow edits * fix: hide unsupported autonomy modes * fix: handle auto-accept flow editor races * fix(debugger): add non-root user support to Dockerfile (#9277) Mirrors the main Windmill Dockerfile pattern: creates a windmill user (UID/GID 1000) and makes cache/work directories world-writable so the image runs cleanly under Kubernetes securityContext.runAsNonRoot or runAsUser: 1000 without permission errors on Bun, pip, or windmill cache writes. Fixes WIN-1969 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path (#9276) * fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path The AI proxy handler accepts an X-Resource-Path header to override the configured workspace AI provider. When supplied, the handler loaded the resource value from the resource table using the root DB pool with no resources:read scope check, so any authenticated workspace user could point X-Resource-Path at a restricted AI resource (e.g. one in a folder they cannot read) and the proxy would use that resource's provider credentials for the outbound AI request. For user-supplied resource paths, now require resources:read:{path} scope and fetch the resource through user_db.begin(&authed) so RLS enforces the same folder/group boundary as the resource API. The RLS- scoped $var: resolution stays in place as defense in depth. The admin-configured workspace/instance ai_config path is unchanged. Fixes WIN-1971 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(ai): regression test for X-Resource-Path RLS enforcement Cover all four cases: - non-admin pointing X-Resource-Path at a restricted resource is rejected - non-admin pointing it at a resource they own still works - admin can point it at any resource - workspace-configured proxy flow (no X-Resource-Path) is unchanged Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add userdraft listing primitives (#9268) * feat: add userdraft listing primitives * fix: cancel stale userdraft discard writes * docs: remove global ai userdraft plan * feat(nsjail): optional disk-backed /tmp via instance setting (#9272) * feat(nsjail): optional disk-backed /tmp via instance setting * test(nsjail): unit-test tmp mount resolver and narrow visibility * refactor(nsjail): switch tmp backing to select + conditional UI * ui(nsjail): make tmpfs the visible default in /tmp backing select * fix(nsjail): refuse preexisting jail_tmp to block symlink escape * fix(nsjail): allow jail_tmp reuse on sequential nsjail calls Codex flagged that python/ruby/rust executors invoke nsjail twice per job_dir (install then run). The previous resolver treated any preexisting jail_tmp as hostile and silently fell back to tmpfs on the second call, so disk-backed mode never reached the main script run for those langs. Use symlink_metadata().is_dir() to distinguish a real directory left by an earlier call in the same job_dir (safe to reuse) from a symlink or other entity (still refused, as the codebase-tar escape requires). Also loosen the frontend visibility predicate: only hide nsjail settings when job_isolation is explicitly 'none' or 'unshare', so deployments that enable nsjail via DISABLE_NSJAIL=false with no DB setting can still see the controls. * chore(main): release 1.706.0 (#9270) * chore(main): release 1.706.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix(nsjail): gate unix-symlink test behind cfg(unix) for Windows build (#9280) The disk_backed_refuses_preexisting_symlink_at_jail_tmp test calls std::os::unix::fs::symlink directly, which doesn't exist on Windows targets. Without a cfg gate, `cargo check --tests` fails on Windows with E0433. Other symlink call sites in this crate (php_executor, bun_executor, rust_executor, etc.) already follow this pattern. Fixes WIN-1972 Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Reduce slim image vulnerability surface (#9279) * Reduce slim image vulnerability surface * chore(docker): drop apt-get upgrade -y from slim images apt-get upgrade hurts build reproducibility (same Dockerfile + same commit at different times produces divergent images) and trips hadolint DL3005. The freshness it buys is dominated by simply rebuilding against the periodically-refreshed debian:bookworm-slim base image. The --no-install-recommends and apt-list cleanup wins are kept. --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> * fix(git-sync): bump to hub/28234 with stateless gpg.program wrapper (WIN-1974) (#9282) * fix(git-sync): revert LATEST_GIT_SYNC_SCRIPT_PATH to hub/28230 to restore GPG-signed deploys (WIN-1974) hub/28231 (PR #9230) is the "thin" script that hands the actual `git commit` to the CLI's hidden `sync git-deploy`. The hub script still does the GPG setup (import key into a fresh GNUPGHOME, dummy `gpg -bsau` to warm the agent passphrase cache, then `git config user.signingkey` + `commit.gpgsign` locally), but the commit no longer runs in the same `git_push` flow — it runs minutes later inside the CLI after workspace API resolution, zip pull, file extraction, and lockfile autofill. By the time the spawned `git commit` asks gpg-agent for the cached passphrase, the cache state is no longer reliable (or the spawned `gpg` ends up talking to a fresh agent), so signing fails non-interactively with `gpg failed to sign the data`. hub/28230 is hub/28217's in-script logic rebuilt with windmill-cli@1.703.3: the GPG setup and the in-script `sh_run("git commit ...")` happen back-to-back in `git_push`, so the cache is always fresh. It preserves wm_deploy / fork branch behavior, the EE deployment-callback `main()` signature is unchanged, and the only min-version check in EE (`is_script_meets_min_version(28103)`) is comfortably below 28230 — so this revert is safe. Forward fix (separate PR): publish a new thin script that, alongside the existing GPG setup, writes a `gpg.program` wrapper using `--pinentry-mode loopback --passphrase-file` so signing is independent of the agent's cache state. Re-bump past 28231 then. Fixes WIN-1974 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(git-sync): check in source-of-truth for the next hub script (gpg.program wrapper) This is the script that will be published to hub.windmill.dev once verified on a customer GPG-signed deploy. It replaces hub/28231's agent-cache pre-warm (`gpg -bsau` with --passphrase) with a stateless gpg.program wrapper + chmod-600 passphrase file. Every git-invoked gpg call goes through the wrapper, which always uses --pinentry-mode loopback (and --passphrase-file when a passphrase exists). Signing no longer depends on gpg-agent having a cached passphrase by the time the CLI's `git commit` runs — which closes WIN-1974. Not wired in yet: LATEST_GIT_SYNC_SCRIPT_PATH stays on hub/28230 until this script is uploaded and the new hub id is known. This file is checked in so the diff is reviewable, future bumps have a source of truth, and a CLI regression test can `cat` it for fixture parity. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): skip format/pattern validation for $var/$res/$jsonvar references in ArgInput A resource field with a `pattern` constraint (e.g. the gpg_key.private_key field, whose pattern enforces a `-----BEGIN PGP PRIVATE KEY BLOCK-----` prefix) rejects values like `$var:u/me/gpg-private-key` with an "invalid format" error in the resource editor — even though `$var:`/`$res:`/`$jsonvar:` are placeholders the backend resolves at runtime, not the actual string that needs to match the regex. Bail out of all format/pattern checks (email, ipv4, ipv6, uuid, custom pattern) when the value is one of these references. Required/numeric bounds/array checks still apply since they're shape-level, not regex. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(git-sync): bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28234 (gpg.program-wrapper fix) hub/28234 is the forward fix for WIN-1974: replaces hub/28231's agent-cache pre-warm (which became stale by the time the CLI's `git commit` ran) with a stateless `gpg.program` wrapper that uses `--pinentry-mode loopback` (and `--passphrase-file` when a passphrase exists) on every gpg invocation. Bundled CLI is windmill-cli@1.705.0. Verified via reproducer at /tmp/git-sync-diff/test-gpg-fix.sh: deliberately killing gpg-agent between GPG setup and `git commit` reproduces the customer's `gpg failed to sign the data` error verbatim under the old flow, and the wrapper signs through it. Holds for passphrase-protected keys, split-subkey [C]+[S] layouts, and unprotected keys. Drops the local source-of-truth copy (`hub-scripts/`) — hub is canonical now that 28234 is published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(git-sync): drop verbose comment above LATEST_GIT_SYNC_SCRIPT_PATH The git history (this PR) carries the why; the constant name + value carry the what. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): wmill sync git-deploy stops committing; caller owns commit+push (#9284) Single contract for the deployment-callback path: the CLI does branch checkout + pull, the caller (hub script in production, test in test) does git add + commit + push. This restores the WIN-1974 invariant — GPG setup and `git commit` run back-to-back in the same process, so the agent's pre-warmed passphrase cache is still warm at sign time — without needing a `--skip-commit` flag for the hub case and a default "also-commit" for everything else. Same behavior in every call site. Changes: - sync.ts: drop the gitSyncDeployPush call from pull()'s deploy path (both the onlyCreateBranch fast-return and the post-pull commit). `gitSyncDeployPush` stays exported for any caller that wants the same commit/push semantics — just not invoked by the CLI subcommand. - gitsync_promotion.test.ts: e2e test now does its own git add + commit + push after `wmill sync git-deploy`, mirroring what the hub script does in production. Same regression coverage (wm_deploy branch created in Case A, main untouched; main updated in Case B, no new wm_deploy). CLI typecheck unchanged (two pre-existing TarAsZip errors at lines 2578/3307, present before this PR). All 743 unit tests still pass. The accompanying hub script (option-C — CLI for branch+pull, script for commit+push) lives at /tmp/git-sync-diff/sync-script-to-git-repo-windmill.option-C.ts. Once published, a follow-up bumps LATEST_GIT_SYNC_SCRIPT_PATH to its id. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * bump git sync to 28236 * fix: fork compare visibility for non-admins and stale-token superadmins (#9283) * fix: use fork-scoped authed for fork visibility in compare_workspaces * test: add EE end-to-end repro for fork rename visibility * chore: restore concurrency_locks sqlx cache lost in cleanup * test: add regression for stale-superadmin-token fork visibility bug * chore: update sqlx cache for new test queries * chore(main): release 1.706.1 (#9281) * chore(main): release 1.706.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * feat: add wmill job rerun subcommand (#9275) * feat: add wmill job rerun subcommand * feat: add wmill job restart subcommand for flow restart-at-step * chore(system_prompts): point plugin skills sync at plugins/windmill/ (#9287) * chore(system_prompts): point plugin skills sync at plugins/windmill/ The plugin checkout's plugin folder is being renamed from `plugins/windmill-code-plugin/` to `plugins/windmill/` to shorten the slash-command namespace and align with the matching Cursor plugin layout. Paired with windmill-labs/windmill-claude-plugin#8. That PR must merge first so the next sync run finds the new folder. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(system_prompts): update plugin-dir example to plugins/windmill Co-authored-by: centdix <centdix@users.noreply.github.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: centdix <centdix@users.noreply.github.com> * fix(cli): wmill sync pull updates wmill-lock.yaml for raw apps (#9289) * fix: flow recording teardown crash + rename package to @windmill-labs/components (#9288) * fix: guard against null recording during FlowRecordingReplay teardown Navigating away from a flow recording inside a workspace file-tree view threw `TypeError: Cannot read properties of null (reading 'flow')` from FlowGraphViewer once during the teardown tick. Svelte 5 compiles child component props as live getters that close over `$$props.recording.flow`. When `recording` flips to null on the parent's navigation, an outer `{#if !recording?.flow}` doesn't stop those getters from firing one more time as derived effects re-evaluate before the unmount lands — so the getter dereferences null and throws. Fix at the two layers where the deref actually happens: - FlowRecordingReplay: use `recording?.flow` at the binding sites (FlowViewer + graph-snippet FlowGraphViewer) so the compiler emits an optional-chained getter, and guard the snippet branch with `{:else if recording?.flow}` so it doesn't mount when there's nothing to show. - FlowGraphViewer: finish the optional chaining the rest of the file already used everywhere else (`flow?.value?.skip_expr`, `flow?.value?.cache_ttl`, `flow?.schema`). When the upstream binding returns undefined during teardown, the graph degrades to an empty frame instead of crashing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: rename package to @windmill-labs/components - frontend/package.json: rename `windmill-components` → `@windmill-labs/components` - frontend/publish.sh: drop the in-place sed rename dance; the checked-in name now matches what's published, so `npm run package && npm publish` is enough - frontend/package-lock.json, system_prompts/auto-generated/prompts.d.ts: regenerated by `npm run package` under the new name Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * default script name * save logic * Keyboard nav * finish keynav * nits * CI fix * nit stop propagation * Merge branch 'main' into feat/asset-graph-view * commit * update * fix: cropped save button on small screens * progress * managed scheduled removed * all * progress * feat: add data upload pipeline trigger with auto S3 picker Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: avoid pane editor remount flicker when deploying a pipeline draft Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: show only the edited script's I/O in the asset graph, not the saved version's Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: derive script asset rows server-side at deploy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: shared fixture corpus keeps annotation parsers in parity Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: dev-run draft pipeline chains, live badges, deploy drift warning Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: ungate cascade producers, squash pipeline migrations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: drop committed cli-sync fixtures and stray screenshots Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: show skip-asset-dispatch flag as badge instead of args row Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: pipeline view mode default with activity feed, drafts overlay chip Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: treat DROP TABLE as table-level write in sql asset parser Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: wmill datatable create + actionable sql extension error Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: ephemeral data-pipelines demo sync repo zip for handoff Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: wmill pipeline list/show renders the asset DAG in the terminal Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nits * nits * nits * nits * fix: defer draft persist-back past the batch so discard sticks first click Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: band-reserving tidy-tree asset graph layout with join breakpoints Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: route skip-layer and long graph edges around occupied columns Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: seed s3 template outputs with canonical leading-slash paths Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * all * feat: bundle data-pipeline drafts into the DB-backed user draft system Pipeline drafts were browser-only (localStorage `pipeline-<folder>`), so they didn't sync across devices, weren't server-visible, and never showed in the drafts list. Store them instead as one per-user `draft` row of a new `data_pipeline` kind, keyed at the folder (`f/<folder>/data_pipeline`), holding the same `{ drafts, activeDraftPath }` bundle. Stage 1 — backend kind: add `data_pipeline` to DRAFT_KIND (migration) and `UserDraftItemKind` (deployed_table=None, private). The list/update handlers and folder-path access check already cover a backing-table-less kind. Stage 2 — sync: add `GET /drafts/get_own/{kind}/{path}` so an editor with no deployed-overlay GET can load its own draft. The pipeline page now hydrates from the DB on mount (one-time localStorage import for in-flight drafts) and persists via UserDraftDbSyncer (debounce + optimistic-concurrency), keeping a localStorage crash mirror. Stage 3 — surface: the drafts review page renders the bundle as a "pipeline" row that opens `/pipeline/<folder>` (open-only; excluded from bulk deploy). Verified end-to-end in-browser: DB-seeded draft hydrates to "Edit (1)", edits persist back, and the row shows with Open pipeline / Discard. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: pipeline Activity panel grouping, run↔graph highlight, deploy-conflict handling Activity panel (view mode): - Group cascade runs by the connected component of the asset-dispatch graph (new GET /jobs/asset_dispatch_edges over the dispatch_event table, incl. join_pending inputs), headed by the earliest originating run + its trigger, with a "+N" chip for joins fed by multiple triggers. - Success/failure count histogram with drag-to-filter brushing, an always-on time axis + per-bar tooltips, a Reset, and Last hour/24h/48h/7/30/90d ranges. - Node run-count/status badges now derive from the same merged historic+live events the panel shows (previously session-only). Run ↔ graph highlight: - Hovering a run row (or a group header → the whole cascade) rings the node(s), animates their incident edges, and borders the adjacent assets in the edge hue (blue write / gray read); expanding a run pins a soft-blue ring. - Switching edit→view re-surfaces the Activity feed. Deploy: - Live-content autosave for the open pipeline draft + an autosave indicator. - Re-saving a script now chains off the hash just created instead of a stale parent_hash (fixes the "lineage must be linear" error on a second save), and a genuine concurrent deploy opens a keep-mine / view-latest conflict modal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: pipeline editor badge requires asset-parse, not just main-function parse A pipeline script's asset lineage is load-bearing — a deploy that can't parse assets silently records no edges. The editor "parsable" dot only reflected inferArgs (the main function), so a body the asset parser rejects (e.g. a trailing `/////` in DuckDB) still showed green and deployed with empty lineage. ScriptEditor gains `requireValidAssets` (set by the pipeline pane); when on, the EditorBar badge is green only if BOTH the main function and inferAssets parse, with the tooltip distinguishing "Main function not parsable" / "Assets not parsable" / "Parsable". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: route asset-graph edges around nodes that sit in their path Edges could draw straight through an unrelated node (a join fan-out or long cross-component edge), making it ambiguous whether that node shared the input. AssetGraphEdge only saw its own endpoints, so it could only detour the near-vertical same-column skip case. The canvas now (once per layout, O(edges × nodes) — no per-frame cost) samples each edge's straight run against every non-incident node center and, on a crossing, passes a clear gutter lane to the edge via `data.detourX`; AssetGraphEdge routes the rounded-orthogonal detour through it. Verified: 0 edge↔node box crossings on the orders pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: deploy pipeline drafts with freshly-inferred assets, not a stale snapshot "Save all" spread `...draft.script` into createScript, which carries a `assets` snapshot that isn't refreshed when the body is edited. So a renamed/removed output (e.g. an old `CREATE TABLE exciting_en32z9` later changed to `exciting_880909`) was re-deployed as a phantom write edge and lingered as an orphan asset on the graph — shown with no producer, and shifting position on click as the graph re-derived. saveDraft now re-runs inferAssets on the current body and passes the result as `assets`, overriding the snapshot — mirroring the per-pane save. The backend clears+reinserts from the sent set, so a re-deploy drops the stale rows. Verified: deploying with the fresh asset set removes the orphan from the graph. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: collect upstream reads from CTAS and CREATE VIEW in SQL asset parser `CREATE TABLE x AS SELECT … FROM y` (and `CREATE VIEW`) recorded only the write to x — the source read of y was silently dropped. Table-level reads are gathered in the `Statement::Query` arm via handle_table_with_joins; the generic table-factor visitor only picks up read-functions and string literals, not plain `FROM <table>` references. The AS-query of a CTAS isn't a `Statement::Query`, so its FROM tables were never walked. On the pipeline canvas this meant a `datatable://…` upstream consumed by a CTAS step showed no read node/edge — the step looked like it produced its output from nothing. Factor the Query arm's read collection into handle_query_reads and call it from the CreateTable (when it has an AS-query) and CreateView arms, balancing the cte_name_stack push in post_visit_statement. Updated the drop_then_create test (which had pinned the old drop-the-read behavior) and added CTAS + CREATE VIEW read coverage. Verified against the rebuilt asset wasm: the live editor now infers the read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * update * updates * refactor: dedup asset-graph code, squash migrations, drop artifacts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf: gate asset dispatch on a cached per-workspace producer set Cache the producer-path→writes map per workspace and invalidate it from the asset-clear paths via the notify_event polling system, so a top-level script/preview completion that isn't an asset producer costs an in-memory lookup instead of a per-completion query. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: remove dead unquote fn that failed backend check under -D warnings Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: green the frontend check (pin published wasm-asset, fix type errors) Pin windmill-parser-wasm-asset to the published 1.728.1 (was a file: link to a gitignored, CI-unbuilt pkg-asset). Exclude test files from svelte-check (the parity test reads a backend fixture via node:fs, which the browser app tsconfig has no @types/node for; vitest still runs them). Fix pre-existing branch type errors: drop the unsupported 2nd getScriptByPath arg, cast script.schema to Schema for inferArgs, coerce has_preprocessor to a definite boolean, and wrap the cancelJob handler so it isn't possibly-undefined. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: move pipeline partition resolution to ee-private (free-CE) Partition resolution becomes a private module (partition_ee in windmill-ee-private, hidden from the public repo) with an OSS no-op fallback (partition_oss); call sites resolve via the aliased windmill_common::partition. Not enterprise-gated — free to run in CE. Bumps ee-repo-ref to the ee branch carrying partition_ee. Verified building in default, private, and private,enterprise (offline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: move asset-cascade join/debounce/retry to ee-private (free-CE) Join barrier, debounce, and retry become the private windmill_queue::cascade module (cascade_ee in windmill-ee-private); OSS gets cascade_oss no-op fallbacks (plain OR fan-out). Core cascade stays public. Bumps ee-repo-ref. Verified default/private/private,enterprise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: skeleton enterprise pipeline freshness + backfill (TODO, ee-private) Gated windmill_common::pipeline_advanced (private; pipeline_advanced_ee) with OSS fallback; entry points return a clear not-implemented error. Deploy surfaces a TODO when a script declares // freshness. Bumps ee-repo-ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: repair asset_trigger_dispatch test after cascade carve-out + cache its queries Stage-2 moved reap_stale_join_slots to windmill_queue::cascade; update the integration test's import. Also commit the test's sqlx query cache (was never prepared with --tests, so SQLX_OFFLINE cargo test failed pre-existing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: invalidate producer-cache in asset dispatch tests (mirror deploy) The tests seed asset rows directly and run no notify poller, so the per-workspace producer cache went stale across tests → 0 dispatched. Clear it at the seed point, as a deploy would via notify_event. All 8 asset_trigger_dispatch tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to ba677ea142011462ad4dfe77e8375a6dd274cdef This commit updates the EE repository reference after PR #619 was merged in windmill-ee-private. Previous ee-repo-ref: 925c350cff55d3ea738d9e2e4098d9ce4bdda418 New ee-repo-ref: ba677ea142011462ad4dfe77e8375a6dd274cdef Automated by sync-ee-ref workflow. * test: disable producer cache in asset dispatch tests (isolated-DB safe) The .remove(WS) approach still raced: #[sqlx::test] gives each test its own DB but they share one workspace id, so the WS-keyed process-global cache clobbered across DBs under concurrent threads. Add an ASSET_PRODUCER_CACHE_DISABLED test hook and set it in the tests so every dispatch reads its own DB. 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: replace asset-cascade depth cap with cycle detection The hardcoded MAX_CHAIN_DEPTH=5 truncated legitimate deep pipelines (silently — the check returned before event logging). Replace it with per-edge cycle detection: carry the producer lineage in trigger.chain and skip only a subscriber already in the chain, recording a visible cycle_detected dispatch_event. Acyclic pipelines of any depth now cascade fully; a high MAX_CHAIN_LEN backstop guards against runaway. Tests + UI label updated; 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update dispatch_event reason examples (depth_cap → cycle_detected) Comment-only; the migration is idempotent and already in the potentially_stale self-heal list, so the checksum change re-applies cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: park cascade retry (P1 dead-end) + clear stale script_triggers on rename Two deploy-path fixes: - Retry is parked: a retried subscriber is wrapped in a SingleStepFlow, whose run is a flow step and ineligible for asset dispatch, so it would silently dead-end the cascade (P1). Stop persisting retry to script_trigger and warn at deploy; TODO(pipeline-retry) to re-enable once dispatch handles flow-wrapped producers. (Dispatch plumbing kept + still tested via direct seeding.) - Rename leaves stale script_trigger rows: clear was keyed on ns.path only, so old-path '// on' edges lingered and could trigger a script later recreated at that path. Also clear the old path on rename (assets already handled via the parent-hash clear). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> Co-authored-by: hugocasa <hugo@casademont.ch> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Arnaud <31803803+Araden14@users.noreply.github.com> Co-authored-by: Diego Imbert <diego@windmill.dev> Co-authored-by: centdix <40307056+centdix@users.noreply.github.com> Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Aldrin Jenson <aldrinjenson@gmail.com> Co-authored-by: centdix <centdix@users.noreply.github.com> |
||
|
hugocasa
|
e80c62b958 |
docs(cli): improve generate-metadata guidance, fix description parser (#9635)
* docs(cli): improve generate-metadata guidance, fix description parser Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): surface dependency version bumps after generate-metadata Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(cli): explain generate-metadata scope, import cascade, and --dry-run troubleshooting Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
4e9e0c024b |
feat(cli): add --yes, --secret/--no-secret and --description to variable add (#9548)
* feat(cli): add --yes, --secret/--no-secret and --description to variable add Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(cli): cover variable add create/update flag semantics Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): warn on secret downgrade in variable add and pin preserve semantics in test Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
hugocasa
|
5bdc4f83ce |
feat(cli): improve agent prompts/skills and workspace fork workflow (#9531)
* feat(cli): improve agent prompts/skills and workspace fork workflow
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(cli): refuse fork --from-branch rename of a base branch
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(cli): auto-detect fork branch workflow, drop rt.d.ts refresh and legacy-name warning
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(skills): reconcile raw-app generate-metadata stance (agent offers+runs)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(skills): agent runs all CLI commands, gated on intent not on user typing them
Extends #9467's safe-vs-destructive model: the agent runs consequential commands (sync push, generate-metadata) itself too, gated on explicit user intent rather than handed to the user to type. The explicit-intent rule is the safeguard; an approval prompt is treated as a possible backstop, not assumed (auto-approve/headless runs have none).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Revert "docs(skills): agent runs all CLI commands, gated on intent not on user typing them"
Reverts
|
||
|
hugocasa
|
220cd35cf7 |
feat: support $f/ and $u/ import path aliases for scripts (#9378)
* feat: support $f/ and $u/ import path aliases for scripts $f/ and $u/ are local-friendly aliases for the absolute workspace import paths /f/ and /u/. Unlike the /-prefixed form (which local tools treat as a filesystem-root path), the $-prefixed form is a bare specifier that can be remapped via tsconfig paths / Deno import maps, so the same import resolves on the Windmill worker and in a local editor. - worker: recognize $f//$u/ in the Deno import map and both Bun loaders - dep-map/parser: normalize $f/->f/, $u/->u/ for lockgen + dep tracking - cli: emit $f/$u path aliases in generated tsconfig.json / deno.json - frontend: ATA + Monaco paths resolve $f//$u/ type hints in the editor Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): split generated tsconfig into managed + user file with refresh command Mirror the AGENTS.cli.md/AGENTS.md prompts model for the IDE tsconfig so the recommended settings can evolve without ever clobbering user customizations: - tsconfig.wmill.json: wmill-managed, always refreshed, holds recommended compilerOptions incl. the $f/$u path aliases (Deno: import_map.wmill.json) - tsconfig.json: user-owned, created once, just extends the managed file; warn (never auto-edit) when an existing one doesn't reference it - add 'wmill refresh tsconfig'; init generates it unconditionally (no longer gated behind resource-type namespace / a bound workspace) - regenerate CLI guidance docs for the new subcommand Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): address PR review on $f/ tsconfig generation - handle existing deno.jsonc so we don't shadow it with a new deno.json (P1 identified by cubic) - fix the bun-types hint that pointed users at the managed do-not-edit tsconfig.wmill.json; tell them to install + re-run 'wmill refresh tsconfig' - document the .ts-extension-only local-resolution limitation (cross-flavor .bun.ts/.deno.ts/.fetch.ts scripts won't resolve in a local editor) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): warn when a project's tsconfig isn't wired to tsconfig.wmill.json Mirror the prompts freshness check for the managed tsconfig so users with an existing setup actually discover they're missing $f//$u/ resolution: - embed a version hash in tsconfig.wmill.json (excludes the env-dependent bun-types 'types' entry so it doesn't false-positive) - add warnIfTsconfigStale to the main.ts freshness hook, gated identically to the prompts check (skips init/refresh/help/version). When a tsconfig.json exists it warns one line (stderr) if the managed file is missing, not referenced via extends, or out of date; silent for non-TS projects. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): make tsconfig setup equivalent to prompts (auto-wire + stale-only) Unify the two managed-file systems so they behave identically: - auto-wire an existing unlinked tsconfig.json/deno.json on init/refresh (add extends / importMap; merge into an array extends), instead of only warning. Parses JSON and falls back to a warning when it can't round-trip (JSONC comments, or a conflicting deno imports/importMap) — never corrupts. - narrow warnIfTsconfigStale to stale-only, gated on the managed file existing, exactly like warnIfPromptsStale: it no longer nags about a missing or unlinked tsconfig.json, so a deliberately-custom/unlinked setup stays silent and a not-yet-initialized project isn't bothered. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): place tsconfig.wmill.json first in extends to preserve user base config Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(cli): migrate legacy tsconfig and require consent for custom configs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(cli): align prompts wiring to the same consent model Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): bump windmill-parser-wasm-ts to 1.714.0 for $f/ $u/ aliases Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(worker): resolve $f/ and $u/ in deno lock generation Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: narrow relative-imports lock-gen guard to deno import-map failure Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): sync bun.lock with windmill-parser-wasm-ts 1.714.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): warn when a custom tsconfig's paths would shadow $f/ $u/ aliases Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
e356bb1f5d |
fix(cli): make encryption key push non-interactive-safe + add --skip-reencrypt-on-key-change (#9402)
When encryption_key.yaml changes and is pushed via `wmill sync push`, pushWorkspaceKey prompted interactively to confirm re-encrypting the remote secrets with the new key. That prompt ignored `--yes` and had no TTY guard, so a CI/non-interactive push that included the key would block (or behave undefinedly) on the prompt. Thread a key-push options object (non-interactive flag + explicit re-encryption choice) through pushObj into pushWorkspaceKey: - Non-interactive (`--yes` or no TTY) and no explicit choice: skip the prompt and default to re-encrypting all remote secrets with the new key (matches the interactive default), preserving their plaintext values. - New `--skip-reencrypt-on-key-change` flag (and the WMILL_NO_REENCRYPT_ON_KEY_CHANGE=true env var for CI) opt out of re-encryption — only safe when the remote ciphertexts are already encrypted with the new key (e.g. workspace/instance migration). - Interactive behavior (TTY, no `--yes`) is unchanged. Regenerates system_prompts for the new option and adds unit tests for the no-op, re-encrypt-by-default, flag-skip, and env-skip paths. Fixes WIN-2005 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
979b086b08 | refactor(cli): fold flow test-step into flow preview --step (#9330) | ||
|
hugocasa
|
36f574ff95 |
feat(cli): add object-storage commands and flow test-step (#9326)
* feat(cli): add object-storage commands and flow test-step * docs(cli): clarify flow test-step doesn't recurse into aiagent tools * fix(cli): correct failure step id in docs, handle bare flow.yaml path |
||
|
hugocasa
|
e0ffea2deb |
feat: add wmill job rerun subcommand (#9275)
* feat: add wmill job rerun subcommand * feat: add wmill job restart subcommand for flow restart-at-step |
||
|
hugocasa
|
1ba8ed8abd |
feat(cli): add wmill init prompts and custom override slot (#9266)
* feat(cli): add `wmill init prompts` and custom override slot Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): replace init prompts with refresh prompts + AGENTS.md/AGENTS.cli.md split Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): dedupe claude skills via @-includes and add prompts freshness check Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(cli): drop migration-choice flags from `refresh prompts` Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(cli): add 'Running and previewing local changes' section to AGENTS.cli.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): write full skill content to .claude/, drop @-include wrapper Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): reconcile CLAUDE.md the same way as AGENTS.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(cli): address PR review nits — argv parsing, lazy import, comment detection, error propagation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
28c8b5c60f |
feat: CLI datatable serve / psql (#9267)
* feat(cli): add datatable list and run commands * feat(cli): render datatable query results as a table * feat(cli): serve datatables as a postgres-wire endpoint * feat(cli): add 'datatable psql' to launch psql against the proxy * feat(cli): route datatable serve by client-supplied database name * override database list + password option * fix: support extended queries in datatable serve * fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma * refactor: extract raw_output envelope encoding into pg_raw_output module --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> |
||
|
Diego Imbert
|
1d04904a47 |
feat(cli): add datatable and ducklake list/run commands (#9257)
* feat(cli): add datatable list and run commands * feat(cli): render datatable query results as a table * feat(cli): add ducklake list/run, scope --name to run subcommand |
||
|
Ruben Fiszel
|
01bad16c0c |
feat: add wmill protection-rules pull/push CLI commands (#9240)
* feat: add wmill protection-rules pull/push CLI commands Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: use directional keys for protection-rules pull --json diff Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review — exit non-zero on failure, resolve override workspace key - failure paths in pull/push now exit 1 so CI/scripts detect failed reconciles - --override writes under the resolved workspace key (findWorkspaceByGitBranch), not the raw branch, so gitBranch-mapped entries aren't left inert - pull --replace clears a shadowing protectionRules override so top-level takes effect (was an infinite pull --diff loop) - push reports applied create/update/delete counts on partial failure and warns loudly when an empty list would wipe all backend rules Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review — dry-run pull --diff no longer writes; --promotion coherent - pull --diff returns before the no-wmill.yaml bootstrap, so a dry run never creates/mutates wmill.yaml - pull --promotion now writes/clears the promotion target's promotionOverrides (the same block getEffectiveSettings reads), instead of the current branch's regular overrides — read and write are now coherent Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: move protection rules to a per-workspace protection-rules.yaml Replaces the wmill.yaml/SyncOptions integration (top-level + overrides + promotionOverrides) with a dedicated protection-rules.yaml keyed by workspace name. This removes the getEffectiveSettings layering that caused the override shadowing / promotion-coherence / dry-run bugs entirely. - protection-rules.yaml: { <workspace>: ProtectionRuleEntry[] }, keys must match wmill.yaml 'workspaces' (source of truth for backend id/baseUrl/token) - commands reduced to: pull/push [workspace] | --all, with --dry-run - per-workspace auth resolved via tryResolveBranchWorkspace + setClient - push remains a full reconcile (create/update/delete) with delete confirm, empty-list wipe warning, partial-failure reporting, non-zero exit on failure - conf.ts reverted to main; SyncOptions no longer carries protectionRules Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review — honor explicit --base-url/--token in protection-rules configureClientForWorkspace bypassed the credential precedence other commands use: explicit --base-url/--token now work for stateless CI (no stored profile or wmill.yaml baseUrl needed), and an explicit --token overrides a stored profile's token. The backend workspace id still derives from the wmill.yaml mapping (feature invariant). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address cubic review — consistent status on partial --all failure cubic found that pull/push reported success:true while exiting non-zero on partial --all failures, and that the push command description was missing from the generated CLI docs. - pull/push now report success:false + partialFailure:true (and exit 1) when any --all workspace fails; success:true only on full success - .description() calls use single string literals (not + concatenation) so system_prompts/generate.py parses them; regenerated CLI docs Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: address review — --json-output must emit only JSON on stdout Codex flagged that workspace resolution (tryResolveBranchWorkspace's log.info) and push's empty-list delete warning print to stdout before the JSON payload, breaking machine callers. Silence human logs via log.setSilent(true) as the first action when --json-output is set (before readConfigFile / resolution); log.error still goes to stderr. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
43b18006f3 |
feat(cli): auto-infer args for wmill app push (#9091)
Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`) with no args. The local path defaults to CWD, and the remote path is derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/ `__app`/`__raw_app` suffixes stripped. Either, both, or neither positional argument can be passed. Also resolves `file_path` against the user's original CWD before `resolveWorkspace` may chdir to the wmill.yaml root, so a relative `file_path` argument is interpreted from where the user invoked the command (previously it could resolve against the wrong directory). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
bc527fd929 |
feat(cli): add --parallel flag to generate-metadata (#9074)
* feat(cli): add --parallel flag to generate-metadata * fix(cli): validate --parallel input and harden flush ordering |
||
|
Ruben Fiszel
|
66c9063919 |
feat(cli): add --as-superadmin flag to workspace list-remote (#9043)
* feat(cli): add --as-superadmin flag to workspace list-remote Wires the existing /workspaces/list_as_superadmin endpoint into the CLI so superadmins can enumerate every workspace on an instance from CI tooling, not just the ones the calling user is a member of. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate CLI prompts and document the regen step Adds an AGENTS.md note pointing to system_prompts/generate.py so future CLI command edits keep the agent-facing CLI docs in sync, and regenerates the auto-generated outputs for the new --as-superadmin flag. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
d60dd745e4 |
feat(forks): handle triggers and schedules in workspace forks (#8976)
* feat(forks): strip operational state from triggers/schedules on git-sync export When the source workspace is a fork (`wm-fork-*`), the tarball export now omits `mode` from triggers and `enabled` from schedules. The trigger update handler also preserves the existing DB `mode` when both fields are absent from the request, instead of falling back to the BaseTriggerData default. This prevents a fork's git-sync round-trip from flipping the parent workspace's enabled/disabled state when a merge applies the fork's YAML back to main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): opt-in fork_triggers flag clones triggers/schedules disabled Adds `workspace.fork_triggers` (default false) and a matching field on CreateWorkspaceFork. When the user opts in, fork creation also runs clone_triggers_and_schedules: every row in schedule and the ten *_trigger tables is copied to the fork with mode='disabled' / enabled=false. Listener identifiers (group_id, replication_slot_name, subscription_name, …) are copied verbatim — the runtime suffix that prevents the fork from competing with the parent ships in a follow-up PR. native_trigger is intentionally skipped: those triggers manage external webhook state we don't want duplicated. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): warn before enabling triggers/schedules that conflict with parent set_trigger_mode and schedule's set_enabled now check whether the parent workspace has the same path actively enabled. If so, the call is rejected with a `fork-conflict:<kind>:<parent_id>` error unless the request includes `force=true`. The frontend interprets the prefix to surface a confirm-to- proceed dialog. This is the placeholder safety net until the Phase 3 listener-suffix work removes the conflict for the namespaceable kinds (Kafka/MQTT/NATS/Postgres/ Azure/GCP-CreateNew). For SQS, GCP-Existing, and schedules — where there's no namespacing fix — the warning is the durable solution. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): UI: opt-in clone-triggers checkbox + confirm-on-fork-conflict Adds the user-facing surface for the fork-trigger work: - CreateWorkspaceInner: new "Clone triggers and schedules" toggle in the fork-creation dialog (default off). Sends fork_triggers in the request. - forkConflict utility: detects the `fork-conflict:<kind>:<parent_id>` error string from the backend, shows a confirm() dialog explaining why the action is blocked, retries with `force: true` if accepted. - Wires withForkConflictRetry into every trigger setMode and the schedule setEnabled call, both in the per-kind editor components and the +page.svelte list views (HTTP, websocket, kafka, NATS, SQS, MQTT, GCP, Azure, Postgres, email, schedule). OpenAPI spec gains the `force` field on each setmode/setenabled body. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(forks): CLI --fork-triggers flag, fork-trigger docs, skill update - Adds --fork-triggers boolean to wmill workspace fork; passes fork_triggers through to the create_fork API call. - New docs/fork-triggers.md describing the model end-to-end (default, opt-in clone, merge-direction filter, conflict warning, future runtime-suffix work). - Updates the adding-a-trigger SKILL.md to mention the fork-export ignore-keys participation and the clone_triggers_and_schedules block that new trigger kinds must extend. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx offline query cache for fork-trigger SQL * fix(forks): replace browser confirm() with ConfirmationModal for fork conflict The fork-conflict warning previously used the browser's native confirm() which doesn't match Windmill's design system. Switches to a singleton ConfirmationModal mounted at the (logged) layout root, driven by a new forkConflictModal store. The withForkConflictRetry helper now sets the store and awaits the user's choice via a Promise, instead of blocking on window.confirm. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): filter unchanged triggers in merge UI, add diff view, surface parent-only ones The fork merge UI listed every trigger from the fork as a deployable item regardless of whether it differed from the parent — so a fork created with fork_triggers=true (which clones triggers in disabled state, otherwise identical) showed every trigger as a "Fork-only" change. The 'Update current' tab also missed triggers newly created in the parent that the fork hadn't pulled yet. This refactor: - fetchAllTriggers now lists both fork and parent in parallel for each trigger kind, then merges by path. - Computes a per-trigger `changeKind` (new / modified / deleted-in-source) using a JSON comparison that strips runtime + fork-local fields (mode/enabled/server_id/last_server_ping/edited_at/edited_by/etc.) so the disabled-on-clone difference doesn't show up as a change. - Filters the trigger items in deployableItems by the current direction: Deploy mode shows fork-side new/modified, Update mode shows parent-side new/modified. - Replaces the always-on "Fork-only" badge with proper New/Modified badges and surfaces a Diff button (modal Drawer + Monaco DiffEditor) for modified triggers — the diff strips the same ignored fields so users see only the meaningful config differences. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks): always clone triggers/schedules disabled, drop opt-in flag Disabled triggers and schedules are inert — no listener attaches, no cron fires — so cloning them by default is safe by construction. Drops the fork_triggers opt-in flag introduced earlier in this PR: - Drops workspace.fork_triggers column (migration removed) - Removes fork_triggers from CreateWorkspaceFork (API + OpenAPI) - Removes the conditional in create_workspace_fork — clone always runs - Removes the toggle from the fork-creation dialog - Removes --fork-triggers from `wmill workspace fork` - Updates docs/fork-triggers.md and adding-a-trigger SKILL.md The merge UI continues to exclude triggers from the deploy/update default selection, so a routine merge from a fork doesn't accidentally push trigger config the user hasn't intentionally changed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(http-triggers): scope route exists check by workspace, skip non-workspaced clones in forks The non-CLOUD branch of `route_path_key_exists` self-excluded by trigger path alone, which silently masked cross-workspace collisions once forks started cloning trigger rows verbatim. Tighten it to exclude only the exact `(workspace_id, path)` row. Fork creation also now skips non-workspaced HTTP triggers — their URL has no workspace prefix, so a clone collides with the parent at the matchit router (which silently drops one of two duplicates) and there is no namespacing escape hatch. The clone copies all rows when CLOUD_HOSTED or HTTP_ROUTE_WORKSPACED_ROUTE forces every route workspaced regardless. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(forks-ui): silent cancel on enable conflict, clean up trigger rows in compare view forkConflict helper now returns undefined when the user dismisses the modal instead of throwing, so the redundant 'Cannot enable: undefined' toast no longer appears. CompareWorkspaces trigger rows now mirror the script row layout: drop the redundant Disabled badge and the Trash/Details buttons (both belong on the dedicated trigger pages, not in the deploy/compare view); pass triggerKind through so RowIcon picks the right kind-specific icon; move extraLabel into the summary line; replace the yellow Modified badge with the same green ↗ ahead / blue ↘ behind treatment scripts use. Trigger diff drawer: switch JSON → YAML for parity with DiffDrawer, fix zero-height monaco render with className=!h-full, drop the redundant Original/Modified label banner above the diff. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(email-trigger): scope local_part exists check, skip non-workspaced clones in forks Mirrors the HTTP route fix for the email-trigger non-CLOUD `email_exists` check (in EE) which had the same path-only self-exclusion bug, and the fork clone of `email_trigger` rows which copied non-workspaced `local_part` verbatim. Skip non-workspaced rows in the clone unless the instance is CLOUD_HOSTED (where lookup is workspace-scoped natively). EE companion change in windmill-trigger-email/src/handler_ee.rs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 78512dd73b4a1c9f70574cff863374179e3a621b This commit updates the EE repository reference after PR #554 was merged in windmill-ee-private. Previous ee-repo-ref: 1ac77f50747b58e720a11162dfd309bc252a24ab New ee-repo-ref: 78512dd73b4a1c9f70574cff863374179e3a621b Automated by sync-ee-ref workflow. * fix(forks): always-warn on parent row, kind-specific modal copy, cancel-aware toggles - Conflict check now fires whenever the parent has the path (regardless of parent's mode), since the cloned upstream identifier is shared by construction; closes the Postgres slot-takeover gap when the parent is disabled. Schedule's set_schedule_enabled gets the same treatment. - Skip the warning entirely for HTTP and Email via a new TriggerCrud::FORK_CONFLICT_ON_ENABLE const — both kinds are workspace- scoped at runtime so cloned rows can't collide with the parent. - Modal copy branches by failure family: split-events (Kafka/NATS/MQTT/SQS/ GCP/Azure), duplicate-firing (Websocket/Schedule), slot-takeover (Postgres). Generic fallback for unknown kinds. - withForkConflictRetry now returns boolean (true=committed, false= cancelled). TriggerModeToggle reuses its existing innerTriggerMode local state via a function binding for the regular Toggle, snapping back to the prop when onToggleMode signals a cancel — needed because the native bind:checked diverges from the parent's prop after a click and Svelte's reactivity won't re-push a same-valued prop down. Schedule list page uses {#key} on a reset version since it renders Toggle directly. - Editor inners revert mode = previousMode on cancel; list pages skip the re-fetch (loadTriggers/loadSchedules) on cancel to avoid pointless network traffic and the schedule "Job stats loading..." flash. - Drop withForkConflictRetry from HTTP and Email editors + list pages since the backend never emits the conflict for those kinds. * fix(forks-ui): widen onToggleMode types, scope schedule toggle reset by path - TriggerEditorToolbar and TriggerSuspendedJobsModal forwarded onToggleMode as `(mode) => void`, dropping the new boolean return so any caller wired through them would silently no-op the cancel-revert. Match the wider TriggerModeToggle signature. - Schedule list page used a single resetVersion counter for every row's {#key}, so cancelling on any one schedule remounted every <Toggle> on the page. Switch to a per-path Record<string, number> bumped only for the affected row. * chore: bump ee-repo-ref to c3a4553 (email FORK_CONFLICT_ON_ENABLE override) * fix(forks): include Suspended in conflict gate, use parent_workspace_id for fork detection Three fixes from the Claude review on PR #8976: - Suspended mode still attaches the listener (it just pauses auto-run of queued jobs); two suspended fork+parent listeners would still split Kafka events / share a PG slot. Gate set_trigger_mode on `mode != Disabled` instead of `mode == Enabled` so Suspended also surfaces the warning. - workspaces_export.rs::fork_*_ignore_keys keyed off the wm-fork-* prefix while set_trigger_mode and set_schedule_enabled key off parent_workspace_id. Switch the export filter to query parent_workspace_id once at the top of tarball_workspace and pass is_fork through. The column is the contract; the prefix is a creation-time naming convention that could in principle drift. - TriggerModeToggle's suspend-dropdown action reassigned the non-bindable `triggerMode` prop instead of the local `innerTriggerMode` mirror, leaking inconsistent state if the dispatch was cancelled. Now writes to innerTriggerMode like the Toggle's on:change handler does. * fix(cli): skip setScheduleEnabled when local YAML lacks `enabled` Tarball export from a fork strips `enabled` from schedules so the fork→parent git-sync round-trip can't flip the parent's operational state. The CLI's pushSchedule called setScheduleEnabled whenever `localSchedule.enabled != schedule.enabled`, which evaluates truthy when local is undefined (fork-pulled YAML) and remote is true/false — sending `{ enabled: undefined }` that serializes to `{}` and gets rejected by the backend (`SetEnabled.enabled` is required). Skip the call when `localSchedule.enabled === undefined` so a sync push of fork-pulled YAMLs preserves the target's existing enabled state instead of erroring out. Trigger updates were already safe — the backend's update_trigger preserves `mode` when the request omits it. * Revert "fix(cli): skip setScheduleEnabled when local YAML lacks `enabled`" This reverts commit |
||
|
hugocasa
|
0b959b8ec6 |
feat(cli): wmill-lock.yaml auto-fill + --rehash-only + path-prefix dedup (#8978)
* fix(cli): canonical lockfile hashes + lock upgrade migration to v3 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): use __app_hash subpath in rehash missing-entry check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): run sync pull lockfile auto-fill regardless of changes Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: regenerate system prompts for new lock and rehash-only commands Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on lock upgrade Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop v3 marker; always run fallback; fail-fast on unknown lockfile version Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): drop yaml-round-trip legacy hash variant; recover via --rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): include legacy hash in script push staleness warning check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * revert(cli): drop canonical hash formula; keep raw-bytes hashing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * perf(cli): reuse change-tracker map for sync pull lockfile auto-fill Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): pin lockfile hash + yaml format and cover regression cases Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): byte-stable snapshot tests for flow.yaml format Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test(cli): add app and script-metadata yaml snapshot fixtures Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude review on rehash-only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): factorize script-path to remote-path derivation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address claude + cubic review (dry-run mutation, rehash short-circuit) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): make rehash a subcommand and factorize fs walks Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): normalize line endings in yaml snapshot tests for windows ci Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address review feedback on rehash + auto-fill - Flat-layout scripts now clearGlobalLock before rehash write so legacy ./-prefixed duplicates get cleaned up (matches flow/app behavior). - Add MalformedLockfileError; sync pull auto-fill re-throws it alongside UnknownLockVersionError instead of silently warning + continuing. - Document the legacy step-removal false-negative in isFlowDirectlyStale / isAppDirectlyStale and the categorizeLocalFiles ignore-filter invariant. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
 Guilhem
|
eebe24d8b0 |
feat(cli): wmill dev with per-flow proxy and responsive Dev UI (#8529)
* feat(cli): add `wmill flow dev` subcommand with per-flow reverse proxy and launch.json Also generates .claude/launch.json for existing flow folders during `wmill init`. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: responsive dev layout and hide splitter for single-pane views Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: clamp flow graph height between minHeight and maxHeight Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(cli): enhance app new with Claude Desktop integration and better defaults - Add .claude/launch.json to generated app scaffold for Claude Code preview support - Add "Open in Claude Desktop?" prompt that creates a CLI session and opens it in Claude Desktop Code mode via the claude://resume deep link - Improve default CSS template with body background, system fonts, and padding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): handle both .flow and __flow suffixes in wmill dev The flow detection in loadPaths only checked the configured suffix (dotted or non-dotted), so users with nonDottedPaths=true who had .flow folders (or vice versa) would see inline script edits treated as standalone script changes instead of flow changes. Now checks both suffix forms everywhere: type classification, folder path extraction, path stripping, and loadWmPath lookup. Also adds raw_app launch.json generation to init and sync pull. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(cli): update generated skills with dev workflow and preview commands Update cli-commands, write-flow, and raw-app skills to document the new local dev workflow (wmill dev --path, --proxy-port, .claude/launch.json). Add wmill script preview and wmill flow preview to all script/flow skills so agents know how to test without deploying. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): include path in dev URL and use open.default for browser - Append &path= to the printed/opened URL when --path is specified - Use open.default(url) instead of open.openApp for more reliable browser opening Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): add Claude CLI/Desktop detection hints in wmill flow new Show contextual instructions for previewing flows based on available tools. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate auto-generated CLI skills for new dev flags Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): handle mixed flow suffixes in dev file watcher The ignore() function uses isFlowPath() which only checks the configured suffix (__flow or .flow), causing files in the other variant to be silently ignored. Bypass the ignore check for any file inside a flow folder and force flow type detection regardless of suffix configuration. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): drop default proxy in flow folders, open browser, add --no-browser Manual `wmill dev` in a flow folder should not implicitly enable the reverse proxy. Both proxy and legacy modes now open the browser; the new --no-browser flag opts out. Claude Code launch.json templates pass --no-browser so the IDE preview doesn't fight a system browser window. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(cli): gate dev broadcasts by --path and push currentLastEdit on connect When --path (or auto-detected flow path) is set, drop file events for any other path so the dev page stays locked to the requested resource and currentLastEdit can never reflect an unrelated edit. The connection handler proactively pushes currentLastEdit so the page renders without waiting for the first file change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(frontend): prefer WebSocket for flow round-trip when wmill dev is connected updateFlow used isInIframe priority, which routed Claude Code's iframe preview through postMessage (no listener) and silently dropped flow edits. Flip the priority: when the wmill dev WebSocket is open, use it (covers standalone tabs and Claude Code's preview); fall back to postMessage only when no WS is connected (the VS Code extension's iframe URL has no `local=true`, so it never opens one). Also stop assigning lastSent before a channel actually accepted the message, so a CONNECTING WS doesn't silently swallow the first change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(dev): url is source of truth for path; add workspace file picker Drops the server-side --path gate added in |
||
 Alexander Petric
|
95d4c6a94d |
feat(cli): non-interactive Slack connect/disconnect + sync round-trip fixes (#8935)
* feat(cli): non-interactive Slack connect/disconnect
Extract create_slack_workspace_artifacts / create_slack_instance_artifacts
from the browser OAuth callbacks and expose them via two new endpoints that
accept a pre-minted xoxb bot token:
- POST /w/{workspace}/workspaces/connect_slack (admin)
- POST /oauth/connect_slack_instance (super-admin)
Both produce bit-for-bit identical DB state to the UI browser flow.
Wire three CLI commands as thin wrappers:
- wmill workspace connect-slack
- wmill workspace disconnect-slack
- wmill instance connect-slack
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): round-trip stability for workspace settings handlers
wmill sync push was destroying UI-configured error_handler/success_handler
state on every deploy. Two orthogonal bugs:
(a) pushWorkspaceSettings called editErrorHandler with `path: undefined`
when the YAML lacked the handler block, which the backend treats as a
clear — so syncing settings.yaml that didn't mention the handler wiped
the DB row. Fix: skip the call entirely when absent from YAML.
(b) edit_error_handler omitted muted_on_cancel / muted_on_user_path when
false, but the CLI always sends them, causing perpetual deepEqual
drift and a spurious editErrorHandler call on every sync push. Fix:
always persist both booleans.
migrateToGroupedFormat now preserves explicit `null` on
error_handler / success_handler as a "clear remote" signal distinct from
absence. Widen ErrorHandlerConfig | null / SuccessHandlerConfig | null to
make this explicit in the type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cli): sync support for workspace-level Slack OAuth override
Add slack_oauth_client_id and slack_oauth_client_secret to the v2 tarball
export and to pushWorkspaceSettings, so the workspace-level OAuth override
is now fully managed as code through settings.yaml.
Semantics:
- both defined and truthy → setWorkspaceSlackOauthConfig (upsert)
- both defined but falsy (e.g. empty strings) and remote has a value
→ deleteWorkspaceSlackOauthConfig
- either omitted → leave remote alone ("not managed by git")
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(cli): normalize workspace settings sync to "omit = clear"
Earlier commits on this branch introduced an "omit = keep" rule for
error_handler / success_handler / slack_oauth_client_{id,secret} that
diverged from every other workspace setting (webhook, deploy_to, etc. all
treat YAML as canonical: absence = clear). Normalize:
- v2 tarball always emits these 4 fields (null when remote is NULL) so
round-trip is bijective and settings.yaml is a complete snapshot.
- pushWorkspaceSettings drops the absent-from-YAML guards; YAML is
canonical. Absence and explicit null both clear the remote — same rule
as every other field.
- set_slack_oauth_config / delete_slack_oauth_config now fire
handle_deployment_metadata so UI mutations reach git-sync-enabled
workspaces' committed settings.yaml.
Policy for users: pull before push (same as every other setting). On first
post-upgrade pull, explicit `null` keys appear for any workspace whose
handlers / oauth override are unset — one-time YAML diff, no semantic
change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(cli): add unit + integration coverage for Slack settings sync
Unit tests (settings_unit.test.ts): cover migrateToGroupedFormat preserving
explicit `null` on error_handler / success_handler, and passthrough of
slack_oauth_client_id / _secret (both populated and null values).
Integration tests (slack_settings_sync.test.ts, skipped on CI per the same
convention as datatable_settings_sync.test.ts): exercise the full backend
via withTestBackend to verify
1. pull emits null for unset error_handler / success_handler /
slack_oauth_client_id / _secret;
2. round-trip with all-null handlers is idempotent;
3. push of populated slack_oauth_config upserts;
4. omitting the slack_oauth keys from YAML clears remote (universal
"omit = clear" rule);
5. explicit null error_handler in YAML clears remote;
6. round-trip preserves a populated error_handler exactly, including the
always-persisted muted_on_cancel / muted_on_user_path booleans.
Also feature-gates `use crate::oauth2_oss::workspace_connect_slack` and its
route registration behind `cfg(feature = "oauth2")`: the import caused a
build failure on subsets of the workspace without the oauth2 feature,
surfaced by the integration test harness.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump ee-repo-ref to 59b6123
Pins windmill-ee-private to the tip of branch alp/slack_cli, which
contains the companion EE changes (helper extraction, non-interactive
Slack connect handlers, git-sync for Slack settings mutations).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Update SQLx metadata
* chore: regenerate system prompts for new slack CLI commands
Captures the new workspace connect-slack, workspace disconnect-slack,
and instance connect-slack commands in the auto-generated files that
CI enforces via system_prompts/check-freshness.sh.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to b4a5ca11e3b96ff03793c2bd396dbc1fe6ea1022
This commit updates the EE repository reference after PR #550 was merged in windmill-ee-private.
Previous ee-repo-ref: d7e44d0519327ec9077625130365e887826f324b
New ee-repo-ref: b4a5ca11e3b96ff03793c2bd396dbc1fe6ea1022
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
|
||
|
hugocasa
|
d6c642b170 |
feat: add Azure Event Grid triggers (#8888)
* feat: add Azure Event Grid triggers (EE)
Introduces a new enterprise trigger kind `azure` that supports three
modes via a single unified trigger type:
- basic_push: Azure Event Grid basic — custom topics, system topics
(Storage, Resource Manager, Key Vault, etc.), domains (push only)
- namespace_push: Event Grid Namespace topics (CloudEvents over HTTP push)
- namespace_pull: Event Grid Namespace topics (HTTP pull with lock-token
ack/reject for dead-lettering)
Auth uses a Service Principal resource (tenant_id, client_id,
client_secret, subscription_id). Subscriptions are created in
CloudEvents 1.0 schema so the push webhook handler and the pull listener
share one payload parser.
Backend
- New crate `windmill-trigger-azure` (OSS stubs + EE impl symlinked from
windmill-ee-private)
- Migration `azure_trigger` table with CHECK constraints enforcing
mode/columns coherence
- `TriggerKind::Azure`, `JobTriggerKind::Azure`,
`DeployedObject::AzureTrigger` variants
- Push route `/api/azure/w/{workspace}/*path` handles classic
Event Grid SubscriptionValidation handshake and CloudEvents 1.0
abuse-protection OPTIONS handshake
- Optional inbound JWT validation (audience check only for v1)
- Feature flag `azure_trigger` propagated through windmill-api,
windmill-store (resource helper), and added to ee_core
Frontend
- `triggers/azure/` editor with mode toggle (basic/namespace-push/
namespace-pull) and per-mode config (topic ARM id / namespace +
topic name / subscription / filters / push auth / pull options)
- Registered in icon map, display names, save functions, badge,
wrapper, editor, add-trigger menu
OpenAPI
- `AzureTrigger`, `AzureTriggerData`, `AzureMode`,
`AzureSubscriptionMode`, `AzureDeliveryConfig`, `TestAzureConnection`
schemas; `/azure_triggers/*` endpoints; client regenerated
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
This commit updates the EE repository reference after PR #541 was merged in windmill-ee-private.
Previous ee-repo-ref: 9689014e8c12c36c1059fd8fa5758d550b8b8bc9
New ee-repo-ref: eaa7c3a9cb37a9ccc93f10a2535d929365acd2d8
Automated by sync-ee-ref workflow.
* feat(azure-trigger): secret-auth push, ARM discovery, capture isolation, CLI + parity
Frontend:
- Split mode selector into Namespace/Basic + Pull/Push
- ARM resource dropdowns (namespaces, Basic topics, namespace topics)
populated from the service principal; cascade with stale-selection
reset on SP / edition change
- Remove stale authenticate toggle + audience input (server-managed
push_auth_config has replaced them)
- Azure listing page: "Create from template" button; "Also delete Azure
subscription" toggle in the delete modal; simplified trigger label
falling back to path
- AzureCapture.svelte: "Test subscription name" with -wm-capture suffix
- CompareWorkspaces.svelte: wire Azure for fork/compare
- Drop Trigger-deployed/event-loss warning (capture subscription is
isolated with -wm-capture)
Backend:
- Shared-secret push auth (see EE crate for detail)
- JSONB push_auth_config column (renamed from delivery_config), #[serde(skip)]
so clients/CLI/exports never see it
- Drop redundant enabled column; mode supersedes
- Azure capture infra: AzureTriggerConfig + set_azure_trigger_config +
azure_payload route + TriggerKind::Azure arm; PT15M queue TTL on
capture subscriptions so they bound storage after tab close
- Granular ACLs, users offboarding, trash, git-sync deployed-object:
all include azure_trigger
CLI:
- Add azure to TRIGGER_TYPES, pushObj dispatch, getTypeStrFromPath,
trigger commands (get/update/create/list/template), sync delete
switch + regex; e2e test for `trigger new --kind azure`
- system_prompts: SCHEMA_MAPPINGS + schema_names include AzureTrigger;
auto-generated/* regenerated
Skill:
- .claude/skills/adding-a-trigger/ checklist covering every file that
needs editing when wiring a new trigger type (learned from this PR)
ee-repo-ref bumped to b0e490cbf3724b7b64c6a5b010e3bdf24acd873c.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): ci — ShareModal Kind + regenerated system_prompts
- frontend/src/lib/components/ShareModal.svelte: add 'azure_trigger'
to the Kind type so the listing page's "Permissions" action compiles
(ts2345 — caught by npm_check on CI, missed by fast-check locally).
- system_prompts/auto-generated/: regenerate to drop the stale
delivery_config / AzureDeliveryConfig fields from the Azure schema
(check-freshness on CI).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(azure-trigger): use workspace constant_time_eq crate
Drop hand-rolled constant-time compare in favour of the workspace
constant_time_eq crate (same one used by http_trigger_auth).
ee-repo-ref bumped to 9659382d47286e7f7f66d01b6f5dd8d4ed34848b.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): pass placeholder + disabled via inputProps
`TextInput`'s `placeholder` and `disabled` go through its `inputProps`
prop — CI's `npm run check` caught the stale top-level passing that
`npm run check:fast` missed. Align with the DefaultEmailConfigSection
pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): correct LATEST_GIT_SYNC_SCRIPT_PATH version to 28213
The hub deploy of the azure-aware sync-script is version 28213, not
28214. Backend was pinning a non-existent hub script, which broke the
git_sync_e2e suite (every deploy's sync step 404'd).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(azure-trigger): add azure_triggers to token scope selector + skill
- windmill-api/src/token.rs: `build_trigger_scope_domains` was missing
`("azure_triggers", "Azure Event Grid")`, so the CreateToken UI's scope
selector didn't surface azure_triggers:read/write. Backend already had
`ScopeDomain::AzureTriggers` wired (scopes.rs), this just exposes it.
- .claude/skills/adding-a-trigger/SKILL.md: capture both scope-related
files under the hardcoded-arrays section so future triggers don't miss
the UI surface.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs(adding-a-trigger-skill): clarify token.rs scope effect
Not a regression — nothing was working before. Skipping TRIGGER_DOMAINS
just means the scope works via API/CLI but has no UI checkbox.
* docs(adding-a-trigger-skill): trim token.rs bullet
* fix(azure-trigger): regen openapi-deref + swap textarea for TextInput
- Run build_openapi.sh to regenerate openapi-deref.{yaml,json} with the
12 azure_triggers paths + schemas. These files are served by the
runtime (include_str! in windmill-api/src/lib.rs) to external SDK
consumers; without this regen the new endpoints wouldn't be advertised.
- Replace the raw <textarea> for event type filters with the
design-system TextInput in textarea mode (frontend/CLAUDE.md bans raw
HTML elements).
Addresses cubic + claude PR review items.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben Fiszel
|
60211c1d19 |
feat: folder default_permissioned_as rules for ownership defaults on deploy (#8801)
* feat: add folder default_permissioned_as rules for ownership defaults on deploy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unnecessary auth guard on default_permissioned_as — rules are advisory only Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts with new CLI commands Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address CI review findings — TOCTOU, race condition, email validation, type coercion Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add sqlx offline cache for test queries (fixes cargo_test CI) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review findings — incomplete request bodies, dead code, redundant import Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address remaining review findings — full script fields, reactive stores, catch-all validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: app/schedule/trigger set-permissioned-as fetch remote first to avoid data loss Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: app set-permissioned-as avoid creating redundant app version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: compact user/group toggle + select for folder default_permissioned_as rules Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: collapse default_permissioned_as section by default in folder editor Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: include default_permissioned_as in FolderFile CLI type for YAML round-trip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: process folder.meta changes before items in push to apply new rules immediately Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: clone default_permissioned_as on fork/rename + add full lifecycle tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add no-op guarantee test — folder without rules behaves like before Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: rename cliBehavior to syncBehavior — more accurate scope Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5b97092997 |
feat: unify CLI config to workspaces, deprecate gitBranches/environments (#8767)
* refactor: unify CLI config to workspaces, deprecate gitBranches/environments Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update frontend examples and regenerate system prompts for workspaces config Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: update test files to use workspaces config instead of gitBranches Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: handle --branch with --base-url correctly in sync pull/push Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: warn when --workspace overrides auto-detected branch or misses config entry Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: show reason why workspace was selected in log message Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: clarify specificItems file naming uses gitBranch as suffix Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: rename branch-specific to workspace-specific, use workspace name as file suffix Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: rename branch-specific to workspace-specific, add comprehensive integration tests Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: simplify bind and init to be workspace-centric Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: make bind/unbind interactive with --workspace and --branch flags Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: make bind interactive with profile selection, workspace name, and optional branch Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: init offers to bind workspace using same flow as wmill workspace bind Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: skip backend git-sync check in init when no workspace was bound Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: skip all API calls in init when no workspace was bound Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: log when RT namespace is skipped, offer to generate it after bind Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: warn when no workspace bound during init Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: init git-sync check uses bound workspace, not active profile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: init uses selected profile directly, avoids re-resolving and duplicate prompt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: init skips requireLogin, uses bound profile token directly Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: auto-pick or prompt workspace from config when no branch matches Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: show configured workspaces list and bind hint in resolution messages Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: cache bound profile to avoid duplicate profile selection prompts in init Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: hoist boundProfile scope, add 2 comprehensive integration tests covering all flows Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rt.d.ts prompt defaults to no when file exists, better description Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove empty overrides from generated config, add specificItems hint Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add inline comments for non-trivial fields, add overrides/promotionOverrides hints to bound workspaces Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: regenerate system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
80c8e076fc | cli nit | ||
|
Ruben Fiszel
|
4c16877366 | update sys prompts | ||
 centdix
|
b960598431 |
fix: hide deprecated cli metadata commands (#8699)
* fix: hide deprecated cli metadata commands Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: simplify generate-metadata guidance Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Alexander Petric
|
6c3c971af5 |
feat: improve CLI flow log streaming and job inspection (#8644)
* fix: improve CLI flow log streaming, sub-job listing, and failure handling Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add hierarchical flow status in job get and aggregated flow logs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove duplicate ansi color hint in job logs output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update cli-commands skill with new job/flow features Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add integration tests for flow job inspection and log aggregation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove internal friction discovery doc from branch Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: trim cli-commands skill to reduce context bloat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: update job command descriptions and regenerate skills.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: commit auto-generated files from system_prompts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address review comments on flow streaming and test assertions - Move for-loop waiting logic outside --silent guard (Cubic #2) - Break outer loop when for-loop module fails (Cubic #3) - Strengthen test assertion: toContain("a") -> toContain("a: Generate data") (Cubic #1) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: generator regex truncating descriptions with parentheses The .command() regex used [^)]+ for the second arg, stopping at the first ')' inside description strings like "(machine-friendly)". Now matches quoted strings properly before falling back. Fixes 6 truncated descriptions across job, flow, and script commands. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
c6ce3197a7 |
fix(cli): phantom diffs, flow safety, trigger DX, lint watch, error clarity (#8588)
* fix(cli): phantom diffs, flow push safety, error messages, digest stability Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): differentiate stale vs missing metadata warnings on script push Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): job list --limit off-by-one, deps push double error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): flow get shows nested steps, lint works on specific directories Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(cli): add lint --watch mode for continuous validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): email trigger template missing local_part, trigger get shows all fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): fix CI — flow push warns instead of failing, lint subdir detection Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f40cdaf434 |
fix(cli): app push crash, lint path, push --message, run validation, history timestamps (#8585)
* fix(cli): app push crash, lint entry point, push --message, run arg validation, history timestamps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): update sqlx cache and fix second history query missing created_at Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(cli): regenerate system prompts after new CLI options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
d29cb234db |
feat(cli): add job, group, audit, token commands and schedule enable/disable (#8581)
* feat(cli): add job, group, audit, token commands and schedule enable/disable Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(cli): regenerate system prompts after new commands Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): address PR review feedback Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(cli): regenerate system prompts after review fixes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor(cli): extract shared formatTimestamp util and remove unused resolveWorkspace in token Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
501a4ff2a9 |
fix: Improve CLI developer experience: error handling, sync workflow, JSON output, workspace forks (#8578)
* fix(cli): address 28 DX friction points across CLI commands Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(cli): regenerate system prompts after help text updates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address PR review feedback Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): update removeType tests to match lenient behavior Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): address CE/EE sync friction and improve JSON output Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): revert instance config masking to avoid breaking push flow Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): mask instance secrets by default with interactive prompt Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(cli): regenerate system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): use stderr for errors, optimize skipped-files scan, rename --auto to --auto-metadata Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(cli): improve workspace fork lifecycle — delete-fork fallback, list-forks, --workspace override Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): update fork merge instructions to reference all merge methods Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): clarify skipped-files warning comment re DynFSElement traversal Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
centdix
|
d06b42613f |
feat(cli): generate commented wmill.yaml and add config reference command (#8546)
* feat: generate commented wmill.yaml template and add config reference command Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing options to config reference (promotion, skipBranchValidation, commonSpecificItems) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: generate YAML template from CONFIG_REFERENCE instead of handwritten string Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: preserve YAML comments when binding workspace profile during init Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: simplify to `wmill config` and reorder table columns Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: generate JSON Schema for wmill.yaml editor autocomplete and validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove redundant templateValue fields and make specificItemsSchema data-driven Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: use native JSON Schema types in CONFIG_REFERENCE, strip non-schema keys for generation Eliminates typeToJsonSchema, specificItemsSchema, codebaseItemSchema, branchConfigSchema, and the complex generateJsonSchema body. Each CONFIG_REFERENCE entry is now a JSON Schema property with extra metadata. Schema generation just iterates and strips non-schema keys. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove typeLabel and displayType — use schema types directly Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove hidden entries, auto-expand nested schemas in reference table Sub-fields (codebases[], gitBranches.<branch>.*) are now derived from the parent's inline schema instead of being maintained as duplicate hidden entries. Removes 29 entries and the hidden field entirely. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use console.log for JSON output and quote YAML-special branch names Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts to include new config command Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback + add tests for template, schema, and config reference - Use console.log for --json output (no ANSI escape codes) - Quote branch names with YAML-special characters - Add 28 tests covering template generation, JSON Schema validation, config reference formatting, and CONFIG_REFERENCE integrity Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add generate-schema script and commit wmill.schema.json to repo Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove schema.json generation from wmill init Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: eliminate read-back cycle, harden yamlKey, fix triple negation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
8cfaa91d43 | update cli freshness | ||
|
centdix
|
435de95e7d |
feat(cli): use local scripts when previewing flows (#8365)
* feat(cli): use local scripts when previewing flows When previewing a flow, PathScript modules (type: "script") now resolve to local file content instead of remote versions. This ensures flow preview and dev mode test the actual local changes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(cli): add tests for PathScript local replacement in flow preview Unit tests for replacePathScriptsWithLocal covering: - basic PathScript→RawScript conversion - tag_override preservation - missing local file fallback - mixed module types - nested structures (loops, branches) Integration test verifying flow preview with a PathScript step uses the local script file content. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): extract shared helpers and add aiagent support for PathScript replacement Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): replace `as any` casts with proper type assertions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): preserve local flow preview script context * fix(cli): normalize inline flow preview bundles for bun * fix(cli): make local flow path scripts opt-in * fix(cli): only merge flow preview config for local mode * chore(system-prompts): regenerate cli command guidance * fix(cli): skip deno defaultTs test in CI without deno runtime Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(cli): clean up local path script helpers * feat(cli): make flow preview use local path scripts * fix(cli): ignore normalized preview metadata drift * chore(cli): address review follow-ups * test(cli): cover custom bundler path quoting --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
hugocasa
|
fe051aa22b |
feat(cli): add --env alias for --branch and environments config alias (#8415)
* feat(cli): add --env alias for --branch and environments config alias Add --env as a CLI alias for --branch on sync pull, sync push, workspace bind, and workspace unbind commands. Add environments as a permanent config alias for gitBranches in wmill.yaml. This helps users who use single-branch multi-environment workflows where "branch" terminology is confusing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate auto-generated system prompts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
87215193ca | system promps generate metadata | ||
|
Ruben Fiszel
|
749964e326 |
ci: add freshness check for auto-generated system prompts (#8338)
* ci: add freshness check for auto-generated system prompts Add a CI workflow and script to verify system_prompts/auto-generated/ stays in sync with its source files (SDKs, schemas, CLI commands, etc). Also remove the hardcoded CLI version from generated output to avoid unnecessary churn on every release. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * imports * imports Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * regenerate system prompts after rebase on main Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
centdix
|
7fb729cc84 |
fix(cli): instruct agent to tell user about generate-metadata and sync push instead of running them (#8318)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
centdix
|
b40cf80fdd |
fix: optimize flow lock generation and add rt.d.ts guidance for TS resource types (#8295)
Instruct AI to pass specific flow folder path to `wmill flow generate-locks` instead of running it on all flows. Also add guidance for TypeScript language files to check `rt.d.ts` for available resource types before using them. Re-ran generate.py to propagate changes to all auto-generated files. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
centdix
|
835db5d290 |
feat(cli): detect missing folders on sync push and add 'wmill folder add-missing' (#8011)
* fix: auto-create missing folders during sync push for non-admin users Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: show missing folders in sync push summary before confirmation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: improve sync push folder auto-creation error handling and json output Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: only treat 404 as missing folder in getFolder check Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: remove obsolete Deno compatibility layer from yaml-validator Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore(cli): add @types/bun dev dependency Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(cli): replace auto-create folders with `wmill folder add-missing` command Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): improve folder commands with summary field and simpler push API Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(cli): add confirmation prompt to folder add-missing command Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor(cli): simplify missing folder check to use local stat instead of remote API Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * update skills * feat(cli): warn admins but block non-admins on missing folder.meta.yaml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * cleaning * cleaning * test(cli): add tests for missing folder detection and folder commands - Add tests for `folder new`, `folder push`, `folder add-missing` commands - Add tests for sync push missing folder.meta.yaml detection (admin warning, non-admin block) - Fix getBasePostgresUrl to strip query params (e.g. ?sslmode=disable) from DATABASE_URL - Add createNonAdminUser and runCLIWithToken test utilities to test_backend.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): unify runCLICommand with optional token parameter Replace separate runCLIWithToken utility with an optional { workspace?, token? } options object on the existing runCLICommand across all backends. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * own workspace * test(cli): isolate folder_missing_meta tests with per-test workspace * test(cli): shorten isolated workspace id/name for workspace limits * test(cli): archive temp isolated workspaces after each folder test --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
centdix
|
a7ce5484b8 |
feat(local-dev): create Claude skills when doing wmill init (#7699)
* use skills * add prompts * update system prompts * generate skills on init * add prompts in cli * better for raw apps * nit * test pipeline draft * better * yaml for triggers and schedules * cleaning * better * add descriptions to ai agent fileds * adjust * better openapi * better * nit * feat: add typed provider and memory schemas for ai agent in openapi Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: improve zod validation errors with dynamic schema extraction Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * regen * fix * cleaning * refactor: deduplicate skill descriptions in generate_skills_ts_export Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * cleaning --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |