Commit Graph
16 Commits
Author SHA1 Message Date
68debab877 feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230)
* feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library

Fixes WIN-2214

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish

Revert the amqp_trigger additions to the shared windmill-utils-internal
TriggerDeployKind and the frontend cross-workspace deploy adapter: the
frontend installs the published npm package, which lacks the new kind
until a release is cut. AMQP create/edit/delete/list/sync/capture are
unaffected (they use local types); only cross-workspace deploy/merge of
AMQP triggers waits on the package bump. Also document the at-most-once
ack in the consumer loop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts

- ack AMQP deliveries only after successful dispatch; nack+requeue on failure
- add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on
  workspace deletion (and the listener stops)
- fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to
  WorkspaceDiffRow.kind
- register AMQP in the generated workspace trigger tool (create_trigger)
- drop banned $bindable defaults on optional props in the config section
- add build_uri unit tests (encoding, ports, vhost)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): stop AMQP poison-message loop and reconnect on transient drops

Chaos testing against a live RabbitMQ broker showed the previous
nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000
critical-error reports/sec) on a poison message, and any connection blip
permanently disabled the trigger (lapin has no built-in reconnect).

- on dispatch failure: nack+requeue then stop consuming; the listener
  framework re-lists the trigger after its ping goes stale (~15s), backing
  redelivery off to that cadence instead of a tight loop (verified: rate
  dropped from ~1000/s to ~1 per ~26s, message preserved)
- on connection/stream error: stop and let the framework reconnect instead
  of disabling; persistent failures are still disabled via get_consumer
  (verified: a forced connection close now auto-reconnects and resumes)
- finish the AI create-trigger action wiring for AMQP: add amqp to
  CreatedResourceTriggerKind, the action-card registry, and the drawer
  registry so the result card renders and its "Open" action works

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): complete AMQP frontend registries and defer merge rows

- add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs
  jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES
- wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union,
  writeTriggerSchema, triggerServices, and the draft adapter
- stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger
  from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred
  until windmill-utils-internal is published — avoids a deploy that fails
  with "Unknown kind: amqp_trigger"
- use design-system TextInput instead of raw <input> in the config section

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): complete AMQP session/draft registries and constrain prefetch

- add amqp to the session-deploy, draft-compare, preview-router, and
  copilot workspace-item registries so AMQP drafts/deploys/nav/path
  resolution work
- include amqp_count in the MoveDrawer attached-trigger rename warning
- replace the raw prefetch <input> with a design-system TextInput bounded
  to an integer 1-65535 (backend u16) and block save on invalid values

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger

lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now
mirrors the Kafka trigger's explicit reconnect loop instead of relying on the
framework re-list (which disabled the trigger once get_consumer failed on a
sustained outage):

- get_consumer returns cheaply; consume owns a (re)connect loop that retries
  with a 30s backoff, reports a critical error every 10 failed attempts, and
  reports a recovered critical error once it reconnects — never disabling the
  trigger on a connectivity failure
- a consumer/stream error breaks out to reconnect rather than disabling
- dispatch failure still nacks+requeues (at-least-once) with a short backoff
  to avoid a tight poison-message loop, keeping the connection alive

Verified against a live RabbitMQ broker: killing the broker keeps the trigger
enabled and retrying (attempt N), and restarting it auto-reconnects (logs
"reconnected after N attempts") and resumes dispatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): complete AMQP capture registries and constrain prefetch contract

- add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor
  from a capture button no longer throws "Unknown TriggerKind: amqp"
- register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the
  script/flow CaptureButton menu
- bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust
  u16) and regenerate clients/prompts
- require a non-empty exchange name when the exchange binding is enabled
- build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types

- asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge
  so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas
- frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu,
  node presentation, event-trigger set, annotation keywords, and the
  editor/service registrations
- git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync
  deployment stages only its .amqp_trigger.* file, not an unrelated same-path object
- preprocessor starters: add the AMQP event to the generated TS/Python/PHP
  trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/
  delivery_tag)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists

- fix a stray edit that corrupted the pre-existing MqttTriggerEditor import
  ($lib/... path) in PipelineTriggerEditors.svelte
- reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0
  as unlimited) and defensively skip basic_qos(0) in build_consumer (covers
  the capture path that bypasses CRUD validation)
- recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add
  amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never
  runs an AMQP-only node as a manual root without an event
- add amqp to the preprocessor intro lists and both pipeline AI instructions

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists

- extract a shared validate_amqp_options used by both CRUD validate_config
  and build_consumer, so capture configs (which bypass CRUD validation) also
  reject prefetch 0 instead of silently connecting with an unlimited buffer
  (+ unit tests for 0/1/65535/None)
- add AMQP to the main script-writing preprocessor-sources prompt and the CLI
  triggers-skill guidance list

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text

- keep the zero-prefetch rationale only on the shared validate_amqp_options
  doc; drop the redundant call-site comments
- correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved")

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60

This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private.

Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380

New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60

Automated by sync-ee-ref workflow.

* chore(migrations): consolidate the four AMQP migrations into one

The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind,
draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the
migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger
table doesn't reference those enum types, so they can share a single migration
instead of four. Verified applying cleanly in a single transaction on a fresh DB.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-21 15:10:29 +00:00
e47aedac0a feat: add SQL migrations for data tables (#9693)
* feat: add datatable_migrations table

* feat: add route to run datatable migrations

* feat: sync datatable migrations as .up.sql/.down.sql files

* feat: add datatable migrate up/down commands and post-push run prompt

* feat: add datatable migrate new command to scaffold migrations

* feat: add datatable migrations management UI

* feat: prompt to create migration on DDL in datatable SQL editors

* feat: support running a single specific datatable migration

* feat: view migration content, run single migration, fix stacked modal

* feat: per-row revert button with out-of-order warning

* fix: avoid migrations list flicker on refresh after an action

* feat: generate initial datatable migration via pg_dump

* fix: surface datatable migration API error details in toasts

* fix: revert created migration if create-and-run fails to run

* fix: include postgres error detail in migration run/rollback failures

* feat: sync datatable migrations as files via the workspace export

* refactor: move datatable migrations to migrations/datatable/ path

* fix: drop redundant datatable_migration label in sync output

* fix: exclude datatable migration sql files from script metadata generation

* feat: run datatable migrations as user-permissioned labeled jobs

* feat: reject invalid datatable migrations on sync push

* feat: datatable migrate up/down default to all datatables, --datatable to target one

* fix: surface postgres error detail when datatable migrations fail to run

* chore: regenerate CLI docs for datatable migrate commands

* feat: default new datatable migration to a BEGIN/END transaction template

* fix: validate datatable migration name and datatable at the API boundary

* fix: ensure detected DDL ends with semicolon when wrapped in transaction

* fix: re-prompt instead of stripping DDL when new-migration modal is cancelled

* feat: refresh datatable schema after running a migration from the SQL REPL

* feat: record db manager DDL on data tables as migrations

* feat: make datatable migrations opt-in per data table

* fix: make migration view editor read-only so its code can scroll

* fix: don't re-prompt DDL guard when creating a migration without running

* feat: generate down migrations for db manager DDL (postgres)

* fix: correct down migration for db manager alters (no double-wrap, serial)

* feat: explain migrations purpose with a tooltip in the migrations modal

* compare paeg

* feat: add datatable_migration kind to workspace diff pipeline

* chore: point ee-repo-ref at datatable_migration git-sync companion

* fix: harden datatable migration version allocation and initial-migration bookkeeping, add tests

* feat: deploy and run datatable migrations on workspace merge

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Refactor + handle datatable setting delete/rename

* refactor: move datatable migration rename/delete cascade into module

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(windmill-utils-internal): bump to 1.7.1 for datatable migration deploy provider methods

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(db-manager): add Migrations button to top bar, make Refresh icon-only

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* BEGIN/END placeholder in down migration

* feat: autofocus migration name input and flag it red when empty

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* border nits

* refresh db manager schema on migrations

* BEGIN/END scaffold in CLI

* feat(cli): push local datatable migrations before running on migrate up

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: flag invalid migration name with red border, not just empty

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop random slug from auto-generated migration names

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: offer revert-and-delete when deleting an installed migration

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: record fork merge as a migration when target datatable opts in

* nit

* clone migrations on fork

* windmill-utils-internal

* fix(datatable-migrations): serialize run/rollback with a per-db advisory lock

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(db-manager): fail closed when migrations-status check errors on DDL apply

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: fix generate_initial migration ordering comment to match code

* chore(datatable-migrations): remove unused update_datatable_migrations endpoint

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: run DDL migration guard on the script editor Test button

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* split

* ee-repo-ref

* chore(frontend): sync package-lock with package.json (@emnapi deps)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(datatable-migrations): never resolve instance credentials into migration job args

datatable_database_arg eagerly resolved instance data-table credentials
(including the shared instance-wide Postgres password) and passed them as the
migration job's plaintext `database` arg, landing in v2_job.args. Since the
run route has no admin gate, a non-admin could run a migration and read
args.database to recover the password, granting cross-workspace psql access to
all instance data-table DBs.

Pass a `datatable://<name>` reference for both resource-backed and instance
data tables instead; the pg executor already resolves it to real credentials
server-side at run time, so nothing sensitive is ever stored in the job args.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit

* fix: handle dollar-quoting and comments when splitting SQL statements

* feat: deploy datatable migrations on merge with explicit opt-in error

* fix(frontend): sync package-lock with npm 11 peer-dep resolution

npm ci failed with 'Missing: @emnapi/core@1.11.2 / @emnapi/runtime@1.11.2 from
lock file'. @napi-rs/wasm-runtime declares @emnapi/core|runtime ^1.7.1 as
peerDependencies while @rolldown/binding-wasm32-wasi pins them to exactly
1.10.0. Newer npm (bundled with node 24 in CI) installs the peer deps at the
highest match (1.11.2) alongside rolldown's nested 1.10.0, so the ideal tree
needs both versions; the committed lock only had 1.10.0.

Regenerate the lock with npm 11.18 so it carries both 1.11.2 (top-level, for
the peer deps) and 1.10.0 (nested, for rolldown's pin). Verified npm ci passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit npm publish

* fix: fail closed on migrations-status error in fork schema merge

* nit CI emnapi/core version

* prevent initial_datatable_migration if migrations already exist

* fix(datatable-migrations): validate persisted data table names as path segments

edit_datatable_config only validated rename segments, not the actual
settings.datatables keys, so a data table could be saved directly under a name
like '..' or one containing '/'. Since new tables default to
migrations_enabled = true, generate_initial_datatable_migration would then
insert a migration row and the sync export would build
migrations/datatable/<name>/... paths from that name, producing malformed or
directory-escaping export paths.

Validate every persisted data table name in edit_datatable_config (alongside
the existing rename checks) and add validate_datatable_path_segment to
generate_initial_datatable_migration for defense in depth.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: scope datatable _wm_migrations by data table and cascade renames/deletes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(system_prompts): resolve nested local command groups in CLI docs generator

The CLI docs generator anchored on the first `new Command()` in a file and
never resolved locally-defined command groups passed as
`.command("name", localCmd)`. For datatable this flattened the nested
`migrate` group: it emitted `datatable new/up/down` plus a bare
`datatable migrate`, and mislabeled the datatable command with the migrate
group's description. jobs was broken the same way (its description was pull's,
and pull/push rendered empty).

Anchor block extraction on the `export default`ed command, recurse into
locally-defined `const x = new Command()` groups mounted as subcommands, and
render nested sub-subcommands. Regenerated docs now show
`datatable migrate new/up/down` and `jobs pull/push` with their real
options.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: drop unreleased _wm_migrations legacy-upgrade handling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: return datatable migration SQL from getItemValue for the diff drawer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(frontend): use windmill-utils-internal 1.8.2 for migration diff drawer

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* nit

* nit

* fix: handle datatable migration renames on push and dedupe timestamps

* fix: reject rewriting an already-applied datatable migration on upsert

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): add missing @emnapi/core and @emnapi/runtime lockfile entries

Resolves npm ci EUSAGE failure: the optional cpu:wasm32 @rolldown/binding-wasm32-wasi
declares deps on @emnapi/core@1.11.2 and @emnapi/runtime@1.11.2 that had no resolved
lockfile entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): datatable migrate up/down default to main datatable, not all

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: fail closed when applied status unreadable on datatable migration rewrite

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: surface full error detail in Database Manager DDL/query errors

* "See migration" button in the toast

* feat: add Enter shortcut to Create-a-migration in the DDL guard

* fix(frontend): warn before running a newly-created datatable migration out of order

The row-level Run action warns when earlier migrations are still pending, but
the create-and-run paths ran a just-created migration with `only` directly,
applying it ahead of older pending migrations without that confirmation.

Reuse the same "Run migration out of order" confirmation across all
create-and-run paths via a shared helper (datatableMigrationUtils):
- NewDataTableMigrationModal "Create and run" (and the DDL guard path)
- DatatableSchemaDiff fork→parent merge
- dbOps schema ops (DB manager create/alter/drop) — the pure factory throws a
  MigrationRunCancelled sentinel on decline, which DBTableEditor treats as a
  silent cancel

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: keep renamed datatable migrations visible in compare view

* fix: record per-migration deployment on datatable migrations disable

* fix(cli): run deployed datatable migrations after workspace merge

The merge command upserted datatable_migration definitions into the target
workspace and reported the item as successfully deployed, but never ran the
migrations. For forked datatables backed by separate databases, this left the
target schema unchanged until someone manually ran `wmill datatable migrate up`,
while the CLI reported a successful merge.

Collect the datatable migrations deployed (not deleted) into the target and,
after the deploy loop, offer to run them via the existing offerToRunNewMigrations
helper — the same post-deploy run prompt the push/sync path uses (interactive
only; `--yes`/non-TTY skip the mutating run, matching push behavior). Export
parseDatatableMigrationDeployPath so the merge path can parse the deployed items.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(backend): serialize datatable migration edits/deletes with the run lock

A migration run snapshots a migration's code_up from datatable_migrations and
only records its version in the data table's _wm_migrations after the job
succeeds. upsert_datatable_migration checked _wm_migrations before allowing an
edit but took no lock, so a concurrent edit could read "not applied yet",
rewrite code_up/code_down, and then the in-flight run would record the version
for the old SQL — leaving _wm_migrations pointing at SQL that was never applied
(migrate up then skips it; rollback runs a down that doesn't match).

Serialize definition rewrites and deletes with the same per-database advisory
lock the run/rollback paths use:
- Factor the connect+advisory-lock into lock_datatable_migration_runs and the
  applied-versions read into read_applied_versions_on_client.
- run_datatable_migrations now snapshots the definitions AFTER taking the lock,
  so code_up can't change between snapshot and version-record.
- upsert (when changing an existing def) and delete take the lock across the
  applied-check and the write; delete now rejects deleting an already-applied
  migration (would orphan its _wm_migrations record), symmetric with upsert.
  Both fail closed if the data table database is unreachable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): stack the out-of-order migration confirm above the DB editor preview

Creating a table on a migrations-enabled data table opened the DB table editor's
"Confirm running the following" preview modal, whose confirm triggers applyDdl,
which then asks for out-of-order confirmation. Both are ConfirmationModals with a
hardcoded z-[9999]; the out-of-order one lives in DBManagerContent (mounted before
the editor), so it rendered behind the still-open preview modal.

Add an optional zIndexClass prop to ConfirmationModal (default z-[9999],
backward-compatible) and give the DB-manager out-of-order confirm z-[10000] so it
stacks on top.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 27672e37df5d9dfde94f19963d5ffcdf8dd5448c

This commit updates the EE repository reference after PR #623 was merged in windmill-ee-private.

Previous ee-repo-ref: 6c287041cd7edd4a77a4bc07ad0e156cec32cce4

New ee-repo-ref: 27672e37df5d9dfde94f19963d5ffcdf8dd5448c

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-07-07 08:25:16 +00:00
+2 7155a0bb96 feat: Data Pipelines alpha (#9193)
* feat: add workspace asset graph view

Workspace-wide canvas of assets and their producer/consumer scripts,
reachable from the assets page. Left-to-right layered layout via
d3-dag sugiyama, rendered with @xyflow/svelte (same stack as the
flow editor). GET /w/:ws/assets/graph returns deduped nodes + edges.

Follow-ups: filters (kind/folder/search), node detail drawer, inline
script edit from a clicked node.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* all

* all

* update

* all

* all

* all

* feat(pipeline): output-kind picker and per-(lang, output) templates

Add a third stage to PipelineInsertMenu that asks what kind of asset the
new script will produce (datatable / ducklake / s3 parquet / s3 object /
none). The picked kind drives a real wmill SDK skeleton — typed
datatable inserts, ducklake CREATE+INSERT, s3 parquet COPY, etc. — with
the upstream asset auto-wired as the input source when added from an
asset node. Reorder languages to bun → duckdb → python → sql so
data-shaped languages surface first.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* chore(main): release 1.693.4 (#8994)

* chore(main): release 1.693.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit (#8997)

* feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: include .yaml variants in collections/roles requirements lookup

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries (#9000)

* fix(cli): only preserve case for raw-app runnableIds, not app/flow summaries

PR #8940 stopped lowercasing in sanitizeForFilesystem to fix #8939, where
a raw-app runnableId like CamelCaseTSRunnable produced a CamelCase YAML
metadata file but a lowercased code file, making them desync and
register as duplicate runnables on push.

That fix overshot. sanitizeForFilesystem is also reached by
newPathAssigner, which serves normal apps and flows where the input is
the script's human summary ("Get Users Data") rather than an identifier.
There the on-disk filename is the only artifact — there's no companion
YAML to keep in sync — so lowercasing was the right behavior. Removing
it changed both the on-disk filename and the !inline reference in
app.yaml / flow.yaml from get_users_data.inline_script.ts to
Get_Users_Data.inline_script.ts on the next pull, surfacing as
unwanted case churn for users updating to 1.693.x.

Add a preserveCase option to sanitizeForFilesystem (default false →
lowercase). newRawAppPathAssigner opts in; newPathAssigner stays on
the default. Update unit tests accordingly and add an end-to-end
raw-app round-trip in raw_app_sync.test.ts that pushes a CamelCase
backend runnable, pulls it back, and asserts both YAML and code file
preserve case with no lowercase orphan.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(cli): use readdir for exact-case orphan check on Windows

The CamelCase round-trip test used fileExists("camelcasetsrunnable.ts")
to assert no lowercase orphan was produced, which false-positives on
Windows since the filesystem is case-insensitive and resolves the
lookup to the existing CamelCaseTSRunnable.ts. Switch to readdir +
toContain so the exact on-disk casing is compared identically on Linux
and Windows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): wmill-lock.yaml auto-fill + --rehash-only + path-prefix dedup (#8978)

* fix(cli): canonical lockfile hashes + lock upgrade migration to v3

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): use __app_hash subpath in rehash missing-entry check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): run sync pull lockfile auto-fill regardless of changes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: regenerate system prompts for new lock and rehash-only commands

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on lock upgrade

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): drop v3 marker; always run fallback; fail-fast on unknown lockfile version

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): drop yaml-round-trip legacy hash variant; recover via --rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): include legacy hash in script push staleness warning check

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* revert(cli): drop canonical hash formula; keep raw-bytes hashing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* perf(cli): reuse change-tracker map for sync pull lockfile auto-fill

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): pin lockfile hash + yaml format and cover regression cases

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): byte-stable snapshot tests for flow.yaml format

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(cli): add app and script-metadata yaml snapshot fixtures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address claude review on rehash-only

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(cli): factorize script-path to remote-path derivation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address claude + cubic review (dry-run mutation, rehash short-circuit)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(cli): make rehash a subcommand and factorize fs walks

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): normalize line endings in yaml snapshot tests for windows ci

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(cli): address review feedback on rehash + auto-fill

- Flat-layout scripts now clearGlobalLock before rehash write so legacy
  ./-prefixed duplicates get cleaned up (matches flow/app behavior).
- Add MalformedLockfileError; sync pull auto-fill re-throws it alongside
  UnknownLockVersionError instead of silently warning + continuing.
- Document the legacy step-removal false-negative in
  isFlowDirectlyStale / isAppDirectlyStale and the categorizeLocalFiles
  ignore-filter invariant.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use otel.status_message for OTLP Status.message on failed jobs (#8995)

tracing-opentelemetry only recognizes otel.status_code and
otel.status_message as fields that map to the OTLP Status proto.
The previously-used otel.status_description fell through to the
generic attribute recorder, leaving Status.message unset and
preventing OTLP consumers from filtering spans on error status.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: route email trigger path through standard info channel (#8996)

* docs(skill): document email triggers and S3 attachments

Add an "Email triggers" section to the triggers skill covering the
local-part config, the parsed_email/raw_email/email_extra_args payload,
the URL-style extras convention, where to find trigger_path (only with
a preprocessor, at event.trigger_path), and — most importantly — that
binary attachments are uploaded to the workspace S3 bucket and surface
as `{ s3: "windmill_emails/<job_id>/attachments/<filename>" }`. Scripts
must use wmill.loadS3File / wmill.load_s3_file to read them.

Also pulls EmailTrigger into the schema mappings so a real
`email_trigger.schema.yaml` is generated, and adds Email/Azure to the
trigger kinds list in the CLI agent guidance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref for email trigger path fix

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 26184ab7a4aadfc529dcedf038aa08d36c7ad381

This commit updates the EE repository reference after PR #553 was merged in windmill-ee-private.

Previous ee-repo-ref: 318a46897a605dc9be3817901f35ba5a99a0a525

New ee-repo-ref: 26184ab7a4aadfc529dcedf038aa08d36c7ad381

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* update git sync version to 1.693.5

* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe (#8999)

* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(pg): wrap encoder errors with arg context, add fallback test

Followups on #8999 review:

- Wrap rust-postgres "error serializing parameter N" failures with the arg
  name, JSON value kind, and asserted Postgres type plus a hint about an
  explicit cast — so users see actionable context instead of an opaque
  WrongType.
- Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree
  on the Type for every recognised arg_t when the JSON value matches its
  natural Rust kind. Catches future drift if either side changes.
- Integration test for the prepare + query_raw fallback path: confirms
  unrecognised arg_t (custom enum) is routed through prepare and the
  server-resolved type appears in the failure surface — flips into a
  test failure if a regression accidentally routes unrecognised types
  through query_typed_raw.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add otyp_inferred flag + regex-based placeholder renumbering

Two follow-ups from the review of #8999:

1. **Issue #1 (Number/Bool + explicit text decl in WHERE)**

   Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets
   it `true` only at the "no info → fall back to text" site (bare `$N`,
   no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep
   it `false`.

   In `convert_val` this flag distinguishes:
   - explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce
     `Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works
     (`text = text` operator). Pre-#8988 behaviour, restored.
   - parser-default text (bare `$N`) — bind the value's natural Rust
     type so the regression case (`Value::Bool` against a real `bool`
     column via `CAST AS bool`) keeps working.

   `Arg` is in `windmill-parser`; the new field has `#[serde(default)]`
   so persisted signatures stay backward-compatible.

2. **Issue #4 ($5/$50 substring rewrite collision)**

   Replace the per-index `String::replace` chain (which turned `$50`
   into `$10` when oidx=5 was processed first) with a single regex
   pass. `\d+` is greedy, so `$5` and `$50` match as distinct units;
   indices outside the mapping are left intact.

3. Tests:
   - parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline-
     cast/decl/mixed shapes.
   - executor unit: `convert_val_bool_against_every_arg_t` and
     `convert_val_*_number_*` split each text-like target into explicit
     vs inferred expectations.
   - executor unit: `renumber_sparse_placeholders_no_collision`.
   - integration: `test_postgresql_arg_type_combinations` adds 4 cases
     covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and
     sparse positional args ($5/$50).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality

Backend:

1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s
   `ToSql for String` / `FromSql for String` reject `Kind::Enum` and
   `Kind::Domain` even though the wire format is plain UTF-8. The wrapper
   accepts those kinds in both directions. End result: explicit
   `$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the
   ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col`
   results come back as JSON strings instead of erroring at the FromSql
   layer.

2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these
   arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text /
   non-temporal arg_t fell through to `Box<String> + TEXT`, which then
   failed at the server (no implicit cast text→numeric in expression
   context). Now strings are parsed into the matching native type with
   clear error messages on parse failure.

3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering
   (which fixed the `$5/$50` substring collision but still walked through
   string literals and comments, mangling `'price: $5'` etc.) with a
   walk over `parse_pg_statement_arg_positions` — the same
   string/comment/dollar-quote-aware tokenizer used for index discovery.
   Adds `parse_pg_statement_arg_positions` to the parser's public API.

SDK:

4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now
   stringifies bigints before serialisation; the executor accepts
   numeric strings into BIGINT arg slots via the existing
   `Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split
   so `BigInt` always resolves to `BIGINT` (was reaching
   `Number.isInteger(BigInt)` which returns false → wrong default).

5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column
   now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers
   primitive types only (number / bigint / string / boolean); mixed or
   nested arrays still fall back to JSON. Mixed int/float widens to
   `DOUBLE PRECISION[]`.

6. **`.query()` positional bug**: previously the `.query()` method
   abused the template-tag builder, which appended `$N::TYPE` after the
   user's literal SQL string instead of binding by position
   (`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()`
   builds the executor-shaped content directly: a `-- $N argN (TYPE)`
   declaration block followed by the user's SQL verbatim.

Tests:

- Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments`
  asserts string literals, comments, and dollar-quoted blocks don't
  produce positions (so renumbering doesn't mangle them).
- Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling`
  uses the new position-aware path and includes string-literal + comment
  + `$$…$$` cases. Existing convert_val tests grow to cover new
  String→numeric/real/double/oid/bool arms.
- Integration: `test_postgresql_arg_type_combinations` adds 13 cases
  (enum round-trip both directions, string→numeric/real/double/bool/oid,
  string-literal `$N` non-mangling). The prepare-fallback test now
  asserts SUCCESS (not failure) for enum encoding via AnyTextValue.
- SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests)
  exhaustively covering inferSqlType primitives + arrays,
  parseTypeAnnotation, datatable() template tag (with all the new
  shapes — BigInt, homogeneous arrays, RawSql, schema preamble),
  datatable().query() positional, and ducklake() shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache)

Found while exhaustively probing custom-type DX: every cached-connection
reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL`
deallocates *all* prepared statements server-side — including the typeinfo
statements that tokio_postgres caches per-Client to resolve custom enum /
domain Oids. tokio_postgres still held `Statement` objects whose names
the server had forgotten, so the next custom-type query failed with
intermittent "prepared statement \"sN\" does not exist" errors. The
failure was easy to reproduce: any sequence that forced typeinfo lookup
for two different custom-type kinds on the same cached connection (e.g.
enum followed by domain) would hit it.

Replace `DISCARD ALL` with a curated reset that explicitly targets the
state we actually care about, *without* touching prepared statements:

  RESET ALL                     — GUC parameters (search_path, application
                                  _name, statement_timeout, …)
  RESET SESSION AUTHORIZATION   — undoes both `SET SESSION AUTHORIZATION`
                                  and `SET ROLE` (RESET ALL does NOT —
                                  these aren't GUC parameters, so without
                                  this an elevated role from a previous
                                  job would silently leak)
  UNLISTEN *                    — drops LISTEN registrations
  CLOSE ALL                     — closes open cursors

Trade-off: temp tables, advisory locks (session-scoped), and user-created
PREPARE statements may persist across cached-connection reuse — rare in
datatable / PG-script workloads. tokio_postgres's typeinfo cache survives
intact, so custom enum / domain queries are fast on subsequent reuse.

Tests:
- `test_postgresql_custom_types_on_cached_connection` — runs 10×
  alternating enum + domain queries on a cached connection. Pre-fix this
  failed with `prepared statement "sN" does not exist` after the first
  reuse; post-fix passes.
- `test_postgresql_set_role_does_not_leak_across_cached_connection` —
  switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres
  role, then runs a follow-up job and asserts current_user/session_user
  are restored. Specifically catches the case where someone might switch
  back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION
  AUTHORIZATION) and silently introduce a permission-leak vector.
- All existing session-isolation tests
  (`test_postgresql_cached_connection_resets_session`,
   `test_postgresql_single_worker_session_isolation`,
   `test_postgresql_100_jobs_cached`) continue to pass.

Found via end-to-end probing of datatable / PG-script DX, not previously
covered: the existing isolation tests only did `SET ROLE postgres`, the
connecting user, so the leak was invisible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): address PR #8999 review (cubic + claude)

cubic (P1, real bug):
- `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but
  chrono `NaiveTime` only encodes for TIME (same caveat as the scalar
  arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz`
  assignment cast at the column site. Add an explicit unit test.

claude (#1, silent failure → explicit error):
- `Bool` + explicit `(char)` / `(character)` decl previously silently
  bound BOOL, hoping the server would cast at the use site — but PG has
  no implicit `bool→char` and the resulting error
  ("operator does not exist: bool = char") was opaque. Now error at
  bind time with an actionable hint to use `bool` decl or pass the
  value as a "t"/"f" string.

claude (#2, asymmetry doc):
- Object/Array still coerce to text on `matches!(typ, Typ::Str(_))`
  (covers both explicit AND inferred-default text), unlike Bool/Number
  which key on `explicit_text_target`. The asymmetry is intentional
  (no implicit `jsonb → text` cast in expression context vs PG having
  implicit `bool/int → text` casts) — added a body comment so future
  maintainers don't try to "align" them.

claude (#3, perf):
- `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions`
  walked the SQL tokenizer twice. Fold into a single pass that derives
  the index set from the position list.

claude (#4, fmt drift):
- `cargo fmt` over the parser crates I touched with perl scripts in the
  earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py,
  rust,graphql,yaml,r}). Net cosmetic.

claude (#5, parseTypeAnnotation):
- One-line caveat in the SDK's `parseTypeAnnotation` that the returned
  string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns
  `"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a
  real PG type, but the only consumer just checks `!== undefined`).

While here — discovered + fixed independently while exhaustively probing
DX:

- **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET
  SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's
  `DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing
  intermittent `prepared statement "sN" does not exist` errors on
  custom-type queries after cached-conn reuse. New regression tests:
  `test_postgresql_custom_types_on_cached_connection` and
  `test_postgresql_set_role_does_not_leak_across_cached_connection`
  (the latter catches the case where someone might switch back to
  `RESET ALL` alone and silently introduce a permission-leak vector —
  RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION).

- **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix
  `TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00")
  and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") —
  neither parseable by `date-fns parseISO`, JavaScript `new Date()` is
  lenient enough to handle them but several frontend `App*Input.svelte`
  components use parseISO and fail silently. Switched to ISO-8601 with
  `T` separator and `+00:00` offset; arg-parsing path still accepts the
  legacy " UTC" suffix for back-compat.

Test coverage:
- 17/17 unit (`pg_executor::tests`)
- 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`)
- 27/27 parser (`windmill-parser-sql`)
- 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling

Found while probing PG-script DX with millions of numeric cells:

1. **Numeric precision-loss warning**: `numeric` results are still serialised
   as JSON Number (back-compat — switching to JSON String would silently
   break user code doing arithmetic on results), but we now detect
   `Decimal -> f64 -> Decimal` round-trip failure and emit a single
   job-log warning recommending a `::text` cast in the SQL. Bounded by
   `NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic
   load + one fetch_sub on the hot path; first lossy value
   short-circuits to a single load thereafter). Worst-case overhead on
   a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic
   loads (vs. ~100ms unbounded).

2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned
   `"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"`
   (TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is
   what the apps `App*Input.svelte` components use, so timestamp values
   silently failed to round-trip into date pickers. Switch to ISO-8601
   (`T` separator + `+00:00` offset) on the result side; arg-parser
   continues to accept the legacy `" UTC"`-suffixed format for
   back-compat.

3. **Float NaN / Infinity results**: `Number::from_f64` returns None for
   NaN / ±Inf, which `pg_cell_to_json_value` was raising as
   "invalid json-float" — failing the *entire* query if any cell held
   one of these special values. Now serialise them as JSON strings
   ("NaN", "Infinity", "-Infinity") and let the rest of the row come
   through. Arg-side: `s.parse::<f64>()` already accepts the same
   strings.

Tests:
- `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit
  cases for the precision-loss predicate.
- `precision_check_budget_caps_per_query_overhead` — locks in the
  budget cap and the loss-flag short-circuit.
- All 9 PG integration tests + 17 unit tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults

While probing PG-script DX further found three more frictions:

1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to
   `RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`
   meant session-scoped advisory locks (`pg_advisory_lock`) leaked
   across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()`
   to the chain — `DISCARD ALL` covered this implicitly via
   `DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it
   in the switch.

2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g.
   `-- $1 amount (numeric)`) but not provided in the args object was
   bound as NULL with no error / warning. Misspelling the key in the
   args object silently produced a row of NULLs — a notorious DX
   debugging trap. Now: collect the names of declared-but-missing
   args during dispatch and emit a single one-shot warning to the job
   logs at end-of-query naming each one. Bound NULL is preserved for
   back-compat.

3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries
   `arg.default = Some(Number(5))`, but the dispatch fell straight to
   NULL when the arg was missing. Now: respect the default —
   user-supplied value > declaration default > NULL. Also fixes the
   warning logic above (only warn for args that *don't* have a default).

Tests: existing 19 unit + 9 integration pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values

Two more frictions found while probing SDK end-to-end against a real
datatable resource:

1. **Multi-word array types lose the [] suffix in the parser**.
   `transform_types_with_spaces` recognises aliases for "double
   precision", "character varying", "timestamp with time zone", etc.
   but its return type was `&'a str` — only the bare alias, never with
   a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at
   the first space, so the regex's own `(?:\[\])?` array-suffix branch
   sees only `"double"` (not `"double precision[]"`); the `[]` was
   silently lost. Result: `$1::double precision[]` (which the SDK now
   emits for homogeneous float arrays via the new auto-tag) routed
   through `Value::Array → Type::JSONB` and the server failed with
   "cannot cast type jsonb to double precision[]".

   Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>`
   and re-check the trailing bytes after a multi-word match. If they
   start with `[]`, return `format!("{alias}[]")` — Owned. Single-word
   types and the no-match path keep returning Borrowed slices, so no
   allocation in the hot path.

2. **Array arms in `convert_vec_val` rejected stringified values for
   numeric / int* / bool / oid / real / double**. The scalar `convert_val`
   already parses strings into the matching native type for these arg_ts,
   but the array variant only accepted JSON-native counterparts. Sending
   `["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for
   bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with
   "Mixed types in array". Now the array arms mirror the scalar ones —
   `as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes
   round-trip cleanly.

Tests: 19 unit + 9 integration pass; existing parser tests cover the
multi-word array forms (the regex-cap behaviour didn't break for
single-word types, and Cow plumbing is transparent to all callers).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files

CI failures: the perl-driven sweep that added `otyp_inferred: false` to
every `Arg { ... }` literal when I introduced the field in the parser
schema covered `src/lib.rs` files but missed:

  - parsers/windmill-parser-bash/src/lib.rs       (mass-edited but a
    later format pass un-applied a few sites)
  - parsers/windmill-parser-go/src/lib.rs         (same)
  - parsers/windmill-parser-graphql/src/lib.rs    (same)
  - parsers/windmill-parser-nu/tests/tests.rs     (test file — not
    swept the first time)
  - parsers/windmill-parser-ts/tests/tests.rs     (test file — same)

Also tightened the regex to handle `oidx: None` without the trailing
comma (some test files had the field as the last initialiser line).

`cargo build --features <CI feature combo> --workspace --all-targets`
is clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string

Two more frictions found while running the actual SDK end-to-end against
a live datatable resource:

1. **JS `Date`** fell into the typeof "object" branch and was tagged
   `::JSON`. It worked accidentally for `${date}::timestamptz` via PG's
   `json → text → timestamptz` implicit cast chain, but `${date}` against
   a `timestamptz` column without a user-supplied cast bound the value
   as a JSON string and the comparison `timestamptz = json` failed. Now:
   `inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`;
   `serializeArgValue` emits `Date.toISOString()` so the executor's
   `Value::String → TIMESTAMPTZ` arm parses it cleanly.

2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)`
   returns `"null"` per the JS spec, so the value reached the executor as
   JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL
   double. Fix: detect non-finite numbers in `serializeArgValue` and
   stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's
   `Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals
   directly, and the result-side already renders the values as JSON
   strings (matching round-trip).

SDK unit tests grow from 42 → 44 passing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(pg): integration coverage for multi-word arrays + stringified array elements

Locks in the two array fixes from the previous commit
(`fix(pg): multi-word PG types with [] suffix lost the array-ness`)
with end-to-end cases in `test_postgresql_arg_type_combinations`:

- `double precision[]`, `character varying[]`, `timestamp without time
  zone[]` — verifies the parser keeps the `[]` suffix after multi-word
  alias resolution.
- `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies
  the array arms of `convert_vec_val` apply the same string-coercion
  the scalar arms do.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: fix indentation drift on otyp_inferred lines

cargo fmt cleanup of leftover indentation where the perl-driven sweep
that introduced the otyp_inferred field landed at the wrong column.
No behaviour change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat: support assigning a worker tag to app inline scripts (#9002)

* feat: support assigning a worker tag to app/raw-app inline scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: omit empty tag field from inline script raw_code payload

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: shrink tag popover width

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat(pipeline): 2-col picker, draft path edit, save-all + leave guard

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* all

* update

* fix(cli): forward HEADERS env var on every backend fetch call (#9075)

Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped
the `HEADERS` env var, causing requests to fail behind auth gateways like
Cloudflare Access (same shape as #6421):

- `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed
  in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw
  `fetch` for the `skip_if_noop` query param.
- Script preview `/jobs/run/preview_bundle`.
- App dev `/jobs_u/getupdate_sse` SSE stream.
- `wmill docs` `/api/inkeep`.

All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()`
so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque
JSON parse failure.

Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that
403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and
otherwise reverse-proxies to the test backend, then runs `wmill sync push`
of a fresh script through the proxy. Negative case (no `HEADERS` env)
verifies the proxy actually gates; positive case asserts every request
including `/scripts/create` reaches the backend with the headers attached.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): add --parallel flag to generate-metadata (#9074)

* feat(cli): add --parallel flag to generate-metadata

* fix(cli): validate --parallel input and harden flush ordering

* perf(flows): skip flow_env DB+transform work when no resolution is needed (#9078)

* fix(cli-tests): stabilize flow lock-gen race + Windows path (#9080)

* fix(cli-tests): stabilize flow lock-gen race + Windows path

Three CLI test failures on the latest main, all flaky on CI:

1. `Mixed Case Paths: pull and push flow with capitalized folder` and
   `Integration: Mixed scripts and flows with nonDottedPaths are
   idempotent`: flow create/update queues an async FlowDependencies job
   that fills inline-script lockfiles and rewrites flow.value. The tests
   pulled/pushed before the worker finished, so dry-run idempotency saw
   phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a
   `waitForFlowDependencyJob` helper that polls `/flows/get` for the
   latest `dependency_job` and `/jobs_u/completed/get` until it lands,
   and called it after each API/CLI flow write in both tests.

2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only,
   added in #9075): the new test built the CLI entrypoint via
   `new URL("..", import.meta.url).pathname`, which yields `/C:/...` on
   Windows and `Bun.spawn` rejected before reaching the proxy, leaving
   `rejectedRequests.length` at 0. Switched to
   `fileURLToPath` + `node:path.join` to match `cargo_backend.ts`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli-tests): use /flows/deployment_status to actually wait for dep job

CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob`
as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`,
but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60)
do not include that field. The helper exited on the first iteration
without polling.

Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns
`{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID
written into `deployment_metadata` in the same tx as the dep-job push
(backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by
the time the create/update API call returns, the response carries the
latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as
before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to
~32s, confirming the helper now actually waits instead of returning
immediately. The 404 short-circuit in `sync_pull_push.test.ts` still
works — `get_deployment_status` returns 404 when the flow is absent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(flows): cache resolved flow_env per flow execution (#9079)

* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures

* chore(main): release 1.698.0 (#9076)

* chore(main): release 1.698.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: reject root-rooted paths in ansible playbook validator on windows (#9081)

* fix(native-triggers): serialize Google channel renewal across replicas (#9060)

* fix(native-triggers): serialize Google channel renewal across replicas

`sync_all_triggers` runs every 5 minutes on every windmill-app replica
with no leader election. Multiple replicas were each rotating the
webhook token, creating a new Google watch channel, and racing the
trigger UPDATE — leaving the loser's new token (in `token`) and channel
(in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week
without the silent best-effort `delete_token_by_hash` ever logging a
warning.

Wrap each per-trigger renewal in a transaction and acquire the row with
`SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row
instead of duplicating the work. The lock spans `rotate_webhook_token`
→ Google API call → `update_native_trigger_service_config` and is only
released on commit. Re-checks `should_renew_channel` after acquiring
the lock so a replica that committed seconds earlier doesn't trigger a
duplicate renewal.

The pattern matches existing batch-cleanup paths in `monitor.rs`
(job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites.

Also logs at `debug!` when `delete_token_by_hash` finds no matching row,
so future investigations can distinguish "deleted" from "not found"
without changing the `Ok(false)` contract.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Address claude review:
- #5: per-skip log info -> debug (expected outcome under SKIP LOCKED)
- #2: warn moved out of delete_token_by_hash to the call site that knows the
  expected state (try_renew_channel_locked); other callers are race-prone and
  shouldn't warn
- #3: NULL service_config now warns (anomalous case)
- #4: post-Google-API DB-update + commit failures log distinctly so the
  channel-orphan case is grep-able

Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration,
mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the
existing 'ephemeral-' filter excludes them from user-token email/critical-alert
paths (no filter changes in 3 places). Orphans now self-clean via the existing
expiry sweep in monitor.rs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Address second-round review:
- Claude #1 (P2): username_override_from_label now strips the 'ephemeral-'
  prefix for ephemeral-webhook-* labels, so created_by stays
  webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-...
  (preserves audit/job-list filter compatibility)
- Codex (P2): updated renew_channel doc — labels are no longer copied; rotate
  mints fresh ephemeral-webhook-google-{rd5} with 14d expiration
- Claude #3 (optional): test_rotate_webhook_token now asserts the rotated
  Google token has an ephemeral-webhook-google-* label and a populated
  expiration

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fixup! fix(native-triggers): serialize Google channel renewal across replicas

fixup! fix(native-triggers): serialize Google channel renewal across replicas

Reconsider the previous fixup: stripping the 'ephemeral-' prefix made
created_by no longer match token.label exactly, defeating the linking
purpose. Just allowlist 'ephemeral-webhook-' alongside the other
recognized webhook/email/ws prefixes — created_by becomes
ephemeral-webhook-google-XXXXX, matching token.label exactly. The
'ephemeral-' substring also informs operators that this is a
system-managed auto-expiring token vs a user-managed webhook trigger.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): bump svelte version in `wmill app new` template (#9084)

* fix(cli): bump svelte version in `wmill app new` template

The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte
compiler bundled in `wmill app dev` emits `$.delegated('click', ...)`
calls. The `delegated` export was added later, so 5.45.2 doesn't have
it — esbuild warns `Import "delegated" will always be undefined`,
replaces the call with `void 0`, and the page crashes at first
event-handler bind (white screen).

Bump to `^5.55.5` so the compiler and runtime stay in sync.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(frontend): bump svelte version in raw_apps UI template

Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a
package.json with `svelte: "5.45.2"`. That works today only because
the bundled rolldown worker also pins 5.45.2 — when the worker is
upgraded past 5.51.1, the compiler will emit `$.delegated()` and the
runtime won't have it, producing the same white-page crash that hit
the CLI.

5.55.5 still exports `event` (used by the current bundled compiler),
so this is forward-compatible: it works with the 5.45.2 compiler now
and won't break when the worker is upgraded.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf(flows): gate flow_env resolve on expr text and share cache with handle_flow (#9085)

* feat: parse windmill_failure field to tag run as failure (#9073)

* feat: parse windmill_failure field in job result to tag run as failure

* feat: preserve top-level fields when windmill_failure tags a run as failure

* fix: address review findings on windmill_manual_failure

* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases

* fix: prefer injected ManualFailure error over sibling name/message in OTel

* fix: hide _ENTRYPOINT_OVERRIDE jobs from script/flow history panel (#9088)

* fix(flows): populate error handler input args from failure picker (#9087)

* fix(flows): populate error handler input args from failure picker

* style(flows): fix indentation in failure-step branch

* fix(python): verify wheel RECORD on cache pull/install, finalize piptar (#9090)

The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.

Three changes that together stop the propagation:

1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
   confirm every listed path exists on disk before writing
   `.valid.windmill`. On failure, wipe the directory and fall through
   to a fresh local install — the next install also self-heals the
   broken object-store entry by pushing a fresh tar.

2. After `uv pip install` succeeds, run the same RECORD check before
   queuing the piptar upload or writing `.valid.windmill`. A bad install
   never becomes the source of a broken tar in the object store.

3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
   for upload, so we never push an unfinalized archive (no end-of-archive
   marker) to the object store.

Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(main): release 1.699.0 (#9082)

* chore(main): release 1.699.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat(cli): auto-infer args for `wmill app push` (#9091)

Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`)
with no args. The local path defaults to CWD, and the remote path is
derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/
`__app`/`__raw_app` suffixes stripped. Either, both, or neither
positional argument can be passed.

Also resolves `file_path` against the user's original CWD before
`resolveWorkspace` may chdir to the wmill.yaml root, so a relative
`file_path` argument is interpreted from where the user invoked the
command (previously it could resolve against the wrong directory).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* all

* fix(pipeline): live-update graph for annotations and body assets

* fix(pipeline): persist draft body edits across node switches

* fix(pipeline): persist live writes per draft to keep output node fresh after switch

* feat(pipeline): animate graph edges only while a runnable is executing

* feat(pipeline): add run button on script nodes + recomputing hint on preview

* feat(pipeline): compact preview layout, two-way Test/Run sync

* fix(pipeline): test button cross-browser placement (no overflow trick)

* style(log-viewer): replace took/mem-peak labels with timer/cpu icons

* style(log-viewer): hyphenate Auto-scroll label and prevent wrapping

* style(log-viewer): lowercase auto-scroll label, force vertical scrollbar

* style(log-viewer): force horizontal scrollbar instead of vertical

* fix(log-viewer): scope overflow-x to top bar so pre doesn't drive panel width

* fix(pipeline): overlay live body-asset writes for persisted scripts too

* fix(pipeline): persist inferred body assets at save so edges survive page reload

* fix(pipeline): snapshot live draft writes at persist time so they survive reload

* fix(pipeline): keep inferred body writes on the canvas across selection changes

* fix(pipeline): untrack inferredWrites cache mutation to break effect loop

* fix(pipeline): refetch asset graph after persisted-script save

* feat(pipeline): optional AI prompt when creating a pipeline script

* all

* all

* test: cover asset-trigger dispatch end-to-end through worker

* feat(pipeline): split-button Test with optional downstream cascade

* feat(pipeline): cascade option on graph Run + match button heights

* style(pipeline): match caret bg/text to Test button's accent-secondary

* feat(pipeline): split Run pill on graph node exposes cascade option

* feat: live run activity + status badges in pipeline asset graph

- folder-scoped queue poll lights up the downstream asset-trigger
  cascade (not just the launched script); zero requests at rest,
  catch-up for fast hops, auto-disarm when idle
- per-runnable node badge: last-run status + session run count
- animate unsaved/live-parsed edges (was unconditionally suppressed)
- background-pane click no longer clears selection
- run-bridge guarded so node selection/save no longer triggers a test

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: live activity log, optimistic badges, node-avoiding graph edges

- collapsible folder activity log (PipelineEventLog): live job feed,
  polls only while open/active, slow idle cadence, capped + pruned
- composable: observe mode + events list + run-count anchored to
  graph-open time (pre-existing history excluded)
- optimistic node badge: launched script shows running instantly via
  the zero-latency activeRunnable hint, keeps the polled run count
- activity pane height capped (min(18rem,40vh)) then scrolls
- route asset-graph edges through sugiyama-computed waypoints so they
  go around nodes instead of under them; bezier fallback for
  adjacent-layer / draft-overlay edges

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: prefetch all folder script assets so graph is stable on load

On pipeline load, eagerly infer body assets for every persisted folder
script and seed the existing inferredWritesByPath overlay, instead of
only filling it when a node is selected. Scripts whose persisted asset
rows are missing (e.g. object-form writeS3File) now have their edges
from first paint, so clicking a node no longer re-layouts the graph.
One-shot per (workspace, base-graph) load, untracked map reads,
generation-cancelled, pool-capped fetches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* perf: guard no-op poll re-layout; dedupe write-asset extraction

- skip reactive ids/states/events reassignment when unchanged, so an
  idle poll tick no longer re-runs the full sugiyama layout every 3-6s
- bound countedJobIds (rebuilt from eventsById in lockstep with prune)
- extract shared extractWrites() helper, replacing 4 copy-pasted
  write-asset filter/map blocks in the pipeline page
- compute activeRunnable node-id once, reuse for the active-edge set
  and the optimistic badge (flattened ternary); trim narrating docs

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: live read-lineage overlay for inferred body assets

Renaming e.g. duckdb read_parquet('s3://...') / loadS3File now updates
the asset->reader edge live instead of only after Save re-derives the
persisted asset rows.

- extractReads() (+ shared refsByAccess) mirroring extractWrites
- inferredReadsByPath sticky cache, filled by handleAssetsChange and
  the load prefetch alongside writes
- replace the write-only overlay loop with one overlayLineage(map,
  access) helper invoked for both 'w' and 'r' (net DRY)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: detect S3 assets passed as SDK object arg in ts parser

Mirrors merged PR #9181 so feat/asset-graph-view is self-contained
(local origin/main is stale and lacks it). Object/{ s3, storage }
form of writeS3File/loadS3File is now detected, not only the bare
s3:// string literal.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: regenerate wasm Cargo.lock + frontend package-lock

Lockfile churn from local wasm-pack (asset target) + npm operations
during the asset-graph work. No source/dependency-intent change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: revert to bezier graph edges; add parsing-assets hint

The sugiyama-waypoint routing looked worse than the original; revert
AssetGraphEdge/assetGraphLayout to the pre-routing bezier logic (same
as the flow editor's BaseEdge) and drop the now-unused route plumbing
from the canvas. Add a small 'Parsing assets…' hint shown while the
load-time prefetch sweep is still inferring folder scripts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: extract pure resolveGraph merge + unit tests

Move the ~230-line graphWithDraft precedence/merge (base < session-
inferred < draft-seeded < open-script-live, +read/write/annotation
overlays, +dedup) out of the 1648-line route into a pure, testable
resolveGraph() module; the route's graphWithDraft is now a thin
$derived. Behaviour extracted verbatim. 10 unit tests cover the
precedence matrix. Phase 1 of the state/render split.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: graph controls top-right, lift minimap, hide Save when unchanged

Controls -> top-right horizontal, no lock toggle; MiniMap !mb-10 so
it clears the activity bar; hide the per-script Save button when the
script is already at its latest save point (drafts still show Create).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: scope runtime-asset prune by id to spare static lineage rows

prune_runtime_assets deleted by (workspace_id, path, kind) tuple, so
trimming surplus usage_kind='job' rows for an s3 path also wiped the
static usage_kind='script'/'flow' producer rows for the same path —
silently breaking the asset-trigger cascade (fetch_producer_writes
found no writes; downstream never dispatched; required band-aid
re-syncs). Delete the surplus job rows by id instead; the inner query
is already scoped to usage_kind='job'.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: don't re-pulse already-running jobs after they finish

The catch-up pulse re-added a completed job to the active set if its
start was within the (lagging) lookback window — even one we'd already
animated the whole time it ran — keeping its edges lit ~a poll
interval past completion (~5s after a 3.5s test). Track job ids seen
in-flight and skip the pulse for them; it still fires for hops whose
whole lifetime fell between two polls. Bound the set in lockstep with
eventsById; cleared on dispose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: don't catch-up-pulse the runnable launched from the graph

If the poll never sampled a launched run's in-flight window, the
catch-up pulse re-flashed its edges one tick after it correctly
stopped (the page already animated it zero-latency via activeRunnable).
arm(launchedId) records the launched runnable id; catch-up skips it.
Cascade hops (other ids) still pulse. launchedIds cleared on stop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style: nudge graph controls left to clear panel toggle

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: partition value resolver + asset-cascade propagation

windmill-common/partition: pure resolver — time kinds (tz/format/start
anchor) + dynamic $.a.b JSONPath; 9 unit tests. asset_dispatch:
read the producer's resolved partition and thread it into every
cascaded subscriber's args + trigger.partition, so a chain resolves
once at the top. No migration (cascade needs no spec lookup). Stage
1+3 of pipeline partition runtime; run-start resolution is Stage 2.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: show args form in compact pipeline preview when script has inputs

AssetGraphDetailsPane keeps the compact (hideArgs) preview but, via a new
previewPanel.argsAboveLogs flag, renders a compact SchemaForm between the
floating Test button and the logs/result panel when the script declares
inputs (e.g. a partitioned script needing a `partition` arg). The preview
pane also grows ~18pts so the args form doesn't shrink logs/result.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: parser join-mode (`// trigger all`) + script_trigger.join_all

Stage A: JoinMode{Any(default),All} + `// trigger any|all` directive in
parse_pipeline_annotations; TriggerSpec::is_partition_bearing() (path
contains {partition}); join_mode threaded through all 4 asset-parser
crates (ts/py/sql/yaml). Stage B: reversible migration adds
script_trigger.join_all; insert_script_trigger writes it; deploy path
sets it from the parsed annotation. No reader yet (AND-join dispatch is
the next stage) so runtime behaviour is unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: resolve pipeline partition at job execution time

Stage C: in handle_code_execution_job, once the script content is loaded,
parse the // partitioned annotation (free here) and resolve the concrete
partition once — schedule fire-time (scheduled_for anchor, not wall-clock)
for time kinds, triggering payload for dynamic. The value is injected
into the in-memory args the body sees (via a shadowed job clone) and
persisted back to v2_job.args so dispatch_asset_triggers propagates the
same value down the cascade. Already-set (explicit/backfill/cascade)
partitions are never re-resolved (run identity immutable); unresolvable
partitioned runs fail with a clear error. Integration test exercises the
full worker loop + cascade propagation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: AND-join barrier for partitioned pipeline subscribers

Stage D: a // trigger all subscriber no longer fires on any input. New
join_pending_inputs slot table keyed (workspace, subscriber, partition);
fetch_subscribers now returns join_all and the dispatch loop records each
partition-bearing input arrival, pushing the subscriber once only when
every partition-bearing input it declares is present for that partition.
Per-partition slots, cleared on fire (re-accumulate, no double-fire),
skew-immune (unlike debounce). Case-3 guard: an unpartitioned producer or
a reference (non-{partition}) input never fires a partitioned join.
Integration test covers wait/fire/isolation/no-double-fire.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: opt-in // debounce for asset-cascade subscribers (parser + schema)

Stage E1+E2. Parser: script-level // debounce <dur> + per-// on
debounce=<dur> override (edge wins, else script default, else none =
fan-out, unchanged); TriggerSpec::Asset carries the per-edge override;
split_trailing_kv_opts separates the ref from trailing key=val opts.
Schema/deploy: reversible migration adds script_trigger.debounce_s;
parse_duration_secs (bare int or <n>s|m|h|d, fail-safe on garbage)
resolves the effective per-edge window at deploy and writes it per row.
No reader yet (dispatch wiring is E3) so runtime is unchanged. New unit
tests for the parser directive and duration parsing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: apply opt-in debounce to asset-cascade subscriber dispatch

Stage E3. fetch_subscribers now also returns debounce_s; push_subscriber
builds real DebouncingSettings (delay + a (subscriber, partition) key,
so distinct partitions never collapse and latest-in-window falls out)
instead of ::default() when the edge opted in. Default stays no-debounce
(fan-out — the prior deliberate behaviour, now overridable rather than
reversed). Wiring test asserts the dispatched job carries the configured
window/key and an undebounced edge carries none.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: atomic AND-join gate + preserve resolved partition; drop scratch artifacts

Addresses local-review findings before PR:
- P1: record_and_check_join_slot was a non-atomic check-then-act on a
  pooled connection; concurrent completion of a subscriber's last two
  partition-bearing inputs on different workers could double-dispatch.
  Now one transaction guarded by a tx-scoped advisory lock keyed on
  (workspace, subscriber, partition) so the gate fires exactly once.
- P2: the preprocessed-args overwrite in result_processor replaced args
  wholesale, dropping a partition resolved by resolve_partition_for_job;
  the UPDATE now preserves an existing persisted partition key.
- P2: gate resolve_partition_for_job on a cheap code.contains check so
  non-pipeline script jobs skip the annotation scan on the hot path.
- P2: remove 40 scratch screenshot PNGs, a flicker-debug script and a
  local scheduler lock accidentally committed; gitignore the lock.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: AND-join fires once under concurrent upstream completion

Regression for the check-then-act race fixed by the advisory-locked
transactional gate: releases N producer dispatches simultaneously via a
barrier and asserts the AND subscriber is pushed exactly once and the
slot is cleared. The invariant holds for the correct gate regardless of
interleaving; a non-atomic regression fails it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test: fuller partitioned join + multi-hop pipeline coverage

Exercises a complex pipeline combining options end to end: two
partitioned producers fanning into a // trigger all join, then a
multi-hop downstream chain. Asserts the resolved partition propagates
unchanged at every hop, chain depth increments per hop, the AND barrier
fires exactly once, and a second partition opens an independent slot
with no cross-partition bleed across the whole graph.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* refactor: simplify pipeline code per review (dedup, single-parse, constant)

- ParseAssetsOutput::new() collapses the 6-line annotation copy-paste
  across the 4 asset-parser crates to one call site.
- asset_dispatch: parse the cascade trigger object once and pass it to
  the depth/partition readers instead of deserializing it twice; add a
  TRIGGER_ARG constant for the previously stringly-typed key (3 sites).
- scripts deploy: drop a redundant debounce_default clone.
No behavior change; 29 parser + 6 dispatch integration tests green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat: reap abandoned AND-join slots after a TTL (default 60d, per-slot)

join_pending_inputs slots are normally cleared when the join fires;
partial slots whose inputs never all arrive (upstream removed/renamed,
one-off dynamic partition key, permanent skew) would otherwise leak.
windmill_queue::asset_dispatch::reap_stale_join_slots, called from the
monitor's delete_expired_items loop, deletes a (workspace, subscriber,
partition) slot only when its MOST RECENT row is older than
JOIN_SLOT_TTL_SECS (60d) — per-slot, never per-row, so a legitimately
slow join is not corrupted mid-accumulation. Conservative default;
per-join configurable TTL via the annotation is a planned follow-up.
Test covers stale-reaped / fresh-kept / mixed-slot-kept.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* update

* feat: path-less native trigger markers + missing-trigger placeholder

* feat: pipeline // tag and // retry annotations + dispatch_event log

* fix: derive test-pane min from split-axis dimension (height in bottom layout)

* feat: show last run logs/result when a script node is selected

* fix: backfill asset rows from script.assets for pre-feature scripts

* feat: job-id link + dispatch popover above script log/result

* style: drop 'dispatched' label, keep just the check icon

* fix: drop tag picker from pipeline script editor (set via // tag annotation)

* Nicer UI

* refactor: move google ai proxy handling to windmill-ai (#9260)

* refactor: add ai proxy execution mode

* refactor: move google ai proxy handling

* refactor: share google ai request building

* fix: early return should consider failure_module result (#9241)

* fix(flows): flag noLogs jobs and lazily resolve them in log panel (#9099)

* fix(flows): flag noLogs jobs and lazily resolve them in log panel

* fix appending to flag

* fix: preserve WM_LOGS_SKIPPED sentinel on SSE/replay completion

pickMoreCompleteLogs resolved both sentinel and undefined to '', so the
SSE completion event (whose job field is fetched .without_logs()) would
clobber the sentinel placed by flagSkippedLogs. The module log panel
then saw '' instead of the sentinel, defeating the lazy-resolve path.

Also wire onLogsResolved on the OutputPickerInner inline LogViewer so a
lazy resolve writes back to flowStateStore.previewLogs, matching
ModulePreviewResultViewer and avoiding repeated fetches on remount.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(main): release 1.705.0 (#9229)

* chore(main): release 1.705.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* chore: add playwright mcp for frontend verification (#9269)

* feat: CLI datatable serve / psql (#9267)

* feat(cli): add datatable list and run commands

* feat(cli): render datatable query results as a table

* feat(cli): serve datatables as a postgres-wire endpoint

* feat(cli): add 'datatable psql' to launch psql against the proxy

* feat(cli): route datatable serve by client-supplied database name

* override database list + password option

* fix: support extended queries in datatable serve

* fix: correct cloud size threshold log and parse CLI descriptions with parens/trailing comma

* refactor: extract raw_output envelope encoding into pg_raw_output module

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>

* oom_adj nit

* feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting (#9271)

* feat: add UV_PYTHON_INSTALL_MIRROR env and instance setting

Allows operators to point `uv python install` at a private mirror of the
python-build-standalone releases. Configurable via the
`UV_PYTHON_INSTALL_MIRROR` env var or the `uv_python_install_mirror`
instance setting, with the env var as the boot fallback and the instance
setting taking precedence at reload.

Fixes WIN-1966

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: hoist uv_python_install_mirror binding above sandboxing branch

The non-sandboxed uv pip install branch referenced a binding that was
only declared inside the sandboxed branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: neutral placeholder for uv_python_install_mirror

The previous placeholder was the default public URL the setting is meant
to redirect away from. A neutral example mirror URL is clearer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(indexer): tell admins when ingress routes search to wrong pod (#9274)

* [ee] fix(indexer): tell admins when ingress routes search to wrong pod

When the IndexReader is absent on the pod handling a search request but
another pod is actively holding the indexer lock, the EE handler now
returns a tailored error pointing at the ingress/load-balancer
configuration instead of the generic "indexer not running" message.

The indexer status endpoint reads the DB lock so it reports "running"
from any pod, but search endpoints need the in-memory IndexReader that
only exists on the lock holder. In multi-replica deployments this looks
like the indexer is healthy but every search 404s.

Companion: windmill-labs/windmill-ee-private#TBD

Fixes WIN-1968.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817

This commit updates the EE repository reference after PR #586 was merged in windmill-ee-private.

Previous ee-repo-ref: 7dd43d1850813071cc18ba49ba090583e7321f4b

New ee-repo-ref: eb18d7b4c0e37fea3f6e1e2cc44e0fddd74ff817

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* feat(cli): add `wmill init prompts` and custom override slot (#9266)

* feat(cli): add `wmill init prompts` and custom override slot

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): replace init prompts with refresh prompts + AGENTS.md/AGENTS.cli.md split

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): dedupe claude skills via @-includes and add prompts freshness check

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): drop migration-choice flags from `refresh prompts`

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(cli): add 'Running and previewing local changes' section to AGENTS.cli.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): write full skill content to .claude/, drop @-include wrapper

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(cli): reconcile CLAUDE.md the same way as AGENTS.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): address PR review nits — argv parsing, lazy import, comment detection, error propagation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add yolo mode for ai chat tools (#9258)

* feat: add yolo mode for ai chat tools

* nit

* fix: align chat footer controls

* feat: add ai chat autonomy modes

* feat: add autonomy mode dropdown

* fix: highlight yolo autonomy icon

* fix: auto accept flow edits

* fix: hide unsupported autonomy modes

* fix: handle auto-accept flow editor races

* fix(debugger): add non-root user support to Dockerfile (#9277)

Mirrors the main Windmill Dockerfile pattern: creates a windmill user
(UID/GID 1000) and makes cache/work directories world-writable so the
image runs cleanly under Kubernetes securityContext.runAsNonRoot or
runAsUser: 1000 without permission errors on Bun, pip, or windmill
cache writes.

Fixes WIN-1969

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path (#9276)

* fix(ai): enforce RLS and scope check on user-supplied X-Resource-Path

The AI proxy handler accepts an X-Resource-Path header to override the
configured workspace AI provider. When supplied, the handler loaded the
resource value from the resource table using the root DB pool with no
resources:read scope check, so any authenticated workspace user could
point X-Resource-Path at a restricted AI resource (e.g. one in a folder
they cannot read) and the proxy would use that resource's provider
credentials for the outbound AI request.

For user-supplied resource paths, now require resources:read:{path}
scope and fetch the resource through user_db.begin(&authed) so RLS
enforces the same folder/group boundary as the resource API. The RLS-
scoped $var: resolution stays in place as defense in depth. The
admin-configured workspace/instance ai_config path is unchanged.

Fixes WIN-1971

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(ai): regression test for X-Resource-Path RLS enforcement

Cover all four cases:
- non-admin pointing X-Resource-Path at a restricted resource is rejected
- non-admin pointing it at a resource they own still works
- admin can point it at any resource
- workspace-configured proxy flow (no X-Resource-Path) is unchanged

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat: add userdraft listing primitives (#9268)

* feat: add userdraft listing primitives

* fix: cancel stale userdraft discard writes

* docs: remove global ai userdraft plan

* feat(nsjail): optional disk-backed /tmp via instance setting (#9272)

* feat(nsjail): optional disk-backed /tmp via instance setting

* test(nsjail): unit-test tmp mount resolver and narrow visibility

* refactor(nsjail): switch tmp backing to select + conditional UI

* ui(nsjail): make tmpfs the visible default in /tmp backing select

* fix(nsjail): refuse preexisting jail_tmp to block symlink escape

* fix(nsjail): allow jail_tmp reuse on sequential nsjail calls

Codex flagged that python/ruby/rust executors invoke nsjail twice per
job_dir (install then run). The previous resolver treated any preexisting
jail_tmp as hostile and silently fell back to tmpfs on the second call,
so disk-backed mode never reached the main script run for those langs.

Use symlink_metadata().is_dir() to distinguish a real directory left by
an earlier call in the same job_dir (safe to reuse) from a symlink or
other entity (still refused, as the codebase-tar escape requires).

Also loosen the frontend visibility predicate: only hide nsjail settings
when job_isolation is explicitly 'none' or 'unshare', so deployments
that enable nsjail via DISABLE_NSJAIL=false with no DB setting can
still see the controls.

* chore(main): release 1.706.0 (#9270)

* chore(main): release 1.706.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix(nsjail): gate unix-symlink test behind cfg(unix) for Windows build (#9280)

The disk_backed_refuses_preexisting_symlink_at_jail_tmp test calls
std::os::unix::fs::symlink directly, which doesn't exist on Windows
targets. Without a cfg gate, `cargo check --tests` fails on Windows
with E0433. Other symlink call sites in this crate (php_executor,
bun_executor, rust_executor, etc.) already follow this pattern.

Fixes WIN-1972

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Reduce slim image vulnerability surface (#9279)

* Reduce slim image vulnerability surface

* chore(docker): drop apt-get upgrade -y from slim images

apt-get upgrade hurts build reproducibility (same Dockerfile + same
commit at different times produces divergent images) and trips hadolint
DL3005. The freshness it buys is dominated by simply rebuilding against
the periodically-refreshed debian:bookworm-slim base image.

The --no-install-recommends and apt-list cleanup wins are kept.

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

* fix(git-sync): bump to hub/28234 with stateless gpg.program wrapper (WIN-1974) (#9282)

* fix(git-sync): revert LATEST_GIT_SYNC_SCRIPT_PATH to hub/28230 to restore GPG-signed deploys (WIN-1974)

hub/28231 (PR #9230) is the "thin" script that hands the actual `git commit`
to the CLI's hidden `sync git-deploy`. The hub script still does the GPG
setup (import key into a fresh GNUPGHOME, dummy `gpg -bsau` to warm the
agent passphrase cache, then `git config user.signingkey` + `commit.gpgsign`
locally), but the commit no longer runs in the same `git_push` flow — it
runs minutes later inside the CLI after workspace API resolution, zip pull,
file extraction, and lockfile autofill. By the time the spawned `git commit`
asks gpg-agent for the cached passphrase, the cache state is no longer
reliable (or the spawned `gpg` ends up talking to a fresh agent), so signing
fails non-interactively with `gpg failed to sign the data`.

hub/28230 is hub/28217's in-script logic rebuilt with windmill-cli@1.703.3:
the GPG setup and the in-script `sh_run("git commit ...")` happen back-to-back
in `git_push`, so the cache is always fresh. It preserves wm_deploy / fork
branch behavior, the EE deployment-callback `main()` signature is unchanged,
and the only min-version check in EE (`is_script_meets_min_version(28103)`)
is comfortably below 28230 — so this revert is safe.

Forward fix (separate PR): publish a new thin script that, alongside the
existing GPG setup, writes a `gpg.program` wrapper using `--pinentry-mode
loopback --passphrase-file` so signing is independent of the agent's cache
state. Re-bump past 28231 then.

Fixes WIN-1974

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(git-sync): check in source-of-truth for the next hub script (gpg.program wrapper)

This is the script that will be published to hub.windmill.dev once verified
on a customer GPG-signed deploy. It replaces hub/28231's agent-cache
pre-warm (`gpg -bsau` with --passphrase) with a stateless gpg.program
wrapper + chmod-600 passphrase file. Every git-invoked gpg call goes
through the wrapper, which always uses --pinentry-mode loopback (and
--passphrase-file when a passphrase exists). Signing no longer depends on
gpg-agent having a cached passphrase by the time the CLI's `git commit`
runs — which closes WIN-1974.

Not wired in yet: LATEST_GIT_SYNC_SCRIPT_PATH stays on hub/28230 until this
script is uploaded and the new hub id is known. This file is checked in so
the diff is reviewable, future bumps have a source of truth, and a CLI
regression test can `cat` it for fixture parity.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(frontend): skip format/pattern validation for $var/$res/$jsonvar references in ArgInput

A resource field with a `pattern` constraint (e.g. the gpg_key.private_key
field, whose pattern enforces a `-----BEGIN PGP PRIVATE KEY BLOCK-----`
prefix) rejects values like `$var:u/me/gpg-private-key` with an "invalid
format" error in the resource editor — even though `$var:`/`$res:`/`$jsonvar:`
are placeholders the backend resolves at runtime, not the actual string
that needs to match the regex.

Bail out of all format/pattern checks (email, ipv4, ipv6, uuid, custom
pattern) when the value is one of these references. Required/numeric
bounds/array checks still apply since they're shape-level, not regex.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(git-sync): bump LATEST_GIT_SYNC_SCRIPT_PATH to hub/28234 (gpg.program-wrapper fix)

hub/28234 is the forward fix for WIN-1974: replaces hub/28231's agent-cache
pre-warm (which became stale by the time the CLI's `git commit` ran) with
a stateless `gpg.program` wrapper that uses `--pinentry-mode loopback`
(and `--passphrase-file` when a passphrase exists) on every gpg invocation.
Bundled CLI is windmill-cli@1.705.0.

Verified via reproducer at /tmp/git-sync-diff/test-gpg-fix.sh: deliberately
killing gpg-agent between GPG setup and `git commit` reproduces the
customer's `gpg failed to sign the data` error verbatim under the old
flow, and the wrapper signs through it. Holds for passphrase-protected
keys, split-subkey [C]+[S] layouts, and unprotected keys.

Drops the local source-of-truth copy (`hub-scripts/`) — hub is canonical
now that 28234 is published.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(git-sync): drop verbose comment above LATEST_GIT_SYNC_SCRIPT_PATH

The git history (this PR) carries the why; the constant name + value carry
the what.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(cli): wmill sync git-deploy stops committing; caller owns commit+push (#9284)

Single contract for the deployment-callback path: the CLI does branch
checkout + pull, the caller (hub script in production, test in test)
does git add + commit + push. This restores the WIN-1974 invariant —
GPG setup and `git commit` run back-to-back in the same process, so
the agent's pre-warmed passphrase cache is still warm at sign time —
without needing a `--skip-commit` flag for the hub case and a default
"also-commit" for everything else. Same behavior in every call site.

Changes:
  - sync.ts: drop the gitSyncDeployPush call from pull()'s deploy path
    (both the onlyCreateBranch fast-return and the post-pull commit).
    `gitSyncDeployPush` stays exported for any caller that wants the
    same commit/push semantics — just not invoked by the CLI subcommand.
  - gitsync_promotion.test.ts: e2e test now does its own git add +
    commit + push after `wmill sync git-deploy`, mirroring what the
    hub script does in production. Same regression coverage
    (wm_deploy branch created in Case A, main untouched; main updated
    in Case B, no new wm_deploy).

CLI typecheck unchanged (two pre-existing TarAsZip errors at lines
2578/3307, present before this PR). All 743 unit tests still pass.

The accompanying hub script (option-C — CLI for branch+pull, script
for commit+push) lives at /tmp/git-sync-diff/sync-script-to-git-repo-windmill.option-C.ts.
Once published, a follow-up bumps LATEST_GIT_SYNC_SCRIPT_PATH to its id.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* bump git sync to 28236

* fix: fork compare visibility for non-admins and stale-token superadmins (#9283)

* fix: use fork-scoped authed for fork visibility in compare_workspaces

* test: add EE end-to-end repro for fork rename visibility

* chore: restore concurrency_locks sqlx cache lost in cleanup

* test: add regression for stale-superadmin-token fork visibility bug

* chore: update sqlx cache for new test queries

* chore(main): release 1.706.1 (#9281)

* chore(main): release 1.706.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* feat: add wmill job rerun subcommand (#9275)

* feat: add wmill job rerun subcommand

* feat: add wmill job restart subcommand for flow restart-at-step

* chore(system_prompts): point plugin skills sync at plugins/windmill/ (#9287)

* chore(system_prompts): point plugin skills sync at plugins/windmill/

The plugin checkout's plugin folder is being renamed from
`plugins/windmill-code-plugin/` to `plugins/windmill/` to shorten the
slash-command namespace and align with the matching Cursor plugin
layout.

Paired with windmill-labs/windmill-claude-plugin#8. That PR must merge
first so the next sync run finds the new folder.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(system_prompts): update plugin-dir example to plugins/windmill

Co-authored-by: centdix <centdix@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>

* fix(cli): wmill sync pull updates wmill-lock.yaml for raw apps (#9289)

* fix: flow recording teardown crash + rename package to @windmill-labs/components (#9288)

* fix: guard against null recording during FlowRecordingReplay teardown

Navigating away from a flow recording inside a workspace file-tree view
threw `TypeError: Cannot read properties of null (reading 'flow')` from
FlowGraphViewer once during the teardown tick.

Svelte 5 compiles child component props as live getters that close over
`$$props.recording.flow`. When `recording` flips to null on the parent's
navigation, an outer `{#if !recording?.flow}` doesn't stop those getters
from firing one more time as derived effects re-evaluate before the
unmount lands — so the getter dereferences null and throws.

Fix at the two layers where the deref actually happens:

- FlowRecordingReplay: use `recording?.flow` at the binding sites
  (FlowViewer + graph-snippet FlowGraphViewer) so the compiler emits an
  optional-chained getter, and guard the snippet branch with
  `{:else if recording?.flow}` so it doesn't mount when there's nothing
  to show.
- FlowGraphViewer: finish the optional chaining the rest of the file
  already used everywhere else (`flow?.value?.skip_expr`,
  `flow?.value?.cache_ttl`, `flow?.schema`). When the upstream
  binding returns undefined during teardown, the graph degrades to an
  empty frame instead of crashing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rename package to @windmill-labs/components

- frontend/package.json: rename `windmill-components` → `@windmill-labs/components`
- frontend/publish.sh: drop the in-place sed rename dance; the checked-in name now matches what's published, so `npm run package && npm publish` is enough
- frontend/package-lock.json, system_prompts/auto-generated/prompts.d.ts: regenerated by `npm run package` under the new name

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* default script name

* save logic

* Keyboard nav

* finish keynav

* nits

* CI fix

* nit stop propagation

* Merge branch 'main' into feat/asset-graph-view

* commit

* update

* fix: cropped save button on small screens

* progress

* managed scheduled removed

* all

* progress

* feat: add data upload pipeline trigger with auto S3 picker

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: avoid pane editor remount flicker when deploying a pipeline draft

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: show only the edited script's I/O in the asset graph, not the saved version's

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: derive script asset rows server-side at deploy

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: shared fixture corpus keeps annotation parsers in parity

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: dev-run draft pipeline chains, live badges, deploy drift warning

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor: ungate cascade producers, squash pipeline migrations

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: drop committed cli-sync fixtures and stray screenshots

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: show skip-asset-dispatch flag as badge instead of args row

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: pipeline view mode default with activity feed, drafts overlay chip

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: treat DROP TABLE as table-level write in sql asset parser

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: wmill datatable create + actionable sql extension error

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: ephemeral data-pipelines demo sync repo zip for handoff

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: wmill pipeline list/show renders the asset DAG in the terminal

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* nits

* nits

* nits

* nits

* fix: defer draft persist-back past the batch so discard sticks first click

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: band-reserving tidy-tree asset graph layout with join breakpoints

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: route skip-layer and long graph edges around occupied columns

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: seed s3 template outputs with canonical leading-slash paths

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* all

* feat: bundle data-pipeline drafts into the DB-backed user draft system

Pipeline drafts were browser-only (localStorage `pipeline-<folder>`), so they
didn't sync across devices, weren't server-visible, and never showed in the
drafts list. Store them instead as one per-user `draft` row of a new
`data_pipeline` kind, keyed at the folder (`f/<folder>/data_pipeline`), holding
the same `{ drafts, activeDraftPath }` bundle.

Stage 1 — backend kind: add `data_pipeline` to DRAFT_KIND (migration) and
`UserDraftItemKind` (deployed_table=None, private). The list/update handlers
and folder-path access check already cover a backing-table-less kind.

Stage 2 — sync: add `GET /drafts/get_own/{kind}/{path}` so an editor with no
deployed-overlay GET can load its own draft. The pipeline page now hydrates
from the DB on mount (one-time localStorage import for in-flight drafts) and
persists via UserDraftDbSyncer (debounce + optimistic-concurrency), keeping a
localStorage crash mirror.

Stage 3 — surface: the drafts review page renders the bundle as a "pipeline"
row that opens `/pipeline/<folder>` (open-only; excluded from bulk deploy).

Verified end-to-end in-browser: DB-seeded draft hydrates to "Edit (1)", edits
persist back, and the row shows with Open pipeline / Discard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: pipeline Activity panel grouping, run↔graph highlight, deploy-conflict handling

Activity panel (view mode):
- Group cascade runs by the connected component of the asset-dispatch graph
  (new GET /jobs/asset_dispatch_edges over the dispatch_event table, incl.
  join_pending inputs), headed by the earliest originating run + its trigger,
  with a "+N" chip for joins fed by multiple triggers.
- Success/failure count histogram with drag-to-filter brushing, an always-on
  time axis + per-bar tooltips, a Reset, and Last hour/24h/48h/7/30/90d ranges.
- Node run-count/status badges now derive from the same merged historic+live
  events the panel shows (previously session-only).

Run ↔ graph highlight:
- Hovering a run row (or a group header → the whole cascade) rings the
  node(s), animates their incident edges, and borders the adjacent assets in
  the edge hue (blue write / gray read); expanding a run pins a soft-blue ring.
- Switching edit→view re-surfaces the Activity feed.

Deploy:
- Live-content autosave for the open pipeline draft + an autosave indicator.
- Re-saving a script now chains off the hash just created instead of a stale
  parent_hash (fixes the "lineage must be linear" error on a second save), and
  a genuine concurrent deploy opens a keep-mine / view-latest conflict modal.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: pipeline editor badge requires asset-parse, not just main-function parse

A pipeline script's asset lineage is load-bearing — a deploy that can't parse
assets silently records no edges. The editor "parsable" dot only reflected
inferArgs (the main function), so a body the asset parser rejects (e.g. a
trailing `/////` in DuckDB) still showed green and deployed with empty lineage.

ScriptEditor gains `requireValidAssets` (set by the pipeline pane); when on, the
EditorBar badge is green only if BOTH the main function and inferAssets parse,
with the tooltip distinguishing "Main function not parsable" / "Assets not
parsable" / "Parsable".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: route asset-graph edges around nodes that sit in their path

Edges could draw straight through an unrelated node (a join fan-out or long
cross-component edge), making it ambiguous whether that node shared the input.
AssetGraphEdge only saw its own endpoints, so it could only detour the
near-vertical same-column skip case.

The canvas now (once per layout, O(edges × nodes) — no per-frame cost) samples
each edge's straight run against every non-incident node center and, on a
crossing, passes a clear gutter lane to the edge via `data.detourX`;
AssetGraphEdge routes the rounded-orthogonal detour through it. Verified: 0
edge↔node box crossings on the orders pipeline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: deploy pipeline drafts with freshly-inferred assets, not a stale snapshot

"Save all" spread `...draft.script` into createScript, which carries a `assets`
snapshot that isn't refreshed when the body is edited. So a renamed/removed
output (e.g. an old `CREATE TABLE exciting_en32z9` later changed to
`exciting_880909`) was re-deployed as a phantom write edge and lingered as an
orphan asset on the graph — shown with no producer, and shifting position on
click as the graph re-derived.

saveDraft now re-runs inferAssets on the current body and passes the result as
`assets`, overriding the snapshot — mirroring the per-pane save. The backend
clears+reinserts from the sent set, so a re-deploy drops the stale rows.
Verified: deploying with the fresh asset set removes the orphan from the graph.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: collect upstream reads from CTAS and CREATE VIEW in SQL asset parser

`CREATE TABLE x AS SELECT … FROM y` (and `CREATE VIEW`) recorded only the
write to x — the source read of y was silently dropped. Table-level reads are
gathered in the `Statement::Query` arm via handle_table_with_joins; the generic
table-factor visitor only picks up read-functions and string literals, not
plain `FROM <table>` references. The AS-query of a CTAS isn't a
`Statement::Query`, so its FROM tables were never walked. On the pipeline
canvas this meant a `datatable://…` upstream consumed by a CTAS step showed no
read node/edge — the step looked like it produced its output from nothing.

Factor the Query arm's read collection into handle_query_reads and call it from
the CreateTable (when it has an AS-query) and CreateView arms, balancing the
cte_name_stack push in post_visit_statement. Updated the drop_then_create test
(which had pinned the old drop-the-read behavior) and added CTAS + CREATE VIEW
read coverage. Verified against the rebuilt asset wasm: the live editor now
infers the read.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* update

* updates

* refactor: dedup asset-graph code, squash migrations, drop artifacts

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf: gate asset dispatch on a cached per-workspace producer set

Cache the producer-path→writes map per workspace and invalidate it from the asset-clear paths via the notify_event polling system, so a top-level script/preview completion that isn't an asset producer costs an in-memory lookup instead of a per-completion query.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: remove dead unquote fn that failed backend check under -D warnings

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: green the frontend check (pin published wasm-asset, fix type errors)

Pin windmill-parser-wasm-asset to the published 1.728.1 (was a file: link to a gitignored, CI-unbuilt pkg-asset). Exclude test files from svelte-check (the parity test reads a backend fixture via node:fs, which the browser app tsconfig has no @types/node for; vitest still runs them). Fix pre-existing branch type errors: drop the unsupported 2nd getScriptByPath arg, cast script.schema to Schema for inferArgs, coerce has_preprocessor to a definite boolean, and wrap the cancelJob handler so it isn't possibly-undefined.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: move pipeline partition resolution to ee-private (free-CE)

Partition resolution becomes a private module (partition_ee in windmill-ee-private, hidden from the public repo) with an OSS no-op fallback (partition_oss); call sites resolve via the aliased windmill_common::partition. Not enterprise-gated — free to run in CE. Bumps ee-repo-ref to the ee branch carrying partition_ee. Verified building in default, private, and private,enterprise (offline). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: move asset-cascade join/debounce/retry to ee-private (free-CE)

Join barrier, debounce, and retry become the private windmill_queue::cascade module (cascade_ee in windmill-ee-private); OSS gets cascade_oss no-op fallbacks (plain OR fan-out). Core cascade stays public. Bumps ee-repo-ref. Verified default/private/private,enterprise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: skeleton enterprise pipeline freshness + backfill (TODO, ee-private)

Gated windmill_common::pipeline_advanced (private; pipeline_advanced_ee) with OSS fallback; entry points return a clear not-implemented error. Deploy surfaces a TODO when a script declares // freshness. Bumps ee-repo-ref. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: repair asset_trigger_dispatch test after cascade carve-out + cache its queries

Stage-2 moved reap_stale_join_slots to windmill_queue::cascade; update the integration test's import. Also commit the test's sqlx query cache (was never prepared with --tests, so SQLX_OFFLINE cargo test failed pre-existing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: invalidate producer-cache in asset dispatch tests (mirror deploy)

The tests seed asset rows directly and run no notify poller, so the per-workspace producer cache went stale across tests → 0 dispatched. Clear it at the seed point, as a deploy would via notify_event. All 8 asset_trigger_dispatch tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ba677ea142011462ad4dfe77e8375a6dd274cdef

This commit updates the EE repository reference after PR #619 was merged in windmill-ee-private.

Previous ee-repo-ref: 925c350cff55d3ea738d9e2e4098d9ce4bdda418

New ee-repo-ref: ba677ea142011462ad4dfe77e8375a6dd274cdef

Automated by sync-ee-ref workflow.

* test: disable producer cache in asset dispatch tests (isolated-DB safe)

The .remove(WS) approach still raced: #[sqlx::test] gives each test its own DB but they share one workspace id, so the WS-keyed process-global cache clobbered across DBs under concurrent threads. Add an ASSET_PRODUCER_CACHE_DISABLED test hook and set it in the tests so every dispatch reads its own DB. 8/8 pass at --test-threads=10. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: replace asset-cascade depth cap with cycle detection

The hardcoded MAX_CHAIN_DEPTH=5 truncated legitimate deep pipelines (silently — the check returned before event logging). Replace it with per-edge cycle detection: carry the producer lineage in trigger.chain and skip only a subscriber already in the chain, recording a visible cycle_detected dispatch_event. Acyclic pipelines of any depth now cascade fully; a high MAX_CHAIN_LEN backstop guards against runaway. Tests + UI label updated; 8/8 pass at --test-threads=10.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: update dispatch_event reason examples (depth_cap → cycle_detected)

Comment-only; the migration is idempotent and already in the potentially_stale self-heal list, so the checksum change re-applies cleanly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: park cascade retry (P1 dead-end) + clear stale script_triggers on rename

Two deploy-path fixes:
- Retry is parked: a retried subscriber is wrapped in a SingleStepFlow, whose run is a flow step and ineligible for asset dispatch, so it would silently dead-end the cascade (P1). Stop persisting retry to script_trigger and warn at deploy; TODO(pipeline-retry) to re-enable once dispatch handles flow-wrapped producers. (Dispatch plumbing kept + still tested via direct seeding.)
- Rename leaves stale script_trigger rows: clear was keyed on ns.path only, so old-path '// on' edges lingered and could trigger a script later recreated at that path. Also clear the old path on rename (assets already handled via the parent-hash clear).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Arnaud <31803803+Araden14@users.noreply.github.com>
Co-authored-by: Diego Imbert <diego@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Aldrin Jenson <aldrinjenson@gmail.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
2026-06-18 18:09:02 +02:00
GuilhemandClaude Opus 4.8 611c70acd2 feat(frontend): adapt AI-chat/sessions drafts to DB-backed model (#9601)
* feat(frontend): adapt AI-chat/sessions drafts to DB-backed model

PR #9351 dropped UserDraft's localStorage layer; the chat adapter's
synchronous save->read-back threw "Could not read written draft". The
adapter now treats the backend as source of truth (in-tab cell used
opportunistically for live-preview coherence) with conflict-on-save,
and read tools fall back to the backend. Collapses the six writeXDraft
functions onto one generic writeDraft + typed per-kind WriteSpec
constants. Terminology: "local draft" -> "draft" (drafts are server-side).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(frontend): autosave indicator + draft-only diff guard in session editors

Thread an explicit (workspace, path) autosave target to the cloud
AutosaveIndicator in the Script/Flow/RawApp session previews so it
watches the same key saves land on (it previously watched an empty path
and never animated). Disable the Diff button with a hint for draft-only
(no_deployed) items consistently across the three editors. Adjust the
script topbar compact breakpoint/layout so the cloud icon is part of the
bar, and stop splitpanes over-constraining session panes on reload.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): session draft diff viewer for schedule/resource/variable

Canonicalize both sides of the draft diff onto one field set and strip
runtime-only fields so rows aren't spuriously marked all-changed; mask
secret values. Map draft itemKinds to deploy-style kinds so the DiffRow
shows the correct icon/label.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): uniform diff-viewer row height regardless of summary

Diff-viewer leaf rows (WorkspaceItemRow) drew two lines when an item had a
summary and one line otherwise, giving unequal heights. Add an opt-in
`uniformHeight` prop that gives the text wrapper a shared min-height and
vertically centers the one-line case; enable it only from the diff viewer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(frontend): address review nits on the drafts diff/guard changes

- Reuse the exported TRIGGER_RUNTIME_IGNORE from utils_deployable instead of
  a verbatim copy, so the runtime-field ignore list has one source of truth.
- Drop the now-redundant `(savedApp as any)` cast in RawAppEditorHeader; the
  prop type already carries `no_deployed`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): add description parameter to the write_flow chat tool

write_flow had no way to set a flow's top-level description (the sibling
of summary in OpenFlow); patch_flow_json only edits the compact value, so
the field was unreachable from the AI chat. Thread an optional description
end-to-end: tool schema -> persisted draft -> read-back -> deploy body.
Structural patches (patch_flow_json/set_flow_module_code) pass no
description, so a previously-set description is preserved. Adds a
deployRequests regression test asserting a draft description reaches the
deploy body, overriding the deployed one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): round-trip top-level fields in session preview draft sync

The session preview's two-way draft sync dedups on a per-kind signature
and mirrors fields between the editor store and the shared UserDraft cell.
Both omitted fields the chat can set, so with the preview open a change to
only that field was swallowed (identical signature) and then clobbered by
the editor's outbound save:
- flow: the signature and applyDraftToStore ignored top-level `description`.
- script: the signature keyed on `content` alone, dropping `summary`/`language`.

Add the missing fields to flowDraftSig and the script codec signature, and
copy `description` in the flow codec's applyDraftToStore (mirroring `summary`).
Raw-app already stringifies the whole draft, so it was unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): deploy draft-only flow from the session preview

Deploying a draft-only flow (a draft with no deployed row) from the session
preview hit two gaps the full-page flow editor already handled:
- create vs update: newFlow keyed on `!savedFlow.val`, but a draft-only flow
  has a synthesized savedFlow (no_deployed=true), so deploy took updateFlow
  against the draft path and 404'd "Flow not found". Key it on no_deployed too.
- friendly name: a brand-new flow is stored under a `draft_<uuid>` path with
  its intended name in `draft_path`. Seed the builder's initialPath from
  `draft_path` (as the full-page editor does) so the Path widget and deploy
  use the friendly name instead of creating a flow named draft_<uuid>.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): deploy draft-only raw app from the session preview

Same create-vs-update bug as the flow session preview: newApp keyed on
`!savedRawApp.val`, but a draft-only app has a truthy synthesized savedApp
(getAppByPath with rawApp:true resolves to the draft kind instead of 404ing,
carrying no_deployed=true), so deploy took updateApp against a path with no
deployed row and 404'd "not found". Key newApp on no_deployed too so a
never-deployed app deploys via createApp. More reachable than the flow case:
it hit any never-deployed app, including chat-created ones at friendly paths.

Keying newApp on no_deployed also exposed that newEditedPath (the breadcrumb
path AND the createApp target) used newApp to mean "brand-new, generate a
random name". A draft-only app is newApp=true but already has a real path
(empty newPath at init, but appPath is set), so it showed and would deploy a
random `*_app` name. Prefer the real appPath before the random fallback, so
only a genuinely new app (appPath === '') still gets a generated suggestion;
the full-page editor is unaffected (it always sets newPath).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): don't re-save a draft after deploying from the session preview

Deploying from a session preview reloaded the editor (expected) but then
immediately POSTed a fresh draft. The full-page editor guards deploy with
discardDraftAfterDeploy (stopSync + arm-restart-on-first-interaction), but the
shared editor header skips that in a session pane (inSessionPane) and routes
post-deploy cleanup through sessionRuntime.syncPreviewWithDeployed, which did
discard + reload without the stopSync guard. UserDraft.discard keeps the cell
entry, so the reload's UserDraft.save fired the cell's reactive effect and
re-POSTed the just-deployed value as a draft.

Wrap the discard + reload in the same UserDraft.stopSync + armRestartOnFirst-
Interaction bracket. One place fixes all three kinds (script/flow/raw_app),
since they all funnel through syncPreviewWithDeployed; autosave resumes on the
next genuine edit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(frontend): address review findings on the session-preview drafts work

- Type `no_deployed` via the GetXByPathResponse/UserDraftOverlay types instead
  of `(result as any)`/`(saved as any)` casts at the sites this branch added
  (sessionRuntime, ScriptBuilder, FlowBuilder, + widened the script/flow
  builder prop types). Pre-existing trigger/variable/resource-editor casts
  left untouched.
- Drop a history-narrating comment parenthetical per the AGENTS.md comment
  policy (RawAppEditorView).
- Add a unit test covering persistGlobalDraft's conflict-on-save / override
  path (conflict-capable updateDraft mock; inert for existing tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): keep the friendly generated path for a brand-new raw app

The earlier draft-only newApp fix made newEditedPath prefer `appPath` before
the random suggestion, but a brand-new app is parked at the storage placeholder
`u/{user}/draft_{uuid}` (the /apps_raw/add redirect target), so it surfaced that
uuid instead of a friendly `<adjective>_app` suggestion. Reject a `draft_`
placeholder segment when choosing the path: a real named/draft-only path is
still kept, a placeholder falls through to the generated suggestion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): show the Diff-button tooltip when it's disabled

A disabled <button> fires no pointer events and browsers suppress its
native title, so the "deploy once to compare" explanation never showed on
hover for a draft-only item's Diff button. Wrap the button in a titled
element and set pointer-events-none on the button when disabled, so the
hover reaches the wrapper. Applied in ScriptBuilder, FlowBuilder, and
RawAppEditorHeader (covers both the full-page editors and the session preview).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): surface draft save failures and conflicts in the AI chat tools

Addresses Codex + Pi review findings on PR #9601 (three P1s, all in the
DB-backed draft adapter reporting success when the write didn't land):

- persistGlobalDraft reported {status:'saved'} even when UserDraftDbSyncer.save
  failed (it records network/5xx into a failure map instead of throwing). Check
  getState().state==='failed' after the save and return a new 'error' status;
  finishDraftWrite now emits success:false with a retry hint.
- saveGlobalAppDraft dropped the conflict/error status (returned only the item),
  so write_app_file/patch_app_file/write_app_runnable reported every stale or
  failed write as saved. It now returns the full DraftPersistResult, and the six
  app write tools route through a shared finishAppDraftWrite helper.
- fetchBackendDraftValue's catch{} swallowed non-404 errors (403/500/network),
  collapsing them to "no draft" so the write merged from the deployed item and
  lost in-progress draft edits. Narrow the catch to status===404; propagate the
  rest.

Adds unit coverage: save-failure -> 'error', non-404 read -> propagates,
raw-app stale write -> 'conflict'. 71/71 pass, check:fast + full build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): surface failed draft deletes + strip tool-only override from schedule drafts

Addresses Codex's second-round review on PR #9601:

- [P1] deleteGlobalDraft reported success even when the server delete failed or
  conflicted (UserDraftDbSyncer.save records failure state instead of throwing) —
  so discard_local_draft / deploy_workspace_item / delete_workspace_item / the
  /global_drafts delete could report a draft removed while the DB still had it.
  Check getState().state and getConflict() after the awaited null save and throw,
  mirroring the write-path guard.
- [P2] writeScheduleDraft persisted the tool-only `override` conflict flag into
  the schedule draft value (mergeDraftConfig cloned every arg field). Strip
  `override` in SCHEDULE_SPEC.buildDraft before merging.

Tests: failed server delete -> throws; schedule draft no longer contains
`override`. 73/73 pass, check:fast + full build clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): /global_drafts "Clear all" deletes persisted drafts, not just cells

Codex review nit (P2): the dev-only global-drafts inspector's "Clear all" called
clearGlobalDrafts(), which only iterates in-tab UserDraft cells — any persisted
backend draft row not currently mounted as a cell survived, so the list re-showed
it after refresh. Iterate the listed drafts and delete each via the backend-aware
deleteGlobalDraft() (continue past per-row failures), matching the per-row delete,
then clear local cells + ephemeral secrets.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-16 15:20:19 +02:00
9de38f9a09 feat(forks): handle triggers and schedules in wmill workspace merge (#9023)
* feat(forks): handle triggers and schedules in wmill workspace merge

Closes #9001. Brings CLI parity with the merge UI by routing trigger and
schedule diffs through the existing workspace_diff tally infrastructure
and lifting the deploy logic into the shared windmill-utils-internal
module.

- Backend: extend tally + compare to all 10 trigger kinds + schedule;
  new compare_two_trigger_or_schedule helper using to_jsonb minus runtime
  ignore set; CompareSummary gains schedules_changed/triggers_changed.
- Operational-state invariant: fork operations never flip target's
  mode/enabled. Triggers strip mode/enabled in both UI and CLI deploy
  payloads (preserved by is_mode_unspecified on backend). Schedules drop
  the setScheduleEnabled mirror entirely on merge — EditSchedule lacks
  enabled by design.
- Shared module: DeployKind extended with schedule + per-kind triggers;
  DeployProvider gains per-kind dispatch methods.
- Frontend: ~600 lines of client-side trigger-diff machinery deleted;
  rows flow through comparison.diffs like every other kind. Diff drawer
  returns full GET response stripped of runtime fields, matching backend
  semantics. Default selection excludes triggers/schedules (opt-in).
- CLI (merge.ts): per-kind provider, GCP-specific transforms (audience
  reset, base_endpoint with /api stripped to match frontend), summary
  table rows for Schedules/Triggers, default-deselect mirroring the UI.
- Bumps windmill-utils-internal to 1.5.0 (new exports for trigger
  per-kind dispatch); frontend depends on ^1.5.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(enterprise): clarify [ee] prefix applies whenever an EE companion PR exists

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 6ee680c25e3413d928fc22002be6deb118092668

This commit updates the EE repository reference after PR #557 was merged in windmill-ee-private.

Previous ee-repo-ref: ad35a056627656fd426fb19856ea945955d4727f

New ee-repo-ref: 6ee680c25e3413d928fc22002be6deb118092668

Automated by sync-ee-ref workflow.

* fix(forks): preserve target state on merge update, mirror source on create

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): strip server-managed trigger fields and honor --include with --skip-conflicts

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-05-07 13:29:10 +00:00
hugocasaandClaude Opus 4.7 da95588b25 fix(forks): strip mode/enabled from merge-UI deploy payload (#9008)
* fix(forks): strip mode/enabled from merge-UI deploy payload

The CompareWorkspaces merge UI deploys a trigger by fetching the full
GET response from the source workspace and spreading it into an
updateXTrigger call on the target. That spread includes `mode` (and the
legacy `enabled`), so a fork→parent (or parent→fork) deploy would
overwrite the target's enabled/disabled state — silently disabling a
parent's trigger when its config is merged from a freshly-cloned fork
(clones are forced `mode='disabled'`). Affects azure/email/gcp/http
whose `update_trigger` SQL writes the `mode` column.

Strip `mode`/`enabled` in `getTriggersDeployData` so the backend's
existing `is_mode_unspecified()` preservation in `update_trigger` keeps
the target row's `mode` untouched. The same preservation already
protects the YAML/CLI round-trip (where the tarball export strips these
fields); this extends the same guarantee to the merge-UI path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(forks): wire Azure and Email triggers through merge UI deploy

CompareWorkspaces.svelte already lists Azure and Email triggers in its
diff (`triggerServices`), but `getTriggersDeployData` and `existsTrigger`
were missing the corresponding branches. Deploying either kind from the
merge UI threw "Unexpected trigger kind". Add the branches with the
same `stripOperationalState` pattern as the rest, and extend the
`triggersKind` whitelist in `checkItemExists`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 15:12:43 +02:00
efb4a27d51 fix: replace email with permissioned_as for triggers/schedules (#8439)
* refactor: replace email with permissioned_as for triggers/schedules

Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.

- Migration: add permissioned_as to all 9 trigger tables + schedule,
  drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
  places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
  in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix sqlx/build

* update ee ref

* refactor: simplify resolve_edited_by to always use authed username

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix compile + migration

* update ref

* test: add trigger trait method tests for permissioned_as queries

Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update sqlx

* fix: use permissioned_as directly for schedules and fix audit RLS for groups

- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
  to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build

* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule

- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
  explaining it carries emails for flows/scripts and permissioned_as for
  triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
  getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
  customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
  EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
  just backwards compat, only permissioned_as matters)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve email column when permissioned_as is preserved on schedule edit

Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update deploy UI labels from "edited by" to "run as" for triggers

Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use u/username format for custom trigger/schedule deploy selection

When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show u/username format for "me" option in trigger deploy selector

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify OnBehalfOfSelector to return the right format per kind

OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username

The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always show u/username format in OnBehalfOfSelector for all kinds

Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace email with permissioned_as in http_trigger test insert

The email column was dropped from trigger tables in the migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review fixes — migration, app policy, capture cleanup, naming

- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to all test schedule INSERTs

Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: strip permissioned_as from exports/sync, fix OpenAPI required field

- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove accidentally committed generated files

These directories are gitignored and should not be tracked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as schema changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove permissioned_as from CLI TriggerFile Omit list

Already stripped in workspace export, no need to also omit from the type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: optimize email cache key and revert TriggerFile Omit change

- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: zero-allocation email cache lookups using Equivalent trait

Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to Schedule required fields in OpenAPI spec

The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle group- prefix in migration UPDATE statements

edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "fix: handle group- prefix in migration UPDATE statements"

This reverts commit 0971392b38.

* fix: use superadmin email to resolve permissioned_as in schedule migration

For users upgrading from older versions where edited_by may not reflect
the actual schedule owner, check if the email belongs to a superadmin
and look up their username. Otherwise fall back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fall back to superadmin email when not in workspace usr table

If the superadmin isn't a member of the workspace, use their email
as raw permissioned_as instead of falling back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always update permissioned_as and email on schedule edit

Consistent with pre-refactor behavior where email and edited_by
were always updated on every edit. permissioned_as is now always
set (to editing user or preserved value), removing the COALESCE
that previously preserved it when not provided.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add schedule permission tests and centralize group prefix constants

Tests: schedule create/update for normal user, workspace admin, and
superadmin not in workspace. Verifies schedule fields (email,
permissioned_as, edited_by) and pushed job fields (permissioned_as,
permissioned_as_email).

Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX,
PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use @unknown.windmill.dev for synthetic email fallback

Prevents privilege escalation: a user with username like
'superadmin_secret' would get superadmin via the synthetic
email matching SUPERADMIN_SECRET_EMAIL. Using a different
subdomain avoids any collision with hardcoded @windmill.dev emails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update ee ref

* sqlx

* chore: regenerate system prompts after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148

This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private.

Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843

New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-20 16:28:38 +00:00
wendrul 7ac93f6ee3 feat: option to preserve on_behalf_of and edited_by for admins and users in the new wm_deployers group (#8079) 2026-02-25 12:05:22 +00:00
centdixandClaude Opus 4.6 37d1277b91 feat(cli): add lint command (#7917)
* feat(yaml-validator)!: unify flow, schedule, and trigger validation

- replace FlowValidator with WindmillYamlValidator.validate(doc, target)

- generate schedule/trigger schemas from backend OpenAPI and OpenFlow refs

- add schedule/trigger/filename-target tests and update AI agent fixtures

- bump windmill-yaml-validator to 2.0.0

BREAKING CHANGE: FlowValidator and validateFlow() are replaced by WindmillYamlValidator.validate(doc, target).

* add lint command

* add deno-compat script and docs for local yaml-validator testing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: make nullable fields pass yaml validation

Add nullable: true to static_asset_config and authentication_resource_path
in HttpTrigger schema. Post-process generated JSON schemas to add null to
enums with nullable: true (AJV doesn't handle OpenAPI 3.0 nullable + enum).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add nullable to all Option<T> fields in trigger and schedule OpenAPI schemas

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): handle nullable fields from updated OpenAPI types

Add ?? undefined coalescing at assignment sites where generated types
now include | null from the OpenAPI nullable additions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(lint): show allowed values in enum validation errors

Instead of "must be equal to one of the allowed values", now shows
"must be one of: 'r', 'w', 'rw'" for enum validation failures.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add nullable to Edit/New trigger and schedule OpenAPI schemas

Ensures create/update request body types accept null for the same
fields that GET response types return as nullable, enabling clean
round-tripping without type mismatches.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* use published package

* publish

* refactor(lint): remove unused --includes/--excludes/--extra-includes CLI options

These options were defined but never wired to the file filtering logic.
The lint command still respects includes/excludes from wmill.yaml via
mergeConfigWithConfigFile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(lint): handle additionalProperties errors and expand test coverage

Add formatting for AJV additionalProperties keyword to show the unknown
property name. Add unit tests for all formatValidationError branches and
integration tests for --json report shape, --fail-on-warn with mixed
files, non-existent directory, and enum error output.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add realistic validator tests for schedules, triggers, and edge cases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add email trigger validation support

Add email trigger schema generation, validation, and linting. Email
triggers are no longer skipped with a warning — they are validated
like all other trigger types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(cli): bump windmill-yaml-validator to 1.1.1 (email trigger support)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* publish

* rm

* fix: address PR review feedback for lint command

- Add email to trigger kinds test loop instead of separate test
- Add email to ValidationTarget docs in README
- Type formatYamlDiagnostics param directly instead of unsafe cast
- Destructure json option before mergeConfigWithConfigFile for clarity

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(cli): add --lint option to sync push command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:41:04 +00:00
hugocasaanddieriba eb284dfabd feat: triggers suspended mode (#7297)
* first commit

* base

* ok

* remove print

* rm packe json

* fix discard

* fix

* update .sqlx

* update

* update ref

* jobs update

* nits

* update ref

* big

* update sqlx

* fix

* update

* nits and fix

* nits

* ok

* sqlx + fix import

* better

* nit

* suspended mode draft

* UI and http triggers

* nits

* done for http triggers

* feat: better retry/error handler check

* generalize to all triggers

* fix merge conflicts

* fix listener

* nits

* nit

* fix merge conflict

* update ee ref

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>
2025-12-08 16:20:49 +00:00
hugocasaandRuben Fiszel 8cd0006498 feat: http routes streaming (#6834)
* feat: http routes streaming

* improve webhook config section for SSE

* feat: add request type config to http routes

* fix check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-17 14:05:17 +00:00
494d662b2d fix cli gcp types (#5644)
* update types

* fix types cli

* fix ci

* fix gcp types and fix used triggers

* update repo ref revert .env and nits

* fix build

* Update frontend/src/lib/utils.ts

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* fix type and deploy to

* update ee-repo-ref

* update .sqlx

* add missing fields

* fix: add sqlx derive macro

* hanfle missing field, fix deploy to other workspace and update ee-repo-ref

* update repo ref

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-04-23 12:18:36 +02:00
6339775404 feat: add gcp trigger (#5501)
* update: add migration for gcp_trigger table, add cli method gcp and add gcp_trigger args for github ci

* update: add gcp module and routing it in lib.rs

* update: added gcp type to TRIGGER KIND type in db, add new feature condition to try_get_fn and added gcp_trigger feature in cargo file

* update: add ui for gcp trigger, added missing triggers type for job kin, add gcp trigger in deploy to functionallity

* adding google cloud crate

* update: handle pull and push delivery

* update: handle pull and push delivery

* update: handle push event, changed front ui

* update: ui

* fix: capture for gcp

* update: automatically manage pub sub subscription and refactoring

* capture done

* update cli types

* fix: wrong func argument and type openapi

* fix: missing import

* update .sqlx

* update: svg color and remove pulse button for capture push gcp

* update: handle deployto

* update script helper and link to hub gcp script/flow

* update gcp representation

* update: add confirmation modal

* add unique index to mitigate same subscriber id, fix `zombie worker`, update test connection logic

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update cli/gen/services.gen.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* update repo ref

* fix feature function

* update dependencies

* update .sqlx

* fix missing import

* update: handle existing subscription and creating new one or update it

* fix ee build

* handle existing config properly

* update .sqlx

* update .sqlx

* remove unused

* update documentation link fix ci

* use on instance of empty_string_as_none fn

* update refo ref

* updat script_helper

* update ui

* update migration

* add missing arguments

* fix and nits

* update repo ref

* remove unused

* nits

* add doc links

* nits

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerPanel.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* use trigger path as default route path

* update .sqlx

* update repo ref

* update ref

* update route path

* update types

* fix deploy to

* nits

* update repo ref

* update .sqlx

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorConfigSection.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* nits

* nits

* remove dupliacte fn and update repo ref

* remove unused import

* fix missing import

* fix: wong name var

* update script helper and template script

* Update frontend/src/lib/script_helpers.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update ee-repo-ref.txt

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-04-10 20:39:01 +00:00
e920101107 feat: api key/basic/hmac auth for http triggers (#5476)
* feat: add webhook configuration in front and start handling logic in back

* feat: add generic implementation

* update: add stripe provider working on auth

* merge main

* handling new webhook, add generic implem

* feat: add auth method and api key and handling custom authentication

* nits

* added generic implem for webhook provider

* feat: added resource webhook_auth to handle http route auth, refactored backend code

* update sqlx

* fix: zoom and twitch challenge request event, call verify function

* removing webhook name to make it more general, properly added custom auth option alongside windmill_auth option

* update .sqlx

* update deply to for http routes

* update .sqlx

* add different auth method for http routes

* add custom script method and custom signature fixed bug with webhook auth

* update auth options

* update .sqlx

* add new auth method

* update .sqlx

* refacto function and fix frontend bug

* fix typo, update .sqlx and handle pototential error

* nits

* nits

* add script template flow

* Update frontend/src/lib/components/triggers/http/utils.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* update: remove wrong trigger type and use right icon for sqs trigger

* nits

* update: .sqlx

* update: reunite flow and script logic for script placeholder value replacement

* Update backend/windmill-api/src/http_trigger_auth.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* delete vite config

* remove darks mode observer

* update: add new option to capture panel for http route

* nits

* revert

* nits

* typo

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-03-27 08:43:42 +00:00
dieribaandellipsis-dev[bot] b210ae36f7 feat: deploy triggers to prod/staging workspace (#5429)
* add deploy to staging prod option to trigger

* remove unused import

* handle schedule trigger, fix update fn

* Update frontend/src/lib/utils_deployable.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/workspaces.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/DeployWorkspace.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* update kind name and handle script kind flow for websocket

* nits: english wording

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-06 17:02:11 +01:00
Ruben Fiszel f300f4cf2e nits whitelabel 2024-08-14 02:09:22 +02:00