Adding a small helper for the upcoming internal admin tooling so we
can iterate on the lookup flow.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [ee] fix(autoscaling): consider dedicated workers in scale decisions
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Update ee-repo-ref.txt
* [ee] fix(autoscaling): mirror worker tag precedence (worker_tags wins)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 862d487032efe30d1e4a3cd0a1ed7169500c4cd9
This commit updates the EE repository reference after PR #556 was merged in windmill-ee-private.
Previous ee-repo-ref: cf87e9dcef2e95b1834b3f5c154209defc5a9ca2
New ee-repo-ref: 862d487032efe30d1e4a3cd0a1ed7169500c4cd9
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix: bind MySQL table listing to configured database name
* refactor: drop DATABASE() sentinel and cover single-table fallback
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: unify MySQL schema-resolution branches via explicit_db binding
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: replace SELECT * with explicit column lists
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore: update sqlx offline query cache
* chore: update sqlx offline query cache
* chore: update sqlx offline query cache with EE support
* chore: update sqlx offline query cache, no deletions
* chore: update sqlx offline query cache after rebase
* fix: correct column names in explicit script query lists
- concurrency_limit → concurrent_limit (matches DB column name)
- runnable_settings → runnable_settings_handle (matches DB column name)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: add missing delete_after_secs column to script queries
Also add integration test covering all explicit-column export queries.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: add workspace export integration test covering all explicit-column queries
Covers tarball_workspace (folder, script, resource, resource_type, variable,
schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test: add tarball export integration test covering all explicit-column queries
Single test creates one of each entity type and exercises every runtime-checked
explicit-column query in tarball_workspace. Uses archive_type=tar to avoid
zip feature-gate in CI.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: fix stale verification step and CI contradiction in update-sqlx skill
- Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff
- Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: remove Co-Authored-By from commit skill template
* refactor: extract SCRIPT_COLUMNS const to single source of truth
Replaces 5 duplicated 44-column lists with a shared const in windmill-types.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* docs: add SAFETY comments to all dynamic SQL call sites
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: address review feedback on SAFETY comments
- Fix missed comment for obo_triggers loop in offboarding.rs
- Fix variable name in comment (table -> table_name) in offboarding.rs
- Fix api-settings comment to reference inline VALID_NAME regex, not validate_dbname()
- Add SAFETY comments to batch_execute calls in api-settings
- Fix db.rs comment: PG_SCHEMA is env var, not compile-time constant
- Add doc comments on RunnableSettingsTraitInternal constants
* docs: remove misleading SAFETY comment on static SQL
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: surface scope errors as 403 and show real message in CLI
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: address review feedback on scope error PR
- Backend: also patch handler-level check_scopes (lib.rs:223) — without
this, endpoints using check_scopes (scripts, flows, jobs, …) still
returned 401 for scope failures, which the CLI would render as the
misleading auth message.
- CLI: strip backend file refs and the duplicated "Permission denied:" /
"Not authorized:" prefix from the surfaced error body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat(forks): strip operational state from triggers/schedules on git-sync export
When the source workspace is a fork (`wm-fork-*`), the tarball export now
omits `mode` from triggers and `enabled` from schedules. The trigger update
handler also preserves the existing DB `mode` when both fields are absent
from the request, instead of falling back to the BaseTriggerData default.
This prevents a fork's git-sync round-trip from flipping the parent
workspace's enabled/disabled state when a merge applies the fork's YAML
back to main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(forks): opt-in fork_triggers flag clones triggers/schedules disabled
Adds `workspace.fork_triggers` (default false) and a matching field on
CreateWorkspaceFork. When the user opts in, fork creation also runs
clone_triggers_and_schedules: every row in schedule and the ten
*_trigger tables is copied to the fork with mode='disabled' /
enabled=false. Listener identifiers (group_id, replication_slot_name,
subscription_name, …) are copied verbatim — the runtime suffix that
prevents the fork from competing with the parent ships in a follow-up
PR.
native_trigger is intentionally skipped: those triggers manage external
webhook state we don't want duplicated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(forks): warn before enabling triggers/schedules that conflict with parent
set_trigger_mode and schedule's set_enabled now check whether the parent
workspace has the same path actively enabled. If so, the call is rejected
with a `fork-conflict:<kind>:<parent_id>` error unless the request includes
`force=true`. The frontend interprets the prefix to surface a confirm-to-
proceed dialog.
This is the placeholder safety net until the Phase 3 listener-suffix work
removes the conflict for the namespaceable kinds (Kafka/MQTT/NATS/Postgres/
Azure/GCP-CreateNew). For SQS, GCP-Existing, and schedules — where there's
no namespacing fix — the warning is the durable solution.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(forks): UI: opt-in clone-triggers checkbox + confirm-on-fork-conflict
Adds the user-facing surface for the fork-trigger work:
- CreateWorkspaceInner: new "Clone triggers and schedules" toggle in the
fork-creation dialog (default off). Sends fork_triggers in the request.
- forkConflict utility: detects the `fork-conflict:<kind>:<parent_id>`
error string from the backend, shows a confirm() dialog explaining
why the action is blocked, retries with `force: true` if accepted.
- Wires withForkConflictRetry into every trigger setMode and the
schedule setEnabled call, both in the per-kind editor components and
the +page.svelte list views (HTTP, websocket, kafka, NATS, SQS, MQTT,
GCP, Azure, Postgres, email, schedule).
OpenAPI spec gains the `force` field on each setmode/setenabled body.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(forks): CLI --fork-triggers flag, fork-trigger docs, skill update
- Adds --fork-triggers boolean to wmill workspace fork; passes
fork_triggers through to the create_fork API call.
- New docs/fork-triggers.md describing the model end-to-end (default,
opt-in clone, merge-direction filter, conflict warning, future
runtime-suffix work).
- Updates the adding-a-trigger SKILL.md to mention the fork-export
ignore-keys participation and the clone_triggers_and_schedules
block that new trigger kinds must extend.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate sqlx offline query cache for fork-trigger SQL
* fix(forks): replace browser confirm() with ConfirmationModal for fork conflict
The fork-conflict warning previously used the browser's native confirm()
which doesn't match Windmill's design system. Switches to a singleton
ConfirmationModal mounted at the (logged) layout root, driven by a new
forkConflictModal store. The withForkConflictRetry helper now sets the
store and awaits the user's choice via a Promise, instead of blocking
on window.confirm.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(forks): filter unchanged triggers in merge UI, add diff view, surface parent-only ones
The fork merge UI listed every trigger from the fork as a deployable item
regardless of whether it differed from the parent — so a fork created with
fork_triggers=true (which clones triggers in disabled state, otherwise
identical) showed every trigger as a "Fork-only" change. The 'Update
current' tab also missed triggers newly created in the parent that the
fork hadn't pulled yet.
This refactor:
- fetchAllTriggers now lists both fork and parent in parallel for each
trigger kind, then merges by path.
- Computes a per-trigger `changeKind` (new / modified / deleted-in-source)
using a JSON comparison that strips runtime + fork-local fields
(mode/enabled/server_id/last_server_ping/edited_at/edited_by/etc.) so
the disabled-on-clone difference doesn't show up as a change.
- Filters the trigger items in deployableItems by the current direction:
Deploy mode shows fork-side new/modified, Update mode shows parent-side
new/modified.
- Replaces the always-on "Fork-only" badge with proper New/Modified
badges and surfaces a Diff button (modal Drawer + Monaco DiffEditor)
for modified triggers — the diff strips the same ignored fields so
users see only the meaningful config differences.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(forks): always clone triggers/schedules disabled, drop opt-in flag
Disabled triggers and schedules are inert — no listener attaches, no cron
fires — so cloning them by default is safe by construction. Drops the
fork_triggers opt-in flag introduced earlier in this PR:
- Drops workspace.fork_triggers column (migration removed)
- Removes fork_triggers from CreateWorkspaceFork (API + OpenAPI)
- Removes the conditional in create_workspace_fork — clone always runs
- Removes the toggle from the fork-creation dialog
- Removes --fork-triggers from `wmill workspace fork`
- Updates docs/fork-triggers.md and adding-a-trigger SKILL.md
The merge UI continues to exclude triggers from the deploy/update default
selection, so a routine merge from a fork doesn't accidentally push
trigger config the user hasn't intentionally changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(http-triggers): scope route exists check by workspace, skip non-workspaced clones in forks
The non-CLOUD branch of `route_path_key_exists` self-excluded by trigger
path alone, which silently masked cross-workspace collisions once forks
started cloning trigger rows verbatim. Tighten it to exclude only the
exact `(workspace_id, path)` row.
Fork creation also now skips non-workspaced HTTP triggers — their URL
has no workspace prefix, so a clone collides with the parent at the
matchit router (which silently drops one of two duplicates) and there is
no namespacing escape hatch. The clone copies all rows when CLOUD_HOSTED
or HTTP_ROUTE_WORKSPACED_ROUTE forces every route workspaced regardless.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(forks-ui): silent cancel on enable conflict, clean up trigger rows in compare view
forkConflict helper now returns undefined when the user dismisses the
modal instead of throwing, so the redundant 'Cannot enable: undefined'
toast no longer appears.
CompareWorkspaces trigger rows now mirror the script row layout: drop
the redundant Disabled badge and the Trash/Details buttons (both belong
on the dedicated trigger pages, not in the deploy/compare view); pass
triggerKind through so RowIcon picks the right kind-specific icon; move
extraLabel into the summary line; replace the yellow Modified badge
with the same green ↗ ahead / blue ↘ behind treatment scripts use.
Trigger diff drawer: switch JSON → YAML for parity with DiffDrawer, fix
zero-height monaco render with className=!h-full, drop the redundant
Original/Modified label banner above the diff.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(email-trigger): scope local_part exists check, skip non-workspaced clones in forks
Mirrors the HTTP route fix for the email-trigger non-CLOUD `email_exists`
check (in EE) which had the same path-only self-exclusion bug, and the
fork clone of `email_trigger` rows which copied non-workspaced
`local_part` verbatim. Skip non-workspaced rows in the clone unless the
instance is CLOUD_HOSTED (where lookup is workspace-scoped natively).
EE companion change in windmill-trigger-email/src/handler_ee.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 78512dd73b4a1c9f70574cff863374179e3a621b
This commit updates the EE repository reference after PR #554 was merged in windmill-ee-private.
Previous ee-repo-ref: 1ac77f50747b58e720a11162dfd309bc252a24ab
New ee-repo-ref: 78512dd73b4a1c9f70574cff863374179e3a621b
Automated by sync-ee-ref workflow.
* fix(forks): always-warn on parent row, kind-specific modal copy, cancel-aware toggles
- Conflict check now fires whenever the parent has the path (regardless of
parent's mode), since the cloned upstream identifier is shared by
construction; closes the Postgres slot-takeover gap when the parent is
disabled. Schedule's set_schedule_enabled gets the same treatment.
- Skip the warning entirely for HTTP and Email via a new
TriggerCrud::FORK_CONFLICT_ON_ENABLE const — both kinds are workspace-
scoped at runtime so cloned rows can't collide with the parent.
- Modal copy branches by failure family: split-events (Kafka/NATS/MQTT/SQS/
GCP/Azure), duplicate-firing (Websocket/Schedule), slot-takeover
(Postgres). Generic fallback for unknown kinds.
- withForkConflictRetry now returns boolean (true=committed, false=
cancelled). TriggerModeToggle reuses its existing innerTriggerMode local
state via a function binding for the regular Toggle, snapping back to
the prop when onToggleMode signals a cancel — needed because the native
bind:checked diverges from the parent's prop after a click and Svelte's
reactivity won't re-push a same-valued prop down. Schedule list page
uses {#key} on a reset version since it renders Toggle directly.
- Editor inners revert mode = previousMode on cancel; list pages skip the
re-fetch (loadTriggers/loadSchedules) on cancel to avoid pointless
network traffic and the schedule "Job stats loading..." flash.
- Drop withForkConflictRetry from HTTP and Email editors + list pages
since the backend never emits the conflict for those kinds.
* fix(forks-ui): widen onToggleMode types, scope schedule toggle reset by path
- TriggerEditorToolbar and TriggerSuspendedJobsModal forwarded
onToggleMode as `(mode) => void`, dropping the new boolean return so
any caller wired through them would silently no-op the cancel-revert.
Match the wider TriggerModeToggle signature.
- Schedule list page used a single resetVersion counter for every row's
{#key}, so cancelling on any one schedule remounted every <Toggle> on
the page. Switch to a per-path Record<string, number> bumped only for
the affected row.
* chore: bump ee-repo-ref to c3a4553 (email FORK_CONFLICT_ON_ENABLE override)
* fix(forks): include Suspended in conflict gate, use parent_workspace_id for fork detection
Three fixes from the Claude review on PR #8976:
- Suspended mode still attaches the listener (it just pauses auto-run of
queued jobs); two suspended fork+parent listeners would still split
Kafka events / share a PG slot. Gate set_trigger_mode on
`mode != Disabled` instead of `mode == Enabled` so Suspended also
surfaces the warning.
- workspaces_export.rs::fork_*_ignore_keys keyed off the wm-fork-* prefix
while set_trigger_mode and set_schedule_enabled key off
parent_workspace_id. Switch the export filter to query
parent_workspace_id once at the top of tarball_workspace and pass
is_fork through. The column is the contract; the prefix is a
creation-time naming convention that could in principle drift.
- TriggerModeToggle's suspend-dropdown action reassigned the non-bindable
`triggerMode` prop instead of the local `innerTriggerMode` mirror,
leaking inconsistent state if the dispatch was cancelled. Now writes
to innerTriggerMode like the Toggle's on:change handler does.
* fix(cli): skip setScheduleEnabled when local YAML lacks `enabled`
Tarball export from a fork strips `enabled` from schedules so the
fork→parent git-sync round-trip can't flip the parent's operational
state. The CLI's pushSchedule called setScheduleEnabled whenever
`localSchedule.enabled != schedule.enabled`, which evaluates truthy
when local is undefined (fork-pulled YAML) and remote is true/false —
sending `{ enabled: undefined }` that serializes to `{}` and gets
rejected by the backend (`SetEnabled.enabled` is required).
Skip the call when `localSchedule.enabled === undefined` so a sync push
of fork-pulled YAMLs preserves the target's existing enabled state
instead of erroring out. Trigger updates were already safe — the
backend's update_trigger preserves `mode` when the request omits it.
* Revert "fix(cli): skip setScheduleEnabled when local YAML lacks `enabled`"
This reverts commit 23ba7e72fc.
* feat(cli): --force flag and friendlier error on fork-conflict for schedule enable
`wmill schedule enable foo/bar` against a fork whose parent has the same
path used to surface the raw `fork-conflict:schedule:<parent>` error
body. The CLI now:
- accepts `--force` to bypass the warning (mirrors the API field and the
UI's "Enable anyway" confirmation),
- detects the `fork-conflict:` prefix on errors and prints a one-screen
explanation pointing at --force instead of the raw body.
Disable doesn't trigger the warning (the gate fires only on transitions
to listener-attaching modes), so no flag there. Trigger enable/disable
isn't exposed as a standalone CLI command — sync push goes through
updateTrigger which has its own backend mode-preservation, so no
fork-conflict surfaces from the CLI for those.
* chore: regenerate cli-commands docs after adding --force to schedule enable
* chore: update ee-repo-ref to 967f961f0a88b027d894aebd03977181129477a8
This commit updates the EE repository reference after PR #555 was merged in windmill-ee-private.
Previous ee-repo-ref: c3a4553296473932e15392a06415dd7fb9aa6591
New ee-repo-ref: 967f961f0a88b027d894aebd03977181129477a8
Automated by sync-ee-ref workflow.
* fix(forks): address CI dead-code, claude/cubic review feedback
- backend: cfg-gate `fork_trigger_ignore_keys` to match its already-gated
callsite. CI compiles with `-D warnings`, so the unused-fn under feature
combos that disable all trigger crates was breaking check_oss/check_ee/
cargo_test/test-linux/test-windows.
- cli: re-apply the `pushSchedule` undefined-skip (originally 23ba7e7,
reverted in 4d172a1). Tarball export from forks strips `enabled`, so
fork-pulled YAMLs that get sync-pushed back via `wmill schedule push`
would otherwise serialize `{ enabled: undefined }` → `{}` and the
backend's required `SetEnabled.enabled` rejects the body. Skipping
preserves the target's existing flag, which is the round-trip-safe
behavior. (`wmill workspace merge` extension to triggers/schedules is
tracked in #9001 — until then sync push is the only CLI path.)
- TriggerModeToggle suspend-dropdown action awaits onToggleMode and
resets `innerTriggerMode = triggerMode` on cancel, matching the Toggle
on:change handler. Without this, dismissing the fork-conflict modal on
a Suspend transition leaves the toggle stuck in 'suspended'.
- forkConflict: when a new modal opens with a previous resolver still
pending, resolve the older promise to false. Avoids a dangling promise
if the user clicks toggles on two rows in quick succession.
- schedules list: bump `toggleResetVersions[path]` on the
permission-denied branch so the Toggle re-mounts back to the prop's
`enabled` value. Without this, a user without write permission could
click the toggle and have it stick visually flipped.
- docs/fork-triggers.md: switch the merge-direction filter description
from `wm-fork-*` prefix to `parent_workspace_id IS NOT NULL` (matches
the code after 4dd38fe). Drop the misleading "merge-direction filter
strips identifier columns too" line in Future Work — the runtime
suffix is applied at listener attach, the stored column never carries
it, so no export filtering is needed there.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix: distinguish job-already-completed from execution failure on OTLP span
handle_queued_job's bool return type conflated two distinct Ok(false)
cases: a real race with another worker (Error::AlreadyCompleted) and any
job execution that returned an error via process_result. The outer "job"
span recorded "job already completed by another worker" for both, so
every failed Python/bun/etc. script ended up with that misleading
otel.status_message even though the real error was correctly recorded
on the inner job_postprocessing span.
Replace the bool with a JobOutcome enum (Completed / Failed { description }
/ AlreadyCompleted). Failed carries the truncated error string, so the
outer span's Status.message now reflects the actual cause.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: extract user-facing error from error_value for span description
Cubic flagged that capturing e.to_string() before the match meant
ExitStatus failures (the most common failure mode for script jobs)
ended up with the generic "exit status: …" string rather than the
script error extracted from job logs by extract_error_value.
Move the description capture to after error_value is built and
deserialise it as ErrorMessage to pull out the structured message.
Falls back to "Job failed" if the value isn't shaped that way.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: use permissive description extraction for OTLP status message
Cubic flagged that strict ErrorMessage parsing downgraded real failures
to a generic "Job failed" whenever the result wasn't shaped as
{message, name} — e.g. agent-worker's "See logs for more details" raw
string, or runtime-written result.json files with a different shape.
Switch to parsing as serde_json::Value and pulling .message out if it's
a string, falling back to the whole value if it's a bare string. Only
fall back to "Job failed" when neither is available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* fix: read inline-script tag from app policy in run mode
Previously the worker tag for app inline scripts was taken from the
client-supplied raw_code on every execute. End users running a deployed
app could intercept the request and submit any tag, redirecting the job
to an arbitrary worker group.
Persist the tag on PolicyTriggerableInputs at deploy time, and in run
mode read it from the policy instead of the request body. Preview mode
(editor-only) still honors the client tag, since the editing user is
already trusted by the policy check.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: also reject client tag in legacy run-mode (no app_script id)
The previous commit only enforced policy-tag in the id-bearing arm.
Apps deployed before the lockfile/app_script entry exists hit the
\`(None, Some(raw_code), None)\` arm in run mode (triggerable keyed by
\`rawscript/<sha>\`), where client tag was still trusted.
Hoist an \`is_preview\` flag from the outer match and route both inline
arms through it: client tag is honored only in preview mode.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: support assigning a worker tag to app/raw-app inline scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: omit empty tag field from inline script raw_code payload
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* style: shrink tag popover width
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: pair PG arg type with actual Rust binding to keep query_typed_raw safe
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(pg): wrap encoder errors with arg context, add fallback test
Followups on #8999 review:
- Wrap rust-postgres "error serializing parameter N" failures with the arg
name, JSON value kind, and asserted Postgres type plus a hint about an
explicit cast — so users see actionable context instead of an opaque
WrongType.
- Drift-prevention meta-test: assert otyp_to_pg_type and convert_val agree
on the Type for every recognised arg_t when the JSON value matches its
natural Rust kind. Catches future drift if either side changes.
- Integration test for the prepare + query_raw fallback path: confirms
unrecognised arg_t (custom enum) is routed through prepare and the
server-resolved type appears in the failure surface — flips into a
test failure if a regression accidentally routes unrecognised types
through query_typed_raw.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): add otyp_inferred flag + regex-based placeholder renumbering
Two follow-ups from the review of #8999:
1. **Issue #1 (Number/Bool + explicit text decl in WHERE)**
Add `Arg::otyp_inferred: bool` to the parser. The PG SQL parser sets
it `true` only at the "no info → fall back to text" site (bare `$N`,
no inline cast, no `-- $N (TYPE)` decl). All other arg sources keep
it `false`.
In `convert_val` this flag distinguishes:
- explicit text-like target (`-- $1 (text)` or `$1::text`) — coerce
`Bool`/`Number` → `Box<String>` so `WHERE text_col = $1` works
(`text = text` operator). Pre-#8988 behaviour, restored.
- parser-default text (bare `$N`) — bind the value's natural Rust
type so the regression case (`Value::Bool` against a real `bool`
column via `CAST AS bool`) keeps working.
`Arg` is in `windmill-parser`; the new field has `#[serde(default)]`
so persisted signatures stay backward-compatible.
2. **Issue #4 ($5/$50 substring rewrite collision)**
Replace the per-index `String::replace` chain (which turned `$50`
into `$10` when oidx=5 was processed first) with a single regex
pass. `\d+` is greedy, so `$5` and `$50` match as distinct units;
indices outside the mapping are left intact.
3. Tests:
- parser: `test_parse_pgsql_otyp_inferred_flag` covers bare/inline-
cast/decl/mixed shapes.
- executor unit: `convert_val_bool_against_every_arg_t` and
`convert_val_*_number_*` split each text-like target into explicit
vs inferred expectations.
- executor unit: `renumber_sparse_placeholders_no_collision`.
- integration: `test_postgresql_arg_type_combinations` adds 4 cases
covering decl(text)+Number/Bool in WHERE, bare $1+Bool, and
sparse positional args ($5/$50).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg+sdk): enum support, extended String arms, position-aware $N rewrite, SDK quality
Backend:
1. **`AnyTextValue` ToSql/FromSql wrapper**: vanilla `tokio_postgres`'s
`ToSql for String` / `FromSql for String` reject `Kind::Enum` and
`Kind::Domain` even though the wire format is plain UTF-8. The wrapper
accepts those kinds in both directions. End result: explicit
`$1::my_enum` / `CAST($1 AS my_enum)` casts now round-trip without the
ugly `CAST($1::text AS my_enum)` workaround, AND `SELECT enum_col`
results come back as JSON strings instead of erroring at the FromSql
layer.
2. **#10 — Value::String → numeric/real/double/oid/bool**. Without these
arms, a string-encoded value (`"3.14"`, `"true"`) for a non-text /
non-temporal arg_t fell through to `Box<String> + TEXT`, which then
failed at the server (no implicit cast text→numeric in expression
context). Now strings are parsed into the matching native type with
clear error messages on parse failure.
3. **Position-aware `$N` rewrite**: replaces the regex-based renumbering
(which fixed the `$5/$50` substring collision but still walked through
string literals and comments, mangling `'price: $5'` etc.) with a
walk over `parse_pg_statement_arg_positions` — the same
string/comment/dollar-quote-aware tokenizer used for index discovery.
Adds `parse_pg_statement_arg_positions` to the parser's public API.
SDK:
4. **BigInt support**: `JSON.stringify(BigInt)` throws. The SDK now
stringifies bigints before serialisation; the executor accepts
numeric strings into BIGINT arg slots via the existing
`Value::String → INT8` parsing arm. SDK-side `inferSqlType` is split
so `BigInt` always resolves to `BIGINT` (was reaching
`Number.isInteger(BigInt)` which returns false → wrong default).
5. **Homogeneous array auto-tag**: `${[1,2,3]}` against an `int[]` column
now emits `$1::BIGINT[]` instead of `$1::JSON`. Detection covers
primitive types only (number / bigint / string / boolean); mixed or
nested arrays still fall back to JSON. Mixed int/float widens to
`DOUBLE PRECISION[]`.
6. **`.query()` positional bug**: previously the `.query()` method
abused the template-tag builder, which appended `$N::TYPE` after the
user's literal SQL string instead of binding by position
(`SELECT $1, $2` became `SELECT $1, $2$1::BIGINT`). Now `.query()`
builds the executor-shaped content directly: a `-- $N argN (TYPE)`
declaration block followed by the user's SQL verbatim.
Tests:
- Parser: `test_parse_pg_statement_arg_positions_skips_strings_and_comments`
asserts string literals, comments, and dollar-quoted blocks don't
produce positions (so renumbering doesn't mangle them).
- Executor unit: `renumber_sparse_placeholders_no_collision_no_string_mangling`
uses the new position-aware path and includes string-literal + comment
+ `$$…$$` cases. Existing convert_val tests grow to cover new
String→numeric/real/double/oid/bool arms.
- Integration: `test_postgresql_arg_type_combinations` adds 13 cases
(enum round-trip both directions, string→numeric/real/double/bool/oid,
string-literal `$N` non-mangling). The prepare-fallback test now
asserts SUCCESS (not failure) for enum encoding via AnyTextValue.
- SDK: new `typescript-client/tests/sqlUtils.test.ts` (42 tests)
exhaustively covering inferSqlType primitives + arrays,
parseTypeAnnotation, datatable() template tag (with all the new
shapes — BigInt, homogeneous arrays, RawSql, schema preamble),
datatable().query() positional, and ducklake() shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): replace DISCARD ALL with curated reset (preserves typeinfo cache)
Found while exhaustively probing custom-type DX: every cached-connection
reuse was running `DISCARD ALL`, whose included `DEALLOCATE ALL`
deallocates *all* prepared statements server-side — including the typeinfo
statements that tokio_postgres caches per-Client to resolve custom enum /
domain Oids. tokio_postgres still held `Statement` objects whose names
the server had forgotten, so the next custom-type query failed with
intermittent "prepared statement \"sN\" does not exist" errors. The
failure was easy to reproduce: any sequence that forced typeinfo lookup
for two different custom-type kinds on the same cached connection (e.g.
enum followed by domain) would hit it.
Replace `DISCARD ALL` with a curated reset that explicitly targets the
state we actually care about, *without* touching prepared statements:
RESET ALL — GUC parameters (search_path, application
_name, statement_timeout, …)
RESET SESSION AUTHORIZATION — undoes both `SET SESSION AUTHORIZATION`
and `SET ROLE` (RESET ALL does NOT —
these aren't GUC parameters, so without
this an elevated role from a previous
job would silently leak)
UNLISTEN * — drops LISTEN registrations
CLOSE ALL — closes open cursors
Trade-off: temp tables, advisory locks (session-scoped), and user-created
PREPARE statements may persist across cached-connection reuse — rare in
datatable / PG-script workloads. tokio_postgres's typeinfo cache survives
intact, so custom enum / domain queries are fast on subsequent reuse.
Tests:
- `test_postgresql_custom_types_on_cached_connection` — runs 10×
alternating enum + domain queries on a cached connection. Pre-fix this
failed with `prepared statement "sN" does not exist` after the first
reuse; post-fix passes.
- `test_postgresql_set_role_does_not_leak_across_cached_connection` —
switches `SET ROLE` and `SET SESSION AUTHORIZATION` to a non-postgres
role, then runs a follow-up job and asserts current_user/session_user
are restored. Specifically catches the case where someone might switch
back to `RESET ALL` alone (which doesn't cover SET ROLE / SESSION
AUTHORIZATION) and silently introduce a permission-leak vector.
- All existing session-isolation tests
(`test_postgresql_cached_connection_resets_session`,
`test_postgresql_single_worker_session_isolation`,
`test_postgresql_100_jobs_cached`) continue to pass.
Found via end-to-end probing of datatable / PG-script DX, not previously
covered: the existing isolation tests only did `SET ROLE postgres`, the
connecting user, so the leak was invisible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): address PR #8999 review (cubic + claude)
cubic (P1, real bug):
- `convert_vec_val` for `timetz` array asserted `Type::TIMETZ_ARRAY`, but
chrono `NaiveTime` only encodes for TIME (same caveat as the scalar
arm). Switch to `Type::TIME_ARRAY`; rely on PG's implicit `time→timetz`
assignment cast at the column site. Add an explicit unit test.
claude (#1, silent failure → explicit error):
- `Bool` + explicit `(char)` / `(character)` decl previously silently
bound BOOL, hoping the server would cast at the use site — but PG has
no implicit `bool→char` and the resulting error
("operator does not exist: bool = char") was opaque. Now error at
bind time with an actionable hint to use `bool` decl or pass the
value as a "t"/"f" string.
claude (#2, asymmetry doc):
- Object/Array still coerce to text on `matches!(typ, Typ::Str(_))`
(covers both explicit AND inferred-default text), unlike Bool/Number
which key on `explicit_text_target`. The asymmetry is intentional
(no implicit `jsonb → text` cast in expression context vs PG having
implicit `bool/int → text` casts) — added a body comment so future
maintainers don't try to "align" them.
claude (#3, perf):
- `parse_pg_statement_arg_indices` and `parse_pg_statement_arg_positions`
walked the SQL tokenizer twice. Fold into a single pass that derives
the index set from the position list.
claude (#4, fmt drift):
- `cargo fmt` over the parser crates I touched with perl scripts in the
earlier commit (windmill-parser-{sql,bash,ts,go,php,java,csharp,nu,py,
rust,graphql,yaml,r}). Net cosmetic.
claude (#5, parseTypeAnnotation):
- One-line caveat in the SDK's `parseTypeAnnotation` that the returned
string is presence-only (e.g. `${x}::DOUBLE PRECISION` returns
`"DOUBLE"`, `CAST(${x} AS int)` returns `"int)"` — neither matches a
real PG type, but the only consumer just checks `!== undefined`).
While here — discovered + fixed independently while exhaustively probing
DX:
- **Replace `DISCARD ALL` with curated reset** (`RESET ALL; RESET
SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`). DISCARD's
`DEALLOCATE ALL` killed tokio_postgres' typeinfo cache, producing
intermittent `prepared statement "sN" does not exist` errors on
custom-type queries after cached-conn reuse. New regression tests:
`test_postgresql_custom_types_on_cached_connection` and
`test_postgresql_set_role_does_not_leak_across_cached_connection`
(the latter catches the case where someone might switch back to
`RESET ALL` alone and silently introduce a permission-leak vector —
RESET ALL doesn't cover SET ROLE / SET SESSION AUTHORIZATION).
- **ISO-8601 timestamp results** (`pg_cell_to_json_value`). Pre-fix
`TIMESTAMP` was rendered with a space separator ("2024-01-15 10:30:00")
and `TIMESTAMPTZ` with " UTC" suffix ("2024-01-15 10:30:00 UTC") —
neither parseable by `date-fns parseISO`, JavaScript `new Date()` is
lenient enough to handle them but several frontend `App*Input.svelte`
components use parseISO and fail silently. Switched to ISO-8601 with
`T` separator and `+00:00` offset; arg-parsing path still accepts the
legacy " UTC" suffix for back-compat.
Test coverage:
- 17/17 unit (`pg_executor::tests`)
- 9/9 integration (`backend/tests/worker.rs`, `test_postgresql_*`)
- 27/27 parser (`windmill-parser-sql`)
- 42/42 SDK (`typescript-client/tests/sqlUtils.test.ts`)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): bounded one-shot warning on numeric precision loss + ISO-8601 + NaN handling
Found while probing PG-script DX with millions of numeric cells:
1. **Numeric precision-loss warning**: `numeric` results are still serialised
as JSON Number (back-compat — switching to JSON String would silently
break user code doing arithmetic on results), but we now detect
`Decimal -> f64 -> Decimal` round-trip failure and emit a single
job-log warning recommending a `::text` cast in the SQL. Bounded by
`NUMERIC_PRECISION_CHECK_BUDGET = 256` cells per query (one atomic
load + one fetch_sub on the hot path; first lossy value
short-circuits to a single load thereafter). Worst-case overhead on
a 1M-cell numeric-heavy query: ~25µs of checks + 5ns × N atomic
loads (vs. ~100ms unbounded).
2. **ISO-8601 timestamps**: `pg_cell_to_json_value` previously returned
`"2024-01-15 10:30:00"` (TIMESTAMP) and `"2024-01-15 10:30:00 UTC"`
(TIMESTAMPTZ) — neither parseable by date-fns `parseISO`, which is
what the apps `App*Input.svelte` components use, so timestamp values
silently failed to round-trip into date pickers. Switch to ISO-8601
(`T` separator + `+00:00` offset) on the result side; arg-parser
continues to accept the legacy `" UTC"`-suffixed format for
back-compat.
3. **Float NaN / Infinity results**: `Number::from_f64` returns None for
NaN / ±Inf, which `pg_cell_to_json_value` was raising as
"invalid json-float" — failing the *entire* query if any cell held
one of these special values. Now serialise them as JSON strings
("NaN", "Infinity", "-Infinity") and let the rest of the row come
through. Arg-side: `s.parse::<f64>()` already accepts the same
strings.
Tests:
- `decimal_fits_f64_losslessly_predicate` — covers fits / doesn't-fit
cases for the precision-loss predicate.
- `precision_check_budget_caps_per_query_overhead` — locks in the
budget cap and the loss-flag short-circuit.
- All 9 PG integration tests + 17 unit tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): add pg_advisory_unlock_all to reset; warn on missing args; honor decl defaults
While probing PG-script DX further found three more frictions:
1. **Advisory lock leak** (cubic P2): switching from `DISCARD ALL` to
`RESET ALL; RESET SESSION AUTHORIZATION; UNLISTEN *; CLOSE ALL;`
meant session-scoped advisory locks (`pg_advisory_lock`) leaked
across cached-connection reuse. Add `SELECT pg_advisory_unlock_all()`
to the chain — `DISCARD ALL` covered this implicitly via
`DISCARD PLANS / DEALLOCATE / pg_advisory_unlock_all` and we lost it
in the switch.
2. **Missing-arg silent NULL**: an arg declared in the SQL (e.g.
`-- $1 amount (numeric)`) but not provided in the args object was
bound as NULL with no error / warning. Misspelling the key in the
args object silently produced a row of NULLs — a notorious DX
debugging trap. Now: collect the names of declared-but-missing
args during dispatch and emit a single one-shot warning to the job
logs at end-of-query naming each one. Bound NULL is preserved for
back-compat.
3. **Declaration defaults ignored**: `-- $1 a (int) = 5` carries
`arg.default = Some(Number(5))`, but the dispatch fell straight to
NULL when the arg was missing. Now: respect the default —
user-supplied value > declaration default > NULL. Also fixes the
warning logic above (only warn for args that *don't* have a default).
Tests: existing 19 unit + 9 integration pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(pg): multi-word PG types with [] suffix lost the array-ness; array arms accept stringified values
Two more frictions found while probing SDK end-to-end against a real
datatable resource:
1. **Multi-word array types lose the [] suffix in the parser**.
`transform_types_with_spaces` recognises aliases for "double
precision", "character varying", "timestamp with time zone", etc.
but its return type was `&'a str` — only the bare alias, never with
a trailing `[]`. The `RE_CODE_PGSQL` regex's `\w+` captures stop at
the first space, so the regex's own `(?:\[\])?` array-suffix branch
sees only `"double"` (not `"double precision[]"`); the `[]` was
silently lost. Result: `$1::double precision[]` (which the SDK now
emits for homogeneous float arrays via the new auto-tag) routed
through `Value::Array → Type::JSONB` and the server failed with
"cannot cast type jsonb to double precision[]".
Fix: switch `transform_types_with_spaces` to return `Cow<'a, str>`
and re-check the trailing bytes after a multi-word match. If they
start with `[]`, return `format!("{alias}[]")` — Owned. Single-word
types and the no-match path keep returning Borrowed slices, so no
allocation in the hot path.
2. **Array arms in `convert_vec_val` rejected stringified values for
numeric / int* / bool / oid / real / double**. The scalar `convert_val`
already parses strings into the matching native type for these arg_ts,
but the array variant only accepted JSON-native counterparts. Sending
`["1.5", "2.5", "3.5"]` against `$1::numeric[]` (e.g. via `unnest` for
bulk loading, or `JSON.stringify(BigInt[])` round-trip) failed with
"Mixed types in array". Now the array arms mirror the scalar ones —
`as_<native>().or_else(|| as_str().and_then(parse))` — so both shapes
round-trip cleanly.
Tests: 19 unit + 9 integration pass; existing parser tests cover the
multi-word array forms (the regex-cap behaviour didn't break for
single-word types, and Cow plumbing is transparent to all callers).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(parsers): add otyp_inferred field to Arg literals in tests + 3 missed src files
CI failures: the perl-driven sweep that added `otyp_inferred: false` to
every `Arg { ... }` literal when I introduced the field in the parser
schema covered `src/lib.rs` files but missed:
- parsers/windmill-parser-bash/src/lib.rs (mass-edited but a
later format pass un-applied a few sites)
- parsers/windmill-parser-go/src/lib.rs (same)
- parsers/windmill-parser-graphql/src/lib.rs (same)
- parsers/windmill-parser-nu/tests/tests.rs (test file — not
swept the first time)
- parsers/windmill-parser-ts/tests/tests.rs (test file — same)
Also tightened the regex to handle `oidx: None` without the trailing
comma (some test files had the field as the last initialiser line).
`cargo build --features <CI feature combo> --workspace --all-targets`
is clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(sdk): Date → TIMESTAMPTZ; NaN / ±Infinity → string
Two more frictions found while running the actual SDK end-to-end against
a live datatable resource:
1. **JS `Date`** fell into the typeof "object" branch and was tagged
`::JSON`. It worked accidentally for `${date}::timestamptz` via PG's
`json → text → timestamptz` implicit cast chain, but `${date}` against
a `timestamptz` column without a user-supplied cast bound the value
as a JSON string and the comparison `timestamptz = json` failed. Now:
`inferSqlType` recognises `Date` and tags `::TIMESTAMPTZ`;
`serializeArgValue` emits `Date.toISOString()` so the executor's
`Value::String → TIMESTAMPTZ` arm parses it cleanly.
2. **JS `NaN` / `±Infinity`** silently became NULL. `JSON.stringify(NaN)`
returns `"null"` per the JS spec, so the value reached the executor as
JSON null — the SDK's `::DOUBLE PRECISION` tag then bound a NULL
double. Fix: detect non-finite numbers in `serializeArgValue` and
stringify them as `"NaN" / "Infinity" / "-Infinity"`. The executor's
`Value::String → FLOAT8` arm (`f64::from_str`) accepts these literals
directly, and the result-side already renders the values as JSON
strings (matching round-trip).
SDK unit tests grow from 42 → 44 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(pg): integration coverage for multi-word arrays + stringified array elements
Locks in the two array fixes from the previous commit
(`fix(pg): multi-word PG types with [] suffix lost the array-ness`)
with end-to-end cases in `test_postgresql_arg_type_combinations`:
- `double precision[]`, `character varying[]`, `timestamp without time
zone[]` — verifies the parser keeps the `[]` suffix after multi-word
alias resolution.
- `numeric[]` / `int[]` / `bool[]` from stringified primitives — verifies
the array arms of `convert_vec_val` apply the same string-coercion
the scalar arms do.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* style: fix indentation drift on otyp_inferred lines
cargo fmt cleanup of leftover indentation where the perl-driven sweep
that introduced the otyp_inferred field landed at the wrong column.
No behaviour change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* docs(skill): document email triggers and S3 attachments
Add an "Email triggers" section to the triggers skill covering the
local-part config, the parsed_email/raw_email/email_extra_args payload,
the URL-style extras convention, where to find trigger_path (only with
a preprocessor, at event.trigger_path), and — most importantly — that
binary attachments are uploaded to the workspace S3 bucket and surface
as `{ s3: "windmill_emails/<job_id>/attachments/<filename>" }`. Scripts
must use wmill.loadS3File / wmill.load_s3_file to read them.
Also pulls EmailTrigger into the schema mappings so a real
`email_trigger.schema.yaml` is generated, and adds Email/Azure to the
trigger kinds list in the CLI agent guidance.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref for email trigger path fix
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to 26184ab7a4aadfc529dcedf038aa08d36c7ad381
This commit updates the EE repository reference after PR #553 was merged in windmill-ee-private.
Previous ee-repo-ref: 318a46897a605dc9be3817901f35ba5a99a0a525
New ee-repo-ref: 26184ab7a4aadfc529dcedf038aa08d36c7ad381
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
tracing-opentelemetry only recognizes otel.status_code and
otel.status_message as fields that map to the OTLP Status proto.
The previously-used otel.status_description fell through to the
generic attribute recorder, leaving Status.message unset and
preventing OTLP consumers from filtering spans on error status.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: ansible delegate_to_git_repo install_requirements, dynamic fields, --limit
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: include .yaml variants in collections/roles requirements lookup
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: redact GitHub App tokens and Slack OAuth secret for non-admins
`GET /workspaces/get_settings` returned the full `git_app_installations`
JSONB to any workspace member. That column caches the GitHub App JWT and
installation token used by git-sync; the installation token is refreshed
on every git-sync action and valid for ~55 minutes, so the value sitting
in the DB is essentially always live. Null it out for non-admins,
matching the existing `slack_oauth_client_secret` redaction.
The tarball export's v2 settings format (added in #8935) included
`slack_oauth_client_secret` with no admin gating, regressing the same
redaction. Mirror the admin check there.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: split get_settings into admin-only + public endpoint
Adds `WorkspacePublicSettings` and `GET /workspaces/get_public_settings`,
which returns only fields safe for any workspace member to read
(workspace_id, slack/teams team identity, mute_critical_alerts, deploy_ui,
large_file_storage, datatable). `get_settings` is now admin-only via
`require_admin`.
Migrates frontend callers: every caller that read non-sensitive fields
(deploy_ui on trigger pages, mute_critical_alerts on the root layout, slack
team identity for handler pickers, etc.) now uses `getPublicSettings`. The
admin-managed settings UI, git-sync admin context, operator settings,
checkout polling, and full settings page stay on `getSettings`.
This replaces the field-level redactions added in the previous commit:
the type system itself defines the public surface, so adding a sensitive
column to `workspace_settings` no longer defaults to leaking — it stays
out of `WorkspacePublicSettings` unless explicitly added.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: sanitize underscores in agent worker suffix
The agent worker token wire format is `jwt_agent_<suffix>_<JWT>`, parsed
server-side with `split_once('_')`. JWTs themselves can contain `_`
(base64url alphabet), so the only sound boundary is "suffix has no `_`".
`instance_name()` is the source of the suffix and previously only
sanitized spaces and `-`. A hostname like `austin_hp` would produce
`worker_suffix=austin_hp-<rand>`, the token `jwt_agent_austin_hp-X_<JWT>`
would split as `("austin", "hp-X_<JWT>")`, and the JWT decoder would
return `InvalidToken` on the garbage second half, surfacing as a 401 on
`/api/agent_workers/update_ping`.
Replace `_` with `-` in the hostname-derived suffix so the parsing
boundary stays unambiguous. Pure source-side fix; no wire format change,
no migration needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: apply underscore->dash before splitting in instance_name
Replace `_` with `-` BEFORE the `split("-").last()` step so underscores
behave the same as dashes (consistent with the split-on-dash idiom) and
the resulting instance_name is a single token. For `austin_hp` this
yields `hp` rather than `austin-hp`. No behavior change for hostnames
without `_`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: strip underscores from hostname instead of replacing with dash
Removing `_` preserves more identifier info than replacing with `-`:
`austin_hp` becomes `austinhp` (single useful token) rather than `hp`
(prefix lost to the dash-split). For k8s-style hostnames that already
contain `-`, the `-` continues to do the splitting and `_` is just a
stray character to strip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes silent timeouts and partial-tree rendering when loading large git
repositories into the in-app viewer (Tony Hoang report: 400+ host_vars
files, 32 roles).
Three coordinated fixes:
1. Frontend (GitRepoViewer.svelte): drop the 60s clone timeout. Long-poll
getJobUpdates until the job completes, with a 30 min hard cap and a
user-cancel button. Stream live job logs into the viewer with a link
to the full job page. After success, verify the
.windmill_clone_complete marker before flipping pathExists, so a
partial S3 directory is no longer rendered as a complete tree.
2. Backend (check_s3_folder_exists, EE): new optional marker_file query
param. When set, the handler short-circuits to head() on the marker
object instead of "any object under prefix exists". The frontend now
always passes marker_file=.windmill_clone_complete.
3. Hub script: cloneRepoToS3forGitRepoViewer points at hub/28216, which
uploads files via a bounded-concurrency pool (16 workers), emits
throttled progress logs, and writes .windmill_clone_complete as its
last action. docs/clone_repo_and_upload_to_instance_storage.bun.ts is
the source for that hub publish; docs/git-repo-viewer-hub-script.md
explains the change.
Also drops three unused legacy hubPaths entries
(cloneRepoToS3forGitRepoViewer_0..2) — none were referenced from
anywhere in the codebase.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Always send queries as unnamed prepared statements (query_typed_raw)
when arg types resolve via otyp_to_pg_type. This eliminates the
intermittent "prepared statement \"sN\" does not exist" error reported
on datatable scripts whose Postgres connection sits behind a
transaction-mode pooler (PgBouncer/Supabase pooler/RDS Proxy), where
prepare and execute can land on different backend connections.
The previous code only used the unnamed-statement path when the parser
detected at least one explicitly typed arg; datatable-generated SQL
with bare $1/$2 (relying on inline ::cast hints) fell back to
prepare + query_raw and accumulated named statements (s0, s1, ...,
s882, ...) on the cached connection, which the pooler then dropped.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The get_flow_by_path_w_draft endpoint omitted flow.labels from its
SELECT and FlowWDraft struct, so the flow editor received undefined
labels. As a result, the labels input rendered empty even when the
flow had labels saved, and adding a new label overwrote the existing
ones (since the frontend sent only the new label and the update SQL
only preserves labels when the field is null).
Closes#8963
* test: isolate WAC v2 python test from test-thread stack overflow
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* ci: bump RUST_MIN_STACK to 4MB for backend tests
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: edit scopes on existing API tokens
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address PR review feedback on token scope edit
- add SECURITY DEFINER to notify_token_scopes_change so trigger fires under windmill_user/admin roles (cubic P1)
- drop banned $bindable(default) on optional props (CLAUDE.md): make ScopesPicker.value and EditTokenScopesModal.open required
- detect MCP only when *every* scope starts with mcp: so mixed/null-scope tokens fall back to standard picker without dropping non-mcp scopes
- audit log scope payload via serde_json instead of Rust {:?}
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add workspace-shared ui/ folder reusable across raw apps
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add shared ui/ drawer in raw app editor sidebar
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: forward workspace shared ui/ to raw app editor iframe
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* all
* all
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: add delete_after_secs and sensitive_inputs to raw app policy
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: simplify sensitive toggle label
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: use tertiary text for sensitive toggle label
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: unset sensitive field when toggled off
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address PR review feedback
- plumb force_viewer_sensitive_inputs/delete_after_secs so editor preview
matches deployed-mode encryption
- reuse resolve_delete_after_secs helper for consistency with scripts/flows
- log+ignore schedule_job_deletion errors so a failed schedule doesn't
surface as an execute_component failure
- fix text-primay typo in CacheTtlPopup and DeleteAfterUsePopup
- tighten extraFields return type to Partial<Pick<...>>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: set OTEL span status on failed jobs and add stderr severity toggle
Record otel.status_code=ERROR and otel.status_description on the job /
job_postprocessing tracing spans when a job fails, so OTel exporters
(Sentry, Honeycomb, Datadog) see the standard span-level failure signal
instead of just the success=false attribute. Description is truncated to
512 chars to keep span payloads bounded.
Add OtelSettings.stderr_default_severity instance setting (error | warn |
info | debug, default error) to let operators downgrade the OTEL severity
used for job stderr output. Python logging routes every record >= WARNING
to stderr, so the historical blind stderr->error mapping produces false
positives for scripts like dlt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref for stderr severity toggle
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: clarify OTEL span description for already-completed jobs
When handle_queued_job returns Ok(false) on Error::AlreadyCompleted
(another worker already finished the job during a race), the span
was labeled "job returned false" which is opaque to operators.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: parse Python logging severity from job stderr
Replace the global stderr_default_severity instance toggle with a
worker-side classifier that recognizes Python's canonical
basicConfig() format (LEVELNAME:logger.name:message) and emits the
corresponding tracing level. Lines that don't match keep the
historical tracing::error! fallback, so genuine failures still
surface and non-Python output is unaffected.
Removes StderrLogSeverity, STDERR_LOG_SEVERITY, and the
otel.stderr_default_severity field; adds
classify_python_logging_line in windmill-common.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: cover classify_python_logging_line + fix truncate_description doc
Add unit tests for the Python stderr-severity classifier and correct
the truncate_description docstring to say "bytes" (the cap is byte-
based, with UTF-8 boundary rounding for safety). Addresses Claude
review feedback on PR #8918.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
* feat: support S3Object input args in native SQL scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: review fixes from local-review
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* update parser
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: support restart from steps inside BranchOne, ForLoop, Subflow
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: preserve original job kind in nested restart, support expanded subflow steps
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: read selected iteration from graph state for nested ForLoop restart
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: iteration selectors per ForLoop in restart popup, more nested restart tests
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: extract useNestedRestartState composable
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: cover deployed-subflow + FlowDependencies path in nested restart
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: update sqlx prepare cache
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: detect BranchOne/ForLoop ancestors inside expanded subflows for nested restart
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: hide restart button for non-restartable steps (parallel containers, untaken branches)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address review feedback on nested restart PR
- preview FlowRestartButton: hide nested case (chain UUIDs aren't resolvable in
preview path; users can use the run page for nested restart instead)
- branchOneAncestorMatchesOriginal: be permissive when status isn't reachable
(don't hide the button for BranchOnes nested deeper than top-level)
- worker_flow.rs: apply nested_restart_payload swap on the is_simple ForLoop
fast path too, so simple iterations don't bypass restart spawn interception
- FlowStatusViewer: reset expandedSubflows cache on jobId change; drop
$bindable({}) banned pattern for the new prop
- API resolver: validate the leaf step exists before returning (fail-fast)
- doc fix: branch_or_iteration_n is 0-based, not 1-based
- selectedJobStepIsTopLevel reset on early-return in composable
- comment iterationCounts collision caveat
- new HTTP-level integration tests covering the API endpoint contract:
happy path (top-level + nested), unknown step, out-of-range iteration,
parallel-loop rejection
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* revert: remove unreachable nested-restart swap on is_simple ForLoop fast path
The swap is unreachable in valid flows: `is_simple_modules` requires the body
to be a single `script` / `rawscript` / `flowscript` (per `FlowModule::is_simple`),
none of which spawn flow-kind children. Any nested-restart chain targeting a
leaf inside such an iteration is rejected by the API at leaf validation. Even
if a chain reached the worker via `JobPayload::RawFlow.restarted_from`, the
resulting `RestartedFlow` would fail to push (script kind isn't a flow kind).
Replaced the swap with an explanatory comment so the next reader knows why
the symmetry with the non-simple path was deliberately not added.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: handle undefined expandedSubflows + tighten branchOne match check
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor: split flow-dep job tx so subprocesses don't hold row locks
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: link flow version from run page to pinned flow viewer
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: time-out dep job phase 1/3 db ops and surface error on flow page
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: dissolve dep_map in phase 1 and recheck flow version unconditionally
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address PR review — view-latest reload, decimal truncation, app version
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: keep dissolve in phase 3 for relative-import dep jobs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: trim verbose comments and refresh sqlx offline cache
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: address cubic — propagate dissolve errors, include workspace in reload key
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor
Three workflow-as-code bug fixes:
- #8945: Python WAC template with `@workflow async def main(...)` was not
detected as `auto_kind = "wac"`. The detection only ran when no `main`
function was found. Hoist the heuristic so it runs whether or not `main`
is the entrypoint.
- #8946: `scripts/list?kinds=script` filtered out WAC scripts because they
set `auto_kind = 'wac'` and the SQL hid everything that wasn't NULL.
Allow both NULL and 'wac' (still excluding 'lib' library scripts).
- #8947: Preprocessor functions defined alongside a WAC workflow were
ignored. Inject the preprocessor invocation into the Python WAC wrapper
so it runs before the workflow on the first iteration, then plumb the
preprocessed args through `handle_wac_v2_output` so inline child
re-runs see the post-preprocessor args via `checkpoint.input_args`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(wac): integration tests for #8946 (scripts/list) and #8947 (preprocessor)
- test_scripts_list_includes_wac: hit GET /scripts/list?kinds=script and
assert WAC scripts are in the response (would have failed pre-#8946 fix
because of the auto_kind IS NULL filter).
- test_python_wac_v2_with_preprocessor: deploy a Python WAC script with a
preprocessor, run with raw event args, assert the workflow saw the
preprocessed shape and v2_job.args/preprocessed were updated.
- New wac_preprocessor.sql fixture with auto_kind = 'wac' set explicitly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(wac): address PR review feedback
Five review fixes:
- python_executor.rs: WAC preprocessor now runs inside the wrapper's
`try:` block so failures route through the same `result.json` error
serializer as workflow failures. Switched async-coroutine handling
from deprecated `asyncio.get_event_loop().run_until_complete(...)` to
`asyncio.run(...)` (the recommended primitive on 3.10+).
- bun_executor.rs: when copying preprocessed args into
`checkpoint.input_args`, surface JSON parse failures via `?` instead
of silently coercing to `Value::Null` (which would persist a corrupted
arg into every child re-run). Also collapsed the redundant double
iteration into a single pass.
- windmill-api-scripts/scripts.rs: switched the runnable-script filter
from an allow-list (`auto_kind IS NULL OR = 'wac'`) to a deny-list
(`<> 'lib'`), so future `auto_kind` values aren't silently filtered
from triggers/dropdowns.
- windmill-parser-py: aligned the parser's WAC heuristic with the
runtime detector `is_wac_v2_py` — `@task` is now optional, matching
the runtime which says workflows that only use inline `step()` are
still WAC. Added a regression test `test_parse_python_wac_step_only`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nativets): forward OTEL-prefixed console logs to tracing events
Nativets jobs run in-process and bypass the handle_child.rs stdout loop
where `OTEL: ` lines are turned into `tracing::event!` calls when
`OTEL_JOB_LOGS=true`. Apply the same prefix handling in the nativets
log receiver so `console.log("OTEL: ...")` reaches the OTEL exporter
like it does for other runtimes.
Moves `OTEL_JOB_LOGS` and `OTEL_PREFIX` into windmill-common so both
crates share the same definition.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(nativets): emit job_log tracing target so logs reach OTEL bridge
For non-native runtimes, `lines_to_stream` → `process_streaming_log_lines`
(EE) emits every stdout line as `tracing::info!(target: "windmill:job_log", ...)`,
which is picked up by the EE `LogContextBridge` and exported to OTEL
(the bridge's filter is `EnvFilter` only, not the targets filter that
drops `windmill:job_log` from stdout/file sinks).
Nativets delivers logs in-process via a channel, so it never goes
through that path and console.log output only reached the Windmill UI.
Emit the same `windmill:job_log` event per line from the nativets log
receiver.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruben FiszelandClaude Opus 4.7
centdixandClaude Opus 4.5