- duration/mem-peak labels replaced by Timer/Cpu icons with tooltips
- 'Auto scroll' -> 'auto-scroll' (lowercase, hyphenated, whitespace-nowrap)
- top bar gets overflow-x-auto so it scrolls horizontally instead of pushing the entire log panel into overflow when narrow
Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`)
with no args. The local path defaults to CWD, and the remote path is
derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/
`__app`/`__raw_app` suffixes stripped. Either, both, or neither
positional argument can be passed.
Also resolves `file_path` against the user's original CWD before
`resolveWorkspace` may chdir to the wmill.yaml root, so a relative
`file_path` argument is interpreted from where the user invoked the
command (previously it could resolve against the wrong directory).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.
Three changes that together stop the propagation:
1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
confirm every listed path exists on disk before writing
`.valid.windmill`. On failure, wipe the directory and fall through
to a fresh local install — the next install also self-heals the
broken object-store entry by pushing a fresh tar.
2. After `uv pip install` succeeds, run the same RECORD check before
queuing the piptar upload or writing `.valid.windmill`. A bad install
never becomes the source of a broken tar in the object store.
3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
for upload, so we never push an unfinalized archive (no end-of-archive
marker) to the object store.
Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: parse windmill_failure field in job result to tag run as failure
* feat: preserve top-level fields when windmill_failure tags a run as failure
* fix: address review findings on windmill_manual_failure
* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases
* fix: prefer injected ManualFailure error over sibling name/message in OTel
* fix(cli): bump svelte version in `wmill app new` template
The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte
compiler bundled in `wmill app dev` emits `$.delegated('click', ...)`
calls. The `delegated` export was added later, so 5.45.2 doesn't have
it — esbuild warns `Import "delegated" will always be undefined`,
replaces the call with `void 0`, and the page crashes at first
event-handler bind (white screen).
Bump to `^5.55.5` so the compiler and runtime stay in sync.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): bump svelte version in raw_apps UI template
Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a
package.json with `svelte: "5.45.2"`. That works today only because
the bundled rolldown worker also pins 5.45.2 — when the worker is
upgraded past 5.51.1, the compiler will emit `$.delegated()` and the
runtime won't have it, producing the same white-page crash that hit
the CLI.
5.55.5 still exports `event` (used by the current bundled compiler),
so this is forward-compatible: it works with the 5.45.2 compiler now
and won't break when the worker is upgraded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(native-triggers): serialize Google channel renewal across replicas
`sync_all_triggers` runs every 5 minutes on every windmill-app replica
with no leader election. Multiple replicas were each rotating the
webhook token, creating a new Google watch channel, and racing the
trigger UPDATE — leaving the loser's new token (in `token`) and channel
(in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week
without the silent best-effort `delete_token_by_hash` ever logging a
warning.
Wrap each per-trigger renewal in a transaction and acquire the row with
`SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row
instead of duplicating the work. The lock spans `rotate_webhook_token`
→ Google API call → `update_native_trigger_service_config` and is only
released on commit. Re-checks `should_renew_channel` after acquiring
the lock so a replica that committed seconds earlier doesn't trigger a
duplicate renewal.
The pattern matches existing batch-cleanup paths in `monitor.rs`
(job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites.
Also logs at `debug!` when `delete_token_by_hash` finds no matching row,
so future investigations can distinguish "deleted" from "not found"
without changing the `Ok(false)` contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Address claude review:
- #5: per-skip log info -> debug (expected outcome under SKIP LOCKED)
- #2: warn moved out of delete_token_by_hash to the call site that knows the
expected state (try_renew_channel_locked); other callers are race-prone and
shouldn't warn
- #3: NULL service_config now warns (anomalous case)
- #4: post-Google-API DB-update + commit failures log distinctly so the
channel-orphan case is grep-able
Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration,
mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the
existing 'ephemeral-' filter excludes them from user-token email/critical-alert
paths (no filter changes in 3 places). Orphans now self-clean via the existing
expiry sweep in monitor.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Address second-round review:
- Claude #1 (P2): username_override_from_label now strips the 'ephemeral-'
prefix for ephemeral-webhook-* labels, so created_by stays
webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-...
(preserves audit/job-list filter compatibility)
- Codex (P2): updated renew_channel doc — labels are no longer copied; rotate
mints fresh ephemeral-webhook-google-{rd5} with 14d expiration
- Claude #3 (optional): test_rotate_webhook_token now asserts the rotated
Google token has an ephemeral-webhook-google-* label and a populated
expiration
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Reconsider the previous fixup: stripping the 'ephemeral-' prefix made
created_by no longer match token.label exactly, defeating the linking
purpose. Just allowlist 'ephemeral-webhook-' alongside the other
recognized webhook/email/ws prefixes — created_by becomes
ephemeral-webhook-google-XXXXX, matching token.label exactly. The
'ephemeral-' substring also informs operators that this is a
system-managed auto-expiring token vs a user-managed webhook trigger.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli-tests): stabilize flow lock-gen race + Windows path
Three CLI test failures on the latest main, all flaky on CI:
1. `Mixed Case Paths: pull and push flow with capitalized folder` and
`Integration: Mixed scripts and flows with nonDottedPaths are
idempotent`: flow create/update queues an async FlowDependencies job
that fills inline-script lockfiles and rewrites flow.value. The tests
pulled/pushed before the worker finished, so dry-run idempotency saw
phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a
`waitForFlowDependencyJob` helper that polls `/flows/get` for the
latest `dependency_job` and `/jobs_u/completed/get` until it lands,
and called it after each API/CLI flow write in both tests.
2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only,
added in #9075): the new test built the CLI entrypoint via
`new URL("..", import.meta.url).pathname`, which yields `/C:/...` on
Windows and `Bun.spawn` rejected before reaching the proxy, leaving
`rejectedRequests.length` at 0. Switched to
`fileURLToPath` + `node:path.join` to match `cargo_backend.ts`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli-tests): use /flows/deployment_status to actually wait for dep job
CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob`
as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`,
but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60)
do not include that field. The helper exited on the first iteration
without polling.
Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns
`{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID
written into `deployment_metadata` in the same tx as the dep-job push
(backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by
the time the create/update API call returns, the response carries the
latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as
before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to
~32s, confirming the helper now actually waits instead of returning
immediately. The 404 short-circuit in `sync_pull_push.test.ts` still
works — `get_deployment_status` returns 404 when the flow is absent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped
the `HEADERS` env var, causing requests to fail behind auth gateways like
Cloudflare Access (same shape as #6421):
- `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed
in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw
`fetch` for the `skip_if_noop` query param.
- Script preview `/jobs/run/preview_bundle`.
- App dev `/jobs_u/getupdate_sse` SSE stream.
- `wmill docs` `/api/inkeep`.
All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()`
so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque
JSON parse failure.
Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that
403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and
otherwise reverse-proxies to the test backend, then runs `wmill sync push`
of a fresh script through the proxy. Negative case (no `HEADERS` env)
verifies the proxy actually gates; positive case asserts every request
including `/scripts/create` reaches the backend with the headers attached.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Run `wmill app push` from inside an app folder (e.g. `f/foo/my_app.app/`)
with no args. The local path defaults to CWD, and the remote path is
derived from CWD relative to `wmill.yaml`, with `.app`/`.raw_app`/
`__app`/`__raw_app` suffixes stripped. Either, both, or neither
positional argument can be passed.
Also resolves `file_path` against the user's original CWD before
`resolveWorkspace` may chdir to the wmill.yaml root, so a relative
`file_path` argument is interpreted from where the user invoked the
command (previously it could resolve against the wrong directory).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Python per-package dependency cache could persist an incomplete wheel
extraction with `.valid.windmill` set, then propagate that broken artifact
to every worker through the object store. Customer hit this on
argon2-cffi==25.1.0 (missing argon2/_utils.py), and previously on
botocore/httpx (truncated tars). Symptom is a runtime ImportError that
looks like a missing dependency declaration rather than a Windmill bug.
Three changes that together stop the propagation:
1. After `pull_from_tar`, parse the wheel's `<dist-info>/RECORD` and
confirm every listed path exists on disk before writing
`.valid.windmill`. On failure, wipe the directory and fall through
to a fresh local install — the next install also self-heals the
broken object-store entry by pushing a fresh tar.
2. After `uv pip install` succeeds, run the same RECORD check before
queuing the piptar upload or writing `.valid.windmill`. A bad install
never becomes the source of a broken tar in the object store.
3. Finalize the tar (`drop(tar.into_inner()?)`) before reading its bytes
for upload, so we never push an unfinalized archive (no end-of-archive
marker) to the object store.
Verified with a 60-package end-to-end integration test (first-fill →
clear-local-cache → re-pull-from-objectstore → corrupt-objectstore-tar
→ detect-and-self-heal). All 27 packages on the live test pulled cleanly,
and the deliberately corrupted argon2-cffi tar was caught with the exact
expected log line ("wheel RECORD lists files missing on disk: argon2/_utils.py")
and replaced with a fresh tar.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: parse windmill_failure field in job result to tag run as failure
* feat: preserve top-level fields when windmill_failure tags a run as failure
* fix: address review findings on windmill_manual_failure
* refactor: rename windmill_manual_failure to wm_failure and add wm_* aliases
* fix: prefer injected ManualFailure error over sibling name/message in OTel
* fix(cli): bump svelte version in `wmill app new` template
The svelte5 template pinned `svelte` to `5.45.2`, but the Svelte
compiler bundled in `wmill app dev` emits `$.delegated('click', ...)`
calls. The `delegated` export was added later, so 5.45.2 doesn't have
it — esbuild warns `Import "delegated" will always be undefined`,
replaces the call with `void 0`, and the page crashes at first
event-handler bind (white screen).
Bump to `^5.55.5` so the compiler and runtime stay in sync.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(frontend): bump svelte version in raw_apps UI template
Mirror the CLI fix: the UI's `Add raw app` flow scaffolds a
package.json with `svelte: "5.45.2"`. That works today only because
the bundled rolldown worker also pins 5.45.2 — when the worker is
upgraded past 5.51.1, the compiler will emit `$.delegated()` and the
runtime won't have it, producing the same white-page crash that hit
the CLI.
5.55.5 still exports `event` (used by the current bundled compiler),
so this is forward-compatible: it works with the 5.45.2 compiler now
and won't break when the worker is upgraded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(native-triggers): serialize Google channel renewal across replicas
`sync_all_triggers` runs every 5 minutes on every windmill-app replica
with no leader election. Multiple replicas were each rotating the
webhook token, creating a new Google watch channel, and racing the
trigger UPDATE — leaving the loser's new token (in `token`) and channel
(in Google) orphaned. Cloud was accumulating ~5 leaked tokens/week
without the silent best-effort `delete_token_by_hash` ever logging a
warning.
Wrap each per-trigger renewal in a transaction and acquire the row with
`SELECT … FOR UPDATE SKIP LOCKED`. Contending replicas skip the row
instead of duplicating the work. The lock spans `rotate_webhook_token`
→ Google API call → `update_native_trigger_service_config` and is only
released on commit. Re-checks `should_renew_channel` after acquiring
the lock so a replica that committed seconds earlier doesn't trigger a
duplicate renewal.
The pattern matches existing batch-cleanup paths in `monitor.rs`
(job-retention sweep) and other `FOR UPDATE SKIP LOCKED` call sites.
Also logs at `debug!` when `delete_token_by_hash` finds no matching row,
so future investigations can distinguish "deleted" from "not found"
without changing the `Ok(false)` contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Address claude review:
- #5: per-skip log info -> debug (expected outcome under SKIP LOCKED)
- #2: warn moved out of delete_token_by_hash to the call site that knows the
expected state (try_renew_channel_locked); other callers are race-prone and
shouldn't warn
- #3: NULL service_config now warns (anomalous case)
- #4: post-Google-API DB-update + commit failures log distinctly so the
channel-orphan case is grep-able
Plus: add 14d expiry to Google webhook tokens via ServiceName::webhook_token_expiration,
mint fresh ephemeral-webhook-{service}-{rd5} labels at create + rotate so the
existing 'ephemeral-' filter excludes them from user-token email/critical-alert
paths (no filter changes in 3 places). Orphans now self-clean via the existing
expiry sweep in monitor.rs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Address second-round review:
- Claude #1 (P2): username_override_from_label now strips the 'ephemeral-'
prefix for ephemeral-webhook-* labels, so created_by stays
webhook-{service}-{rd5} instead of changing to label-ephemeral-webhook-...
(preserves audit/job-list filter compatibility)
- Codex (P2): updated renew_channel doc — labels are no longer copied; rotate
mints fresh ephemeral-webhook-google-{rd5} with 14d expiration
- Claude #3 (optional): test_rotate_webhook_token now asserts the rotated
Google token has an ephemeral-webhook-google-* label and a populated
expiration
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fixup! fix(native-triggers): serialize Google channel renewal across replicas
fixup! fix(native-triggers): serialize Google channel renewal across replicas
Reconsider the previous fixup: stripping the 'ephemeral-' prefix made
created_by no longer match token.label exactly, defeating the linking
purpose. Just allowlist 'ephemeral-webhook-' alongside the other
recognized webhook/email/ws prefixes — created_by becomes
ephemeral-webhook-google-XXXXX, matching token.label exactly. The
'ephemeral-' substring also informs operators that this is a
system-managed auto-expiring token vs a user-managed webhook trigger.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli-tests): stabilize flow lock-gen race + Windows path
Three CLI test failures on the latest main, all flaky on CI:
1. `Mixed Case Paths: pull and push flow with capitalized folder` and
`Integration: Mixed scripts and flows with nonDottedPaths are
idempotent`: flow create/update queues an async FlowDependencies job
that fills inline-script lockfiles and rewrites flow.value. The tests
pulled/pushed before the worker finished, so dry-run idempotency saw
phantom `*.inline_script.lock` adds and `flow.yaml` edits. Added a
`waitForFlowDependencyJob` helper that polls `/flows/get` for the
latest `dependency_job` and `/jobs_u/completed/get` until it lands,
and called it after each API/CLI flow write in both tests.
2. `HEADERS env var is forwarded on every CLI fetch` (Windows-only,
added in #9075): the new test built the CLI entrypoint via
`new URL("..", import.meta.url).pathname`, which yields `/C:/...` on
Windows and `Bun.spawn` rejected before reaching the proxy, leaving
`rejectedRequests.length` at 0. Switched to
`fileURLToPath` + `node:path.join` to match `cargo_backend.ts`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli-tests): use /flows/deployment_status to actually wait for dep job
CI reviewers (Claude, Codex) flagged the prior `waitForFlowDependencyJob`
as a no-op: it read `flow.dependency_job` from `/api/w/{ws}/flows/get`,
but `Flow` / `FlowWithStarred` (backend/windmill-types/src/flows.rs:20-60)
do not include that field. The helper exited on the first iteration
without polling.
Switch to `/api/w/{ws}/flows/deployment_status/p/{path}`, which returns
`{ lock_error_logs, job_id }`. `job_id` is the FlowDependencies UUID
written into `deployment_metadata` in the same tx as the dep-job push
(backend/windmill-api-flows/src/flows.rs:660-672 and :1275-1292), so by
the time the create/update API call returns, the response carries the
latest dep-job UUID. Then poll `/jobs_u/completed/get/{job_id}` as
before. Local runtime for `mixed_case_paths.test.ts` jumps from ~9s to
~32s, confirming the helper now actually waits instead of returning
immediately. The 404 short-circuit in `sync_pull_push.test.ts` still
works — `get_deployment_status` returns 404 when the flow is absent.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Several `fetch()` callers in the CLI bypassed `OpenAPI.HEADERS` and skipped
the `HEADERS` env var, causing requests to fail behind auth gateways like
Cloudflare Access (same shape as #6421):
- `pushScript()` `/scripts/create` and `/scripts/create_snapshot` — regressed
in #8936 when the call switched from `wmill.createScript()` (SDK) to a raw
`fetch` for the `skip_if_noop` query param.
- Script preview `/jobs/run/preview_bundle`.
- App dev `/jobs_u/getupdate_sse` SSE stream.
- `wmill docs` `/api/inkeep`.
All four now spread `getHeaders()` and call `detectAuthGatewayChallenge()`
so a Cloudflare/SSO challenge surfaces a clear error instead of an opaque
JSON parse failure.
Adds `test/headers_env_var.test.ts`: spins up an auth-gateway proxy that
403s requests missing `CF-Access-Client-Id` / `CF-Access-Client-Secret` and
otherwise reverse-proxies to the test backend, then runs `wmill sync push`
of a fresh script through the proxy. Negative case (no `HEADERS` env)
verifies the proxy actually gates; positive case asserts every request
including `/scripts/create` reaches the backend with the headers attached.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ruben Fiszel
Arnaud
hugocasa