mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-04 16:03:06 +00:00
a015ae00a5de8a8efc6d689b153e019d6bdbe3c1
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
c2deea13b7 |
fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr) (#10124)
* fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr)
Privilege escalation: an app/flow/schedule/trigger execution policy's `on_behalf_of`
(which a `wm_deployers` member can set) could point at a superadmin email. The
resulting job `WM_TOKEN` then passed the email-based superadmin checks, granting
instance superadmin. `forbid_superadmin_job_token` only guarded ~15 of ~75 routes.
Fix at the token layer: a WM_TOKEN must never satisfy a superadmin gate,
regardless of whose email it runs as (sentinel OR a real superadmin).
- `ApiAuthed` gains a `job_id` field, stamped once in `AuthCache::get_opt_job_authed`
from the resolved token's job_id (correct even on cache hits).
- `require_super_admin(db, email)` -> `require_super_admin(db, &ApiAuthed)`, rejects
`authed.job_id.is_some()`. `require_super_admin_email` kept for the few internal
callers without an ApiAuthed.
- `is_super_admin_authed(db, &ApiAuthed)` for the boolean `is_super_admin_email`
authorization branches on request handlers (workspace deletion, fork drops,
dev-workspace attach/archive, object-storage SSRF exemption, custom dbname, EE GHES
+ connected repositories, ...). Migrate ~75 sites (OSS + EE).
- CUSTOM_INSTANCE_DB reads the *authenticated* job_id, not the caller-supplied
`?job_id` query param. Worker-tag check takes a precomputed job-aware `is_super_admin`
on the request path.
Execution-time on-behalf checks (scheduled/flow worker-tag, Cloud enqueue quota,
is_devops_email) are hardened in a follow-up — see
docs/followup-onbehalf-execution-privilege-hardening.md.
Regression tests: a superadmin-email WM_TOKEN is rejected on `require_super_admin`
routes, on `DELETE /workspaces/delete/{w}` (403, workspace preserved), and on the
CUSTOM_INSTANCE_DB lookup with no `?job_id` (401); real superadmin tokens still succeed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: cap devops role at workspace admin and reject reserved on_behalf_of identities
Extends the job-token cap with three pieces:
- `require_devops_role` takes `&ApiAuthed` and rejects job tokens.
`is_devops_email` is true for superadmin emails, so every worker-management,
instance-config and service-log route was reachable by the same superadmin
`WM_TOKEN` that `require_super_admin` already rejects.
- A `job_id` claim that does not parse as a uuid rejects the token rather than
resolving to `None`, which would clear the job provenance and uncap it. Applies
to the internal JWT and the external `jwt_ext_` path.
- Defense in depth at store time: `validate_on_behalf_of` refuses the reserved
internal sentinels as an `on_behalf_of` on apps/flows/scripts/schedules/triggers,
and app execution refuses a policy carrying one — covering already-persisted and
forked-app rows that predate the cap. Deploying on behalf of a real user,
including a real superadmin, stays allowed; the cap handles that at execution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(mcp): preserve job-token provenance when minting the proxy JWT
The MCP endpoint-tool proxy re-mints a JWT from the caller's ApiAuthed to
forward the proxied request, but passed job_id: None. A job's WM_TOKEN is
capped at workspace admin (GHSA-hfh4-cx4h-3fcr); dropping the job_id here
re-minted an uncapped token that satisfies require_super_admin /
require_devops_role on the proxied route (e.g. listWorkers exposing worker
IPs, job/workspace IDs, and sensitive tags).
Carry api_authed.job_id into create_jwt_token. Adds an in-module regression
that decodes the forwarded JWT and asserts the job_id is preserved for a job
caller and absent for a non-job caller.
Reported by Codex CI review (P1) on #10124.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: cap the admin-or-devops gate at workspace admin for job tokens
require_admin_or_devops (the EE critical-alerts endpoints) grants when the
caller is a workspace admin OR an instance devops. is_devops_email is true
for superadmins, so a WM_TOKEN running on-behalf of a superadmin who is not a
member of the target workspace could clear the devops branch and read/ack that
workspace's critical alerts (GHSA-hfh4-cx4h-3fcr). This gate takes a bare
email, not an ApiAuthed, so the token-layer cap could not see it.
Thread the caller's job-token provenance and reject the devops branch for job
tokens, matching require_devops_role. The workspace-admin branch stays allowed
— that is the cap ceiling. Adds an enterprise-gated regression proving the
bypass is closed and a real superadmin token still clears the gate.
Found while auditing the PR for bare-email gates the choke-point cap misses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: cap instance-global is_admin gates at workspace admin for job tokens
Three instance-global routes gate on the caller's own `is_admin` claim, which
`ApiAuthed.is_admin` carries into a WM_TOKEN (it is a workspace-admin claim,
true for superadmins too). A job token is capped at workspace admin
(GHSA-hfh4-cx4h-3fcr), so its is_admin claim must not authorize instance
actions on a route with no workspace binding:
- `unarchive_workspace` — unarchive an arbitrary workspace by id
- `prune_concurrency_group` — delete a global concurrency group
- `list_worker_groups` — return unobfuscated `env_vars_static` (may hold secrets)
Add job-token-aware `is_instance_admin` / `require_instance_admin` helpers (the
same shape as `require_super_admin` / `require_devops_role`) and use them at
these three sites. Workspace-scoped `require_admin(authed.is_admin, ...)` gates
are intentionally left unchanged — a workspace-admin job token is within the
cap there. Regression added covering all three; verified it lets a WM_TOKEN
unarchive/leak without the fix and is blocked with it.
Reported by Codex CI review (P1) on #10124.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(mcp): drop orphaned path_field_renames from EndpointTool test helper
The merge with main adopted main's mcp path-substitution refactor (#10162),
which removed the `path_field_renames` field from `EndpointTool` and its
consumer (`substitute_path_params` no longer takes per-field path renames).
main's `runner.rs` `ep` test helper still constructed the struct with
`path_field_renames: None`, so the workspace test build (cargo test --all,
which compiles windmill-mcp's own #[cfg(test)] module under the `server`
feature) failed with E0560. A plain `cargo check` does not compile that test
module, so it only surfaced in CI's cargo_test.
Remove the orphaned field to match the struct.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test: describe the sentinel-rejection policy the forged-identity test asserts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: complete ApiAuthed initializers in feature-gated tests after merge
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: stop job tokens minting credentials that shed their provenance
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: cap the MCP OAuth approval mint at the same elevated-job-token gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: cap the self-service password reset at the elevated-job-token gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: cap app embed/SDK mints and scope widening at the elevated-job-token gate
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: keep job tokens from destroying the account they run on behalf of
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: deny job tokens a foreign-workspace admin claim and workspace ejection
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs: keep the follow-up inventory in the PR instead of the repo
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: make the session workspace status gate job-token aware
session_workspace_status derived its superadmin branch from a bare email
check, so a job token carrying a superadmin identity resolved the existence
of workspaces it has no relationship with rather than seeing them as
deleted. Switch to is_super_admin_authed, matching every other instance
gate reached from a request ApiAuthed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* revert: leave the global concurrency-group listing on the plain admin gate
The listing exposes concurrency keys across workspaces, which is metadata
rather than a capability, and it 401s rather than degrading. Keep the guard
on the prune route next to it, which is the destructive one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: keep the instance-admin gate on the global concurrency listing
The listing spans every workspace's concurrency keys, and the gate rejects
only job tokens: the !is_admin branch is the pre-existing check, so
workspaced tokens and interactive admins are unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to d30af67d38954f9012f7bad08da23e347344b4c6
This commit updates the EE repository reference after PR #664 was merged in windmill-ee-private.
Previous ee-repo-ref: 7870573dbc3360f99bada143f094c67dce0d9e9c
New ee-repo-ref: d30af67d38954f9012f7bad08da23e347344b4c6
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben Fiszel
|
0dbd9c1231 |
perf: eliminate dual-connection DB pool contention across worker, queue, and api (#9798)
* perf: eliminate dual-connection DB pool contention across worker, queue, and api Reuse the held transaction (or move pool reads before begin()) instead of checking out a second pool connection while a tx is open, extending the fix from #9789/#7861. Targets the per-worker pool (max 5) hot paths plus several server-pool API handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: pass owned pool to get_email_from_permissioned_as in http trigger handler The generified signature takes impl PgExecutor; the http trigger handler passed &db where db is already &DB, yielding &&Pool which does not impl PgExecutor (only surfaced under the full feature set in CI). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep RLS-exposed reads on the non-RLS pool and isolate flow-eval reads in a savepoint Addresses review of the dual-connection sweep: - worker_flow: wrap the stop_after_all_iters_if reads in a SAVEPOINT. The caller swallows the error and keeps using tx, so a DB read failure must not leave the outer transaction aborted (it would fail the later commit). Matches the previous pool-read semantics. - Revert reads that were moved onto an RLS (user_db) transaction back to the non-RLS pool, since RLS row-visibility/role context can change results: push_scheduled_job (email/tag/settings lookups; reachable with a user_db tx from api-schedule/api-flows), push_inner native-retry dedicated_worker routing (RLS isolation variants), resources.rs app-namespace folder auto-create (non-admins must not be blocked), and the script archive/delete UPDATEs. Non-RLS db.begin() reuse and move-before-begin are kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: failpoint proving the stop_after_all_iters_if savepoint isolates an aborted read Adds a worker-crate failpoints feature and a data-driven hook: when the stop_after_all_iters_if expr is the magic sentinel, the in-evaluation read runs SELECT 1/0 to abort its (savepoint) transaction. The test asserts the flow still completes (iteration marked failed) — which only holds if the savepoint keeps the outer status-update transaction committable. Without the savepoint the abort would poison the outer tx and the job would never complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
1174d7d77f |
refactor: replace SELECT * with explicit column lists (#9010)
* refactor: replace SELECT * with explicit column lists Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * chore: update sqlx offline query cache * chore: update sqlx offline query cache * chore: update sqlx offline query cache with EE support * chore: update sqlx offline query cache, no deletions * chore: update sqlx offline query cache after rebase * fix: correct column names in explicit script query lists - concurrency_limit → concurrent_limit (matches DB column name) - runnable_settings → runnable_settings_handle (matches DB column name) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add missing delete_after_secs column to script queries Also add integration test covering all explicit-column export queries. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add workspace export integration test covering all explicit-column queries Covers tarball_workspace (folder, script, resource, resource_type, variable, schedule, usr, group_) and the mcp_oauth_client SELECT query from windmill-mcp. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add tarball export integration test covering all explicit-column queries Single test creates one of each entity type and exercises every runtime-checked explicit-column query in tarball_workspace. Uses archive_type=tar to avoid zip feature-gate in CI. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: fix stale verification step and CI contradiction in update-sqlx skill - Regenerate current_files.txt after EE cache restoration so step 4 reports accurate diff - Scope "Never use SQLX_OFFLINE=true" to local prepare (CI legitimately uses it) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * docs: remove Co-Authored-By from commit skill template * refactor: extract SCRIPT_COLUMNS const to single source of truth Replaces 5 duplicated 44-column lists with a shared const in windmill-types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
73fab0c264 |
fix(autoscaling): native worker stuck at max + wrong TimeAgo (#8930)
* fix(autoscaling): return applied_at with UTC timezone in events API autoscaling_event.applied_at is a naive TIMESTAMP column. Serializing as NaiveDateTime produces an ISO string with no timezone, which the browser parses as local time — for users west of UTC this lands in the future and TimeAgo's Math.max(0, …) clamps every event to "0s ago". Cast the column with AT TIME ZONE 'UTC' and type the field as DateTime<Utc> so the response includes a Z suffix. Also pulls in the EE count-distinct fix for native worker autoscaling. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 4128203739a973330599dacfb054203cf9832f3a This commit updates the EE repository reference after PR #548 was merged in windmill-ee-private. Previous ee-repo-ref: 32636bc3e3996101554d5ef504785346929a593b New ee-repo-ref: 4128203739a973330599dacfb054203cf9832f3a Automated by sync-ee-ref workflow. * chore: bump ee-repo-ref for applied_at UTC insert fix Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
 hugocasa
|
bffa61e33f |
fix: dedicated worker dispatch, cross-workspace deps, UI improvements (#8689)
* feat: restore bun as default runtime for dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add context comment for bun dedicated worker nodejs migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for flows + add E2E tests - Add workspace_id prefix to dedicated worker map lookup keys - Update ee-repo-ref for dedicated worker path handling fix - Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers - Add 6 E2E tests for dedicated workers: - test_dedicated_flow_rawscript (regression for "Script not found" bug) - test_dedicated_flow_workspace_script - test_dedicated_flow_multiple_steps - test_dedicated_standalone_script - test_dedicated_runner_group - test_dedicated_flow_runners - Add dedicated_flows.sql fixture with scripts, flows, and worker config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always run dependency job for dedicated worker scripts When a script with dedicated_worker=true is deployed with a pre-computed lock (e.g. via wmill sync push), no dependency job was created, so the dedicated worker never detected the update and kept running the old version. Now dedicated worker scripts always generate a dependency job regardless of whether a lock is provided. The dependency job runs on the dedicated worker and triggers a restart so it picks up the new script version. Fixes #8638 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races Dedicated worker tests need non-default worker tags in the global WORKER_CONFIG. When run in parallel (CI uses --test-threads=10), multiple tests clobber each other's config. Use #[serial] to ensure dedicated worker tests run sequentially. Also load worker config from DB via load_worker_config() instead of manually setting WORKER_CONFIG fields, ensuring consistency with the monitor's reload path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests Fix script_path shadowing in bun_executor nodejs branch where the wrapper file path was passed to handle_dedicated_process instead of the logical path, causing "Script not found" for all //nodejs dedicated workers. Add E2E tests for dedicated flows in all supported languages: - test_dedicated_flow_deno - test_dedicated_flow_python - test_dedicated_flow_bunnative (V8 PrewarmedIsolate path) - test_dedicated_flow_bun_nodejs (//nodejs annotation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify dedicated worker dispatch + add serialization and E2E tests - Unified lookup: always use {workspace}:{runnable_path} for dedicated worker dispatch, replacing the flow_step_id iteration approach - Added serialization_semaphore parameter to executor start_worker fns - Added E2E tests: cross-workspace isolation, conflicting flow step IDs, preprocessor on dedicated worker - Added workspace field to RunJob for cross-workspace test support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: cross-workspace workspace dependencies on workers page Add two new instance-level endpoints to the configs router: - GET /configs/list_all_workspace_dependencies - GET /configs/list_all_dedicated_with_deps Both require devops role and return data across all workspaces, enabling the workers page to show a consistent view of which workspace dependencies exist regardless of which workspace the user is browsing. Update DedicatedWorkersSelector to use the new cross-workspace endpoints with fallback to per-workspace calls for non-devops users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include dedicated worker lookup simplification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update ee-repo-ref.txt with new reference * sqlx * fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings - Remove serialization_semaphore from executor start_worker signatures - Remove serialization test and fixtures - Fix DedicatedWorkersSelector to preserve tags from other workspaces when toggling in the picker - Track workspace deps per-workspace for conflict detection - Show warning when dep exists in another workspace but not the script's - Group runner groups per-workspace to prevent cross-workspace merging - Add workspace to dep badge link URL Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify exec protocol — execd: for single-script, exec: for runner groups Add execd:/execd_preprocess: commands to bun/deno/python wrappers for single-script dedicated workers (no path needed). Runner groups keep exec:/exec_preprocess: with path for multi-script disambiguation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add unit tests for execd:/exec: wrapper protocol Verify generate_multi_script_wrapper produces both execd: (single-script) and exec: (runner group) protocol handlers, including preprocessor variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update commit reference in ee-repo-ref.txt * fix: remove beta badge from squash loop, keep tooltip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update protocol tests to use execd: for single-script wrappers Deno and bun single-script protocol tests now send execd:{args} instead of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script (runner group) tests continue to use exec:{path}:{args}. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused TEST_SCRIPT_PATH in deno protocol tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — down migration, push_as workspace, UI improvements - Use regexp_replace in down migration for positional accuracy - Fix push_as() to use self.workspace_id instead of hardcoded value - Remove per-workspace API fallbacks, use cross-workspace endpoints only - Skip devops-only API calls when user is not devops (disabled prop) - Fix duplicate key error for cross-workspace runner groups - Add workspace to RunnerGroup for unique keying - Reuse tagRow snippet for standalone items with expand/collapse - Fix picker alignment: remove empty column for non-expandable items Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive dedicated worker test coverage, fix Python execd_preprocess - Add Python execd_preprocess: handler (was missing for single-script dedicated workers) - Add 10 E2E tests: flow+standalone conflict, mixed lang fallback, unsupported lang flow runners, python runner group, bun/python/deno/bunnative preprocessors, runner group preprocessors, branchone flow - Add 4 Python unit tests for execd:/execd_preprocess: protocol - Update EE ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — migration escaping, deno try/catch, loadRunnables guard - Down migration: use E'...' so \n matches actual newlines - Up migration: anchor regex with ^ to avoid mid-content matches - Deno execd_preprocess: move JSON.parse inside try/catch - DedicatedWorkersSelector: skip devops-only API calls when disabled Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add dedicated worker relative import tests for bun and python Verifies that build_loader's CURRENT_PATH correctly resolves workspace- relative imports when running on a dedicated worker subprocess. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for nested flow structures (branches/loops) - Add extract_flow_root() to strip nesting segments from runnable_path - Dispatch uses flow_root/flow_step_id for nested paths, runnable_path for flat paths — deterministic, O(1) - Fix assert_ran_on_dedicated_worker to BFS all descendants - Fix python mode labels (python vs python3 for runner groups) - Add tests: simple forloop, multi-step forloop, whileloop, branchall, nested branch-in-loop, mixed lang fallback, unsupported lang runners Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: fix ee-repo-ref SHA Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide picker and skip API calls for read-only users, hide empty runner badge - Hide "Add more scripts/flows" section when disabled (read-only) - Skip per-runnable API calls (getScriptByPath, getFlowByPath) for disabled users — just show path info - Hide "0 runners" badge on flows with no eligible steps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 9422b189762ae27edfc346541ae668a4ad728325 This commit updates the EE repository reference after PR #503 was merged in windmill-ee-private. Previous ee-repo-ref: 4c6ba214bfc23fff05d1dc3200ac59e650af3f4f New ee-repo-ref: 9422b189762ae27edfc346541ae668a4ad728325 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
61a867f086 |
Revert "feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)" (#8687)
This reverts commit
|
||
|
hugocasa
|
619ebb65ce |
feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)
* feat: restore bun as default runtime for dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add context comment for bun dedicated worker nodejs migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for flows + add E2E tests - Add workspace_id prefix to dedicated worker map lookup keys - Update ee-repo-ref for dedicated worker path handling fix - Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers - Add 6 E2E tests for dedicated workers: - test_dedicated_flow_rawscript (regression for "Script not found" bug) - test_dedicated_flow_workspace_script - test_dedicated_flow_multiple_steps - test_dedicated_standalone_script - test_dedicated_runner_group - test_dedicated_flow_runners - Add dedicated_flows.sql fixture with scripts, flows, and worker config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always run dependency job for dedicated worker scripts When a script with dedicated_worker=true is deployed with a pre-computed lock (e.g. via wmill sync push), no dependency job was created, so the dedicated worker never detected the update and kept running the old version. Now dedicated worker scripts always generate a dependency job regardless of whether a lock is provided. The dependency job runs on the dedicated worker and triggers a restart so it picks up the new script version. Fixes #8638 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races Dedicated worker tests need non-default worker tags in the global WORKER_CONFIG. When run in parallel (CI uses --test-threads=10), multiple tests clobber each other's config. Use #[serial] to ensure dedicated worker tests run sequentially. Also load worker config from DB via load_worker_config() instead of manually setting WORKER_CONFIG fields, ensuring consistency with the monitor's reload path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests Fix script_path shadowing in bun_executor nodejs branch where the wrapper file path was passed to handle_dedicated_process instead of the logical path, causing "Script not found" for all //nodejs dedicated workers. Add E2E tests for dedicated flows in all supported languages: - test_dedicated_flow_deno - test_dedicated_flow_python - test_dedicated_flow_bunnative (V8 PrewarmedIsolate path) - test_dedicated_flow_bun_nodejs (//nodejs annotation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify dedicated worker dispatch + add serialization and E2E tests - Unified lookup: always use {workspace}:{runnable_path} for dedicated worker dispatch, replacing the flow_step_id iteration approach - Added serialization_semaphore parameter to executor start_worker fns - Added E2E tests: cross-workspace isolation, conflicting flow step IDs, preprocessor on dedicated worker - Added workspace field to RunJob for cross-workspace test support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: cross-workspace workspace dependencies on workers page Add two new instance-level endpoints to the configs router: - GET /configs/list_all_workspace_dependencies - GET /configs/list_all_dedicated_with_deps Both require devops role and return data across all workspaces, enabling the workers page to show a consistent view of which workspace dependencies exist regardless of which workspace the user is browsing. Update DedicatedWorkersSelector to use the new cross-workspace endpoints with fallback to per-workspace calls for non-devops users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include dedicated worker lookup simplification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update ee-repo-ref.txt with new reference * sqlx * fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings - Remove serialization_semaphore from executor start_worker signatures - Remove serialization test and fixtures - Fix DedicatedWorkersSelector to preserve tags from other workspaces when toggling in the picker - Track workspace deps per-workspace for conflict detection - Show warning when dep exists in another workspace but not the script's - Group runner groups per-workspace to prevent cross-workspace merging - Add workspace to dep badge link URL Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify exec protocol — execd: for single-script, exec: for runner groups Add execd:/execd_preprocess: commands to bun/deno/python wrappers for single-script dedicated workers (no path needed). Runner groups keep exec:/exec_preprocess: with path for multi-script disambiguation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add unit tests for execd:/exec: wrapper protocol Verify generate_multi_script_wrapper produces both execd: (single-script) and exec: (runner group) protocol handlers, including preprocessor variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update commit reference in ee-repo-ref.txt * fix: remove beta badge from squash loop, keep tooltip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update protocol tests to use execd: for single-script wrappers Deno and bun single-script protocol tests now send execd:{args} instead of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script (runner group) tests continue to use exec:{path}:{args}. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused TEST_SCRIPT_PATH in deno protocol tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
0389d9601c |
chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add route reachability tests for ~80 previously untested endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update new trash routes to axum 0.8 path syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: upgrade route tests to assert 2xx responses with proper data setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: restore npm_proxy and ai_routes tests using local echo servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate workspace fork test behind enterprise feature flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings from axum 0.8 upgrade - Use cookie value_trimmed() instead of value() for cookie 0.18 compat - Update comments still referencing old :workspace_id syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1 This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private. Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1 Automated by sync-ee-ref workflow. * test: add test for new get_imports endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused import in raw_apps test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
d8b4132b9a |
fix: native mode now properly sets DB pool size and sleep queue (#8332)
Native mode (from DB config) was not accounted for when sizing the connection pool or setting SLEEP_QUEUE, because both read NUM_WORKERS from env which is never set when native mode is configured via the worker group config in the database. - Resolve native mode early (before connect_db) by querying the config table with the initial DB connection - Pass num_workers directly to connect_db instead of re-reading env - Replace SLEEP_QUEUE lazy_static with sleep_queue() function that checks NATIVE_MODE_RESOLVED at runtime (returns 300ms for native) - Set NATIVE_MODE_RESOLVED immediately when is_native_mode_from_env() - Allow native_mode in CE worker group config (was silently stripped) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
 Diego Imbert
|
7d9d16a6a3 |
feat: runScript inline for path and hash (#8019)
* runScript inline for path and hash * Update backend/windmill-api/src/jobs.rs Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * refactor: unify inline script param structs and deduplicate closures - Replace RunInlineScriptByPathFnParams and RunInlineScriptByHashFnParams with a single RunInlineScriptFnParams using InlineScriptTarget enum - Collapse two nearly-identical closures in worker.rs into one - Merge duplicate InlineByPath/InlineByHash into InlineScriptArgs - Extract shared run_inline_script_inner helper in API handler - Add missing check_scopes to run_inline_script_by_hash endpoint - Fix duplicate lines from prior commit in run_inline_script_by_path - Change tag from "inline_preview" to "inline" for deployed scripts Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> * Integration tests * rm * rename feature to run_inline * Run inline integration tests * Fix tests * check path scope * openapi fix * nits * remove register_potential_assets_on_inline_execution * unused variable * refactor * Pass user_db to check script permission --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com> |
||
|
Ruben Fiszel
|
8b8e33e2dc |
feat: IaC hints, YAML editor for worker configs (#7956)
* feat: add CRD configuration hints to instance settings and workers pages Add subtle text notes with doc links to help K8s users discover that instance settings and worker group configs can be managed via the WindmillInstance CRD. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: IaC hints, YAML editor for worker configs, and EE badges - Move IaC hint from form view to YAML mode in Instance Settings - Add YAML editor drawer for worker group configs with diff review - Add EE-only badges to gated fields in worker config drawer - Make init_bash available in CE (frontend + backend gate) - Promote Init/Periodic scripts to top-level sections - Add "YAML editor" button to worker config drawer action bar - Strip cache_clear and empty fields from YAML output - Sort default/native groups first in YAML editor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review feedback on worker YAML editor - Extract serializeWorkerGroupsAsYaml() shared helper (dedup) - Fix name:undefined leak using proper destructuring - Protect default/native groups from accidental deletion - Add loading indicator during YAML save - Add restart warning for periodic script changes - Use consistent on:click event syntax in YAML drawer actions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: use onClick prop instead of on:click for Button components Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
6cf3f5f4a3 |
feat: open-source worker group configuration UI (#7954)
* feat: gate private registry configuration behind enterprise feature Add read_ee_registry<T> helper that returns None and emits a job log in CE builds when a private registry is configured. Applied to all language executors: pip, npm/bun, deno, cargo, go, maven, nuget, powershell, and ruby. Maven settings.xml file write in monitor.rs is also gated. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: open-source worker group configuration UI Allow CE users to create worker group configs and edit worker_tags. EE-only features (dedicated workers, custom env vars, init/periodic scripts, autoscaling, priority tags, etc.) remain gated behind the enterprise license in the UI. Backend now filters CE config updates to only allow worker_tags and cache_clear fields instead of rejecting the entire request. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
e26e437dd1 |
refactor: extract 12 leaf crates from windmill-api (#7899)
* feat(backend): extract 12 leaf crates from windmill-api to improve incremental compilation Extract independent modules from windmill-api (90k LOC monolith) into separate leaf crates to reduce incremental compilation times. Modules extracted: assets, configs, debug, flow-conversations, inputs, npm-proxy, openapi, schedule, settings, workers, agent-workers, and alerting (from windmill-common). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add windmill-api-settings dep to root crate, make ee_oss public Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add agent_workers integration tests 8 tests covering agent worker lifecycle: - Simple script execution (bun) - Script with arguments - Script with logs (verified in job_logs table) - Script failure handling - Complex result (nested objects/arrays) - Agent token creation via API - Token creation + Initial/MainLoop ping cycle - Multiple sequential job execution Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |