Commit Graph

8 Commits

Author SHA1 Message Date
Ruben Fiszel c2deea13b7 fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr) (#10124)
* fix(security): a WM_TOKEN job token can never be a global superadmin (GHSA-hfh4-cx4h-3fcr)

Privilege escalation: an app/flow/schedule/trigger execution policy's `on_behalf_of`
(which a `wm_deployers` member can set) could point at a superadmin email. The
resulting job `WM_TOKEN` then passed the email-based superadmin checks, granting
instance superadmin. `forbid_superadmin_job_token` only guarded ~15 of ~75 routes.

Fix at the token layer: a WM_TOKEN must never satisfy a superadmin gate,
regardless of whose email it runs as (sentinel OR a real superadmin).

- `ApiAuthed` gains a `job_id` field, stamped once in `AuthCache::get_opt_job_authed`
  from the resolved token's job_id (correct even on cache hits).
- `require_super_admin(db, email)` -> `require_super_admin(db, &ApiAuthed)`, rejects
  `authed.job_id.is_some()`. `require_super_admin_email` kept for the few internal
  callers without an ApiAuthed.
- `is_super_admin_authed(db, &ApiAuthed)` for the boolean `is_super_admin_email`
  authorization branches on request handlers (workspace deletion, fork drops,
  dev-workspace attach/archive, object-storage SSRF exemption, custom dbname, EE GHES
  + connected repositories, ...). Migrate ~75 sites (OSS + EE).
- CUSTOM_INSTANCE_DB reads the *authenticated* job_id, not the caller-supplied
  `?job_id` query param. Worker-tag check takes a precomputed job-aware `is_super_admin`
  on the request path.

Execution-time on-behalf checks (scheduled/flow worker-tag, Cloud enqueue quota,
is_devops_email) are hardened in a follow-up — see
docs/followup-onbehalf-execution-privilege-hardening.md.

Regression tests: a superadmin-email WM_TOKEN is rejected on `require_super_admin`
routes, on `DELETE /workspaces/delete/{w}` (403, workspace preserved), and on the
CUSTOM_INSTANCE_DB lookup with no `?job_id` (401); real superadmin tokens still succeed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: cap devops role at workspace admin and reject reserved on_behalf_of identities

Extends the job-token cap with three pieces:

- `require_devops_role` takes `&ApiAuthed` and rejects job tokens.
  `is_devops_email` is true for superadmin emails, so every worker-management,
  instance-config and service-log route was reachable by the same superadmin
  `WM_TOKEN` that `require_super_admin` already rejects.
- A `job_id` claim that does not parse as a uuid rejects the token rather than
  resolving to `None`, which would clear the job provenance and uncap it. Applies
  to the internal JWT and the external `jwt_ext_` path.
- Defense in depth at store time: `validate_on_behalf_of` refuses the reserved
  internal sentinels as an `on_behalf_of` on apps/flows/scripts/schedules/triggers,
  and app execution refuses a policy carrying one — covering already-persisted and
  forked-app rows that predate the cap. Deploying on behalf of a real user,
  including a real superadmin, stays allowed; the cap handles that at execution.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(mcp): preserve job-token provenance when minting the proxy JWT

The MCP endpoint-tool proxy re-mints a JWT from the caller's ApiAuthed to
forward the proxied request, but passed job_id: None. A job's WM_TOKEN is
capped at workspace admin (GHSA-hfh4-cx4h-3fcr); dropping the job_id here
re-minted an uncapped token that satisfies require_super_admin /
require_devops_role on the proxied route (e.g. listWorkers exposing worker
IPs, job/workspace IDs, and sensitive tags).

Carry api_authed.job_id into create_jwt_token. Adds an in-module regression
that decodes the forwarded JWT and asserts the job_id is preserved for a job
caller and absent for a non-job caller.

Reported by Codex CI review (P1) on #10124.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: cap the admin-or-devops gate at workspace admin for job tokens

require_admin_or_devops (the EE critical-alerts endpoints) grants when the
caller is a workspace admin OR an instance devops. is_devops_email is true
for superadmins, so a WM_TOKEN running on-behalf of a superadmin who is not a
member of the target workspace could clear the devops branch and read/ack that
workspace's critical alerts (GHSA-hfh4-cx4h-3fcr). This gate takes a bare
email, not an ApiAuthed, so the token-layer cap could not see it.

Thread the caller's job-token provenance and reject the devops branch for job
tokens, matching require_devops_role. The workspace-admin branch stays allowed
— that is the cap ceiling. Adds an enterprise-gated regression proving the
bypass is closed and a real superadmin token still clears the gate.

Found while auditing the PR for bare-email gates the choke-point cap misses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix: cap instance-global is_admin gates at workspace admin for job tokens

Three instance-global routes gate on the caller's own `is_admin` claim, which
`ApiAuthed.is_admin` carries into a WM_TOKEN (it is a workspace-admin claim,
true for superadmins too). A job token is capped at workspace admin
(GHSA-hfh4-cx4h-3fcr), so its is_admin claim must not authorize instance
actions on a route with no workspace binding:

- `unarchive_workspace` — unarchive an arbitrary workspace by id
- `prune_concurrency_group` — delete a global concurrency group
- `list_worker_groups` — return unobfuscated `env_vars_static` (may hold secrets)

Add job-token-aware `is_instance_admin` / `require_instance_admin` helpers (the
same shape as `require_super_admin` / `require_devops_role`) and use them at
these three sites. Workspace-scoped `require_admin(authed.is_admin, ...)` gates
are intentionally left unchanged — a workspace-admin job token is within the
cap there. Regression added covering all three; verified it lets a WM_TOKEN
unarchive/leak without the fix and is blocked with it.

Reported by Codex CI review (P1) on #10124.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(mcp): drop orphaned path_field_renames from EndpointTool test helper

The merge with main adopted main's mcp path-substitution refactor (#10162),
which removed the `path_field_renames` field from `EndpointTool` and its
consumer (`substitute_path_params` no longer takes per-field path renames).
main's `runner.rs` `ep` test helper still constructed the struct with
`path_field_renames: None`, so the workspace test build (cargo test --all,
which compiles windmill-mcp's own #[cfg(test)] module under the `server`
feature) failed with E0560. A plain `cargo check` does not compile that test
module, so it only surfaced in CI's cargo_test.

Remove the orphaned field to match the struct.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: describe the sentinel-rejection policy the forged-identity test asserts

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: complete ApiAuthed initializers in feature-gated tests after merge

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: stop job tokens minting credentials that shed their provenance

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: cap the MCP OAuth approval mint at the same elevated-job-token gate

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: cap the self-service password reset at the elevated-job-token gate

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: cap app embed/SDK mints and scope widening at the elevated-job-token gate

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: keep job tokens from destroying the account they run on behalf of

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: deny job tokens a foreign-workspace admin claim and workspace ejection

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs: keep the follow-up inventory in the PR instead of the repo

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: make the session workspace status gate job-token aware

session_workspace_status derived its superadmin branch from a bare email
check, so a job token carrying a superadmin identity resolved the existence
of workspaces it has no relationship with rather than seeing them as
deleted. Switch to is_super_admin_authed, matching every other instance
gate reached from a request ApiAuthed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* revert: leave the global concurrency-group listing on the plain admin gate

The listing exposes concurrency keys across workspaces, which is metadata
rather than a capability, and it 401s rather than degrading. Keep the guard
on the prune route next to it, which is the destructive one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix: keep the instance-admin gate on the global concurrency listing

The listing spans every workspace's concurrency keys, and the gate rejects
only job tokens: the !is_admin branch is the pre-existing check, so
workspaced tokens and interactive admins are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to d30af67d38954f9012f7bad08da23e347344b4c6

This commit updates the EE repository reference after PR #664 was merged in windmill-ee-private.

Previous ee-repo-ref: 7870573dbc3360f99bada143f094c67dce0d9e9c

New ee-repo-ref: d30af67d38954f9012f7bad08da23e347344b4c6

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-08-19 22:33:46 +02:00
Ruben Fiszel 2ff5a918d5 feat(worker-tags): add * fork marker to workspace-scoped custom tags (#10177)
* feat(worker-tags): add `*` fork marker to workspace-scoped custom tags

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(worker-tags): pin fork-marker tag admission through real lineage

Also render WorkspaceMatcher in its authored form in the operator-facing
"not in the allowed CUSTOM_TAGS" error, and correct the authorization note
on workspace_with_fork_ancestors.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(worker-tags): gate fork-lineage tag lookup on workspace membership

exists_workers_with_tags is a global route whose workspace is a query
param. Resolving its fork lineage for a caller who is not a member
disclosed whether an arbitrary workspace descends from one named by a
tag(parent*) rule. Require membership before the lineage lookup, and
sharpen the workspace_with_fork_ancestors contract accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(sqlx): add offline cache for fork-marker test INSERT query

The new test_fork_marker_tag_admission_through_lineage integration test
inserts a fork workspace with a bind-parameterized query that had no
cached entry, failing the SQLX_OFFLINE=true CI build.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-17 18:18:54 +02:00
Ruben Fiszel 8bf7fd2c92 feat(queue): stochastic admission + EE availability of workspace fairness algorithm (#9321) 2026-05-26 04:51:53 +00:00
Ruben Fiszel 577a730e90 audit-log workspace-fairness cap transitions (#9306)
* feat(queue): audit-log workspace-fairness cap transitions

When the cloud per-workspace fairness mechanism adds a workspace to the
capped set or releases one, write `workspace_fairness.capped` /
`workspace_fairness.uncapped` audit-log entries to the affected workspace.
The cluster admin can review the full timeline from the `admins` workspace
audit view with `all_workspaces=true`; per-workspace owners see their own
events in their normal audit list.

Only the per-cycle refresh winner emits entries (matching where the heavy
aggregation runs), so a fleet of N workers does not produce N duplicates
per transition. The diff is computed against the value already in
`background_task_state` rather than the winner's in-memory cache, so a
freshly-restarted process winning the claim does not spuriously emit
"newly capped" entries for workspaces that were already capped before it
started.

Audit writes are best-effort: failures are logged via tracing and do not
abort the refresh cycle.

Fixes WIN-1984

* feat(queue): scope fairness audit to admins workspace + queue-metrics pane

- Write `workspace_fairness.capped` / `workspace_fairness.uncapped` to the
  `admins` workspace (was: per-affected-workspace) with the affected
  workspace_id moved to the `resource` field. Cluster admins now get the
  full timeline in one place without `all_workspaces=true`.
- Add `GET /workers/workspace_fairness_events` returning the last 100
  events. Cloud-gated (returns `[]` on non-cloud) and devops-only.
- Add a `WorkspaceFairnessEvents` Section to the Queue Metrics drawer,
  rendered only when `isCloudHosted()` is true. Shows time / event
  badge / workspace / parameters with a refresh button.

Fixes WIN-1984
2026-05-25 14:51:25 +00:00
Ruben Fiszel 5b3913052e refactor: convert read-hot globals to AtomicBool/I64 and ArcSwap (#8815)
* refactor: extract load helpers from reload_setting family

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert atomic primitive globals to AtomicBool/AtomicI64

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert CRITICAL_*/HUB_API_SECRET/INSTANCE_EVENTS_WEBHOOK/JWT_SECRET to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to arcswap-refactor EE branch commit

* refactor: convert BASE_URL/HUB_BASE_URL/MIN_VERSION/LICENSE_KEY*/LICENSE_KEY_ID to ArcSwap

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert worker hot-path globals to ArcSwap (WORKER_CONFIG et al)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: pin ee-repo-ref to combined arcswap-urls+worker EE commit

* chore: update ee-repo-ref to d8be8f88cb8898c8f6b27421989d53528223815d

This commit updates the EE repository reference after PR #532 was merged in windmill-ee-private.

Previous ee-repo-ref: c375aaaac9ec0fc0480993627d0defc8054c31a4

New ee-repo-ref: d8be8f88cb8898c8f6b27421989d53528223815d

Automated by sync-ee-ref workflow.

* fix: cleanup unused imports + fix 2 missed WORKER_CONFIG readers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to ce0f8fbbbde09c4a858312d2d8716d224e99042c

This commit updates the EE repository reference after PR #534 was merged in windmill-ee-private.

Previous ee-repo-ref: 450b601b5aba0ca0b2045f4b5071aa8701b4bfb7

New ee-repo-ref: ce0f8fbbbde09c4a858312d2d8716d224e99042c

Automated by sync-ee-ref workflow.

* fix: secret_backend_integration test — BASE_URL.write().await → .store()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: convert APP_WORKSPACED_ROUTE to AtomicBool for symmetry with HTTP_ROUTE_WORKSPACED_ROUTE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to e587df8 (post-#535 merge)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-14 00:04:10 +00:00
wendrul 920a7f9fa4 fix: devops getting logged out on workers page (#8416)
* fix: devops getting logged out on workers page

* rename local vars
2026-03-17 17:04:18 +00:00
hugocasa 535e108cbf feat: native mode (#7939)
* feat: native mode

* improve

* fix build

* review fixes

* tracing nit
2026-02-16 23:36:41 +00:00
Ruben Fiszel e26e437dd1 refactor: extract 12 leaf crates from windmill-api (#7899)
* feat(backend): extract 12 leaf crates from windmill-api to improve incremental compilation

Extract independent modules from windmill-api (90k LOC monolith) into
separate leaf crates to reduce incremental compilation times. Modules
extracted: assets, configs, debug, flow-conversations, inputs,
npm-proxy, openapi, schedule, settings, workers, agent-workers, and
alerting (from windmill-common).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add windmill-api-settings dep to root crate, make ee_oss public

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add agent_workers integration tests

8 tests covering agent worker lifecycle:
- Simple script execution (bun)
- Script with arguments
- Script with logs (verified in job_logs table)
- Script failure handling
- Complex result (nested objects/arrays)
- Agent token creation via API
- Token creation + Initial/MainLoop ping cycle
- Multiple sequential job execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:31:03 +00:00