starred/windmill
1
0
Fork 0
mirror of https://github.com/windmill-labs/windmill.git synced 2026-08-21 00:02:23 +00:00
Code Issues Packages Projects Releases Wiki Activity
13,639 Commits 732 Branches 1,511 Tags
fix/wac-python-cache-install-race
Commit Graph

55 Commits

Author SHA1 Message Date
hugocasa 2879cbb65a feat(apps): opt-in sandbox isolation for published & raw apps (alpha) (#9420) 
* feat(apps): sandbox published & raw apps with a scoped embed token

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: point ee-repo-ref at embed-token EE commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): allow top-navigation from the sandboxed app iframe

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): share app localStorage across apps via the embedder

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): publisher disable-sandbox option with per-version viewer consent

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(sqlx): cache for disable-sandbox queries

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump ee-repo-ref to disable-sandbox EE commit

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): always sandbox the served raw-app wrapper + viewer fixes

The raw-app wrapper served by get_raw_app_data now always carries
`CSP: sandbox`. The publisher "disable sandbox isolation" opt-out is applied
entirely on the viewer side, which (after per-version consent) builds its own
same-origin blob wrapper — so the backend-served document stays isolated
regardless of how it is reached, never via a relaxed real-origin URL.

Also:
- CORS on the global /apps_u mount so the opaque viewer can load custom-path
  public apps cross-origin.
- Reject runnable-bridge messages unconditionally until the iframe is bound.
- Relay the viewer's in-app hash up to the embedder address bar so deep links
  stay shareable (hash only; embedder keeps its own pathname).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(apps): render public raw apps single-iframe (drop embed token)

Public raw apps now render directly on the real origin with a single
opaque bundle iframe and the page credential, instead of the opaque
viewer + scoped-token indirection. The author bundle stays isolated in
its own opaque iframe (CSP-sandboxed); low-code apps, whose code runs in
the viewer frame, keep the opaque viewer + scoped token.

embed_token now reports raw_app and skips minting a token for raw apps;
the access check still gates visibility.

Also set disable_sandbox: None in the remaining Policy constructors so
the full feature build (all_sqlx_features, enterprise, license) compiles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: bump ee-repo-ref to single-iframe raw-app EE commit

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(apps): grandfather existing apps as legacy-unsandboxed + authed-only consent

Existing apps are stamped by migration as `legacy_unsandboxed` so they keep
running same-origin on upgrade — no breakage and no consent prompt. New apps are
sandboxed by default; re-deploying an app clears the flag.

The publisher `disable_sandbox` consent prompt is now shown only to authenticated
viewers — an anonymous viewer has no session to expose, so the prompt was
meaningless friction.

embed_token reports `legacy_unsandboxed` and `authed`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: bump ee-repo-ref to legacy-unsandboxed EE commit

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(apps): deploy-time migration prompt for legacy-unsandboxed apps

On the first re-deploy of a grandfathered (legacy-unsandboxed) app, the
publisher must explicitly choose: enable sandbox isolation (the flag is
cleared → the app becomes sandboxed) or keep running without isolation
(→ disable_sandbox, with per-version viewer consent). updatePolicy() no
longer carries the legacy flag through a deploy, so the choice is what
sticks.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(apps): disable the sandbox-isolation toggle until the app is deployed

The Deploy-drawer "Disable sandbox isolation" toggle called setPublishState()
— which updates the app by path — even before the app was first deployed, when
the path is empty, throwing an error. Guard it with disabled={!savedApp},
matching the adjacent visibility toggle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(apps): sandbox the in-workspace low-code app viewer in an opaque iframe

Extend the opaque-origin iframe isolation to the logged-in /apps/get viewer.
/apps/get becomes an embedder that keeps the workspace chrome + Edit button and
renders the app inside a cookieless, chrome-less /app_embed viewer route, handed
a scoped embed token minted from the member's session. The app frame runs in an
opaque origin (no allow-same-origin), so it cannot reach the member's session
cookie or window.parent.

- apps.rs: get_app_embed_token_for_path (authed, by-path, scope + RLS gated);
  mint_app_embed_token grants a path-scoped apps:read:{path} so the viewer can
  load its own app definition and no other
- lib.rs: CORS on /apps (bearer-token only, no cookies) for the opaque viewer's
  by-path reads
- new /app_embed/[workspace]/[...path] viewer route (private analog of /public)
- PublicAppFrame: viewerUrl prop to point the opaque iframe at the viewer route

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): unify in-workspace app viewers on the shared sandboxed path

Route every in-workspace app display (low-code and raw) through the same
PublicAppFrame -> PublicApp machinery as the public viewer, so the sandbox /
legacy-unsandboxed / disable-sandbox-consent behavior is identical on every page.

- new InWorkspaceAppViewer renders both app types via PublicAppFrame; /apps/get
  and /apps_raw/get become thin wrappers over it
- /apps_raw/get previously rendered RawAppPreview directly (always isolated, with
  no legacy-grandfathering or consent handling); now consistent with the rest
- retire the legacy same-origin raw viewer /apps/get_raw/[version] and re-point the
  apps-list row to /apps_raw/get; remove the dead /apps_raw/[ws]/[version] route
- load the raw bundle secret in the shared viewer (getAppByPath doesn't return it)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): address PR review feedback (scope + policy hardening, nits)

- require handler-level apps:read on list_apps / list_search_apps so a scoped
  embed token cannot read app definitions through the list endpoints. The route
  layer treats apps:run as satisfying read; the handler check (which does not)
  closes the gap.
- treat legacy_unsandboxed as backend-owned: strip any client-provided value in
  create/update so it can only be set by the grandfather migration, not the API.
- document mint_app_embed_token's caller-verifies-access contract.
- use Button's declared onClick prop for the consent action (was onclick, which
  fell into the rest-spread and bypassed the component's click handling).
- test: lock that the embed scopes cannot satisfy domain-level apps:read.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(apps): document embed-token endpoints in openapi + fix doc nit

Second-round review nits:
- add the three app embed-token endpoints (apps/embed_token/p/{path},
  apps_u/embed_token/{secret}, and the EE apps_u/embed_token_by_custom_path) plus
  the EmbedTokenResponse schema to openapi.yaml; note .html on get_data
- mint_app_embed_token doc: "Both" -> "All" (it lists three call sites)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): bound embed-token scopes to the caller's own

The embed-token mint now enforces ensure_scopes_within_caller, so the
minted scope set is always within the calling credential's own scopes
(a no-op for regular unscoped sessions). Adds a unit test locking the
boundary and documents the contract on mint_app_embed_token.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): raw-app ctx in external embeds + page credential in direct render

- RawAppPreview: engage the storage relay only in opaque frames (probe Web
  Storage instead of just window.parent), so a public raw app embedded in an
  external iframe hydrates ctx/storage directly; add a relay-timeout fallback
  so an unresponsive parent can never stall the ctx handshake.
- PublicAppFrame: in direct render, expose the page's own bearer credential
  through the AuthToken context (JWT public URLs), matching the previous
  route behavior; opaque-viewer mode keeps the embed token.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): sandbox isolation UI polish + COI embed support for raw apps

- Deploy drawer: move the sandbox toggle out of "Public URL" into its own
  "Sandbox isolation" section (the setting applies to every viewing surface,
  not just the public URL), with positive phrasing, visible helper text, and
  state-aware alerts (warning when disabled, info for pre-isolation apps).
  Toggling it now toasts its own message instead of the login-mode one.
- Extract the deploy-time migration prompt into a shared
  LegacySandboxMigrationModal built on the common Modal component, and wire
  it into the raw app editor header too (it previously had no prompt, so
  re-deploying a pre-isolation raw app silently changed behavior).
  updateRawAppPolicy now also drops the backend-owned legacy flag, matching
  the low-code updatePolicy.
- Viewer consent prompt: use the common ConfirmationModal and show the app
  path (new appPath prop) instead of the route pathname, falling back to
  "this app" when the path isn't known yet.
- COI embeds: propagate the wm_coep opt-in to the raw-app wrapper document
  and have the backend assert COEP require-corp on it when the flag is
  present — required for the bundle iframe to load when the public app page
  is embedded inside a cross-origin-isolated page. Previously this only
  worked in dev because the Vite proxy injects the header; the production
  response lacked it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): app navigation parity across sandboxed and direct viewers

- Navbar component: same-app items relay query + hash to the embedder page
  (which mirrors them onto the root URL, keeping its own pathname and
  transport params), app items navigate the top page through a validated
  wm_embed_navigate relay instead of the cookieless viewer iframe, and
  external items keep opening a new tab. Selected-item detection now
  recognizes the /app_embed viewer route and ignores transport params.
- Frontend-script `goto` and button `onSuccess: gotoUrl`: same-window
  navigation goes through a shared appNavigateSameWindow helper that relays
  to the embedder inside the opaque viewer (same-origin paths SPA-navigate,
  http(s) URLs do a full load, other schemes rejected) and keeps plain
  window.location everywhere else.
- /apps/get and /apps_raw/get: key the viewer by workspace/path so in-route
  navigation fully remounts it — previously the URL changed but the app (and
  in sandbox mode its path-scoped token) did not follow.
- wm_embed/wm_embedder_origin added to the reserved query params so they no
  longer leak into the app's ctx.query.
- Raw apps: drop the sandbox attribute entirely for the unsandboxed
  (grandfathered/consented) blob path, matching the pre-isolation viewer
  exactly — the attribute added no isolation there and sandboxed popups
  (e.g. OAuth flows).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): preserve grandfathered policy across updates + in-workspace viewer parity

Round of compatibility hardening so pre-existing apps behave exactly as
before on every surface:

- `legacy_unsandboxed` is now preserved across app updates unless the payload
  explicitly clears it (`false`, sent by the editor's migration prompt and the
  sandbox toggle). Unrelated update paths — CLI / git-sync redeploys,
  publish-mode toggles, cross-workspace promotion — no longer silently drop
  the grandfathering. Clients still can never SET the flag.
- The embed-token endpoints (secret, path, EE custom-path) read only the
  sandbox-decision policy fields, leniently, and no longer mint a token for
  raw / legacy / disable_sandbox renders: the token is only consumed by the
  sandboxed low-code render, and minting for the others wrote a useless token
  row per view and could fail the render for scope-restricted callers.
- In-workspace viewer parity with the pre-sandbox `/apps/get`: new
  `inWorkspace` mode on PublicApp (no "Powered by Windmill" badge / user
  overlay, no HTML-result approval gate, column flex wrapper, `hideRefreshBar`
  honored again), and the page's query/hash are forwarded into the opaque
  viewer so `ctx.query` / `ctx.hash` reach the app.
- Raw apps: `window.ctx` is always `{ctx, workspace}` again (anonymous viewers
  of pre-existing bundles rely on `ctx.workspace`), and the runnable bridge's
  job-id scoping now applies only to sandboxed renders (`gateJobIds`) — an
  unsandboxed bundle holds the same credential as the bridge, so gating there
  only broke pre-existing apps polling persisted or runnable-returned job ids.
- Document `disable_sandbox` / `legacy_unsandboxed` in the openapi Policy
  schema; add a unit test for the lenient policy read.
- bump ee-repo-ref to the matching EE commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): keep share-link viewer credentials out of the isolated app context

The JWT path segment of authenticated share URLs is an embedder-side
credential, consumed only to mint the scoped embed token. Two transport
channels still copied it into the isolated frame where app-authored code
runs:

- the opaque viewer iframe src defaulted to window.location.href — the
  public and custom-path routes now pass a sanitized viewerUrl (JWT segment
  stripped, query/hash preserved, captured once so the hash relay does not
  reload the iframe);
- document.referrer on the same-origin iframe navigation carried the full
  embedder URL — both app iframes now set referrerpolicy="no-referrer"
  (sandboxed renders only for the raw bundle iframe, keeping exact legacy
  parity; nothing reads the referrer).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(frontend): drop unused import inherited from main merge

`slide` import in AssistantMessage.svelte (from #9539) turns `npm run check`
red on this branch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): redirect the removed raw-app viewer path to the unified viewer

The old same-origin raw-app viewer route (/apps/get_raw/{version}/{path}) was
removed in favor of the sandboxed unified viewer. Re-add a thin client route at
the old path that redirects stale bookmarks to /apps_raw/get/{path}, preserving
query + hash (the pinned version is dropped — the unified viewer shows latest).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): narrow embed-token scopes and base consent on browser session

- Embed token: resource access is metadata-only (list/type/exists) via a
  `resources:run` marker — resource values (get/get_value/get_value_interpolated/
  list_search) are no longer reachable. Job reads are by-id only: an `app_embed`
  sentinel blocks the workspace-wide job enumeration/export routes (jobs/list,
  list_filtered_uuids, queue/list, completed/list, queue/export) while by-id
  result polling keeps working.
- disable_sandbox consent now gates on whether the browser holds any Windmill
  session (cookie-only whoami) rather than workspace-scoped auth, so a viewer
  logged into a different workspace is still prompted before a same-origin render.
- db-explorer: resolve the MySQL database name server-side (the metadata query
  already falls back to DATABASE()) instead of reading the resource value
  client-side; getTablesByResource derives the default db from the schema.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore(apps): trim embed-scope and consent comments

Reduce duplication — state the resource/job route exclusions and the
workspace-session-vs-cookie rationale once at their source and reference them
elsewhere; drop contrast/justification phrasing. No behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): make app sandbox isolation opt-in (alpha)

Replace the disable_sandbox + legacy_unsandboxed policy pair and the
per-version viewer consent with a single positive `sandbox` opt-in flag.
Apps are unsandboxed by default (same-origin, full session — the
pre-isolation behavior), so existing apps are unchanged and no migration
is needed. Publishers opt an app into isolation from the deploy drawer,
flagged alpha.

- Policy.sandbox: Option<bool>; EmbedTokenResponse -> {token, expiration,
  raw_app, sandbox}; mint an embed token only for sandboxed low-code apps.
- Drop the legacy-unsandboxed migration and the deploy-time migration
  prompt; remove the consent modal and the browser-session probe.
- Deploy drawer: a single "Sandbox isolation" toggle (alpha), off by
  default, shared by the low-code and raw editors.
- Bump ee-repo-ref to the companion EE commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): confine embed token to its intended user/folder/job routes

The embed token's broad read scopes spanned whole domains while the
matching routers are CORS-enabled for the opaque app iframe:

- users:read / folders:read were domain-wide, so the token could reach
  users/list, users/list_usage, users/username_to_email/*, folders/list,
  etc. Restrict to an app_embed-sentinel allowlist: only users/whoami and
  folders/listnames; deny the rest of those domains.
- jobs:read allowed jobs/completed/export, missed by the job denylist.
  Add it alongside jobs/queue/export.

Extend the embed-scope allow/deny test matrix to cover all of these.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(apps): align sandbox comments with the opt-in model

The consent prompt, deploy-time migration, and legacy-unsandboxed
grandfathering were removed when sandbox isolation became an opt-in
policy flag; update the comments that still described them so they
match the two-state (default-unsandboxed / opt-in-sandboxed) reality.
Comments only, no behavior change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): confine embed-token job reads to runs the app launched

App component jobs are stamped `created_by = the viewer`, so an embed token
reads its own runs via the launched-by-viewer fast path. The token then also
inherited the viewer's broader job access (share links, folder ACLs, admin
RLS), letting user-authored app JS reuse it to read unrelated jobs by id. Stop
embed tokens at the fast path: only jobs the viewer launched, never those
merely visible to them. Return NotFound so the untrusted app can't probe
existence.

Regression test: an embed token reads its own launched job but is denied the
foreign job (result/logs/getupdate) an admin viewer's normal token can read.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): allowlist embed-token apps/jobs routes + scope run to the app

The embed token's apps:run/jobs:read reached more than a running app needs.
Replace the job denylist with strict per-domain allowlists on the app_embed
sentinel:

- Apps: only the app's own definition (apps/get/p/<path>) and the public
  app-serving endpoints (apps_u/*). Denies workspace app inventory
  (exists, custom_path_exists, list, list_paths*).
- Jobs: only the by-id poll routes the frontend JobLoader uses. Denies job
  counts and the job_signature/resume_urls capability-minting routes (the
  by-id reads remain confined to the app's own runs).

Drop unqualified apps:run from APP_EMBED_SCOPES; mint apps:run:<path> instead
and authorize apps:run:<requested path> first in execute_component, so the
token can only run its own app's components, not another app's.

Extend the embed-scope route matrix and add a path-scoped run unit test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(apps): clarify the sandbox toggle vs the on-behalf-of model

The deploy-drawer sandbox copy leaned on "session" in a way that collided
with the on-behalf-of permissioning right above it. Reword it to say the
toggle governs what the app's browser-side code can reach in the viewer's
browser — distinct from who its runnables execute as — and rename the label
to "Isolate the app from the viewer's browser session".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): path-scope embed-token S3 download to its own app

The apps_u/* allowlist also admitted apps_u/download_s3_file/<path>, whose
handler authorized any authenticated caller — so an embed token minted for app
A could download app B's S3 files via B's on-behalf policy. Add the same
path-scoped guard execute_component uses: download_s3_file_from_app now checks
apps:read:<path> first, confining the token to its own app. Other path-taking
apps_u routes are already covered (writes lack apps:write; embed_token/p
path-checks; public_resource is type-constrained).

Extend the path-scoping unit test to cover apps:read (download) alongside
apps:run (execute).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): path-scope public-app-by-secret read to the embed token's app

The apps_u/* allowlist admitted apps_u/public_app/<secret>, whose handler only
checked the viewer's read access — so an embed token minted for app A could read
app B's definition by secret (confused deputy via the viewer's identity).
get_public_app_by_secret now binds a scoped caller to the resolved app with
check_scopes(apps:read:<path>), confining it to its own app; unscoped sessions
and anonymous access are unchanged.

get_raw_app_data needs no binding (pure secret capability, no caller identity).
Document the full set of app-resolving handlers the path-scoped read covers.

Bump ee-repo-ref for the companion custom-path fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): preserve pre-sandbox behavior for db-explorer, edit link, jwt

Three behavior-parity fixes for non-sandboxed (existing) apps that the
sandbox-isolation refactor changed incidentally:

- DB-explorer MySQL table picker: when the connection can see multiple
  non-system schemas, label the default db's tables unprefixed again. The
  resource-value read was removed globally, so identify the default db from
  the introspection script's `DATABASE() AS default_db_name` (carried on
  SQLSchema.defaultDb) instead of guessing "the single schema key". Equivalent
  to the prior resource.database match; editor-only (table picker).
- In-workspace Edit button: restore `?nodraft=true` on both /apps/get and
  /apps_raw/get, so opening the editor from the viewer loads the deployed
  version, not a draft.
- Custom-path (/a) viewer: restore the "could not authenticate user with jwt
  token" toast when a path JWT fails to resolve a user, instead of silently
  falling through.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): confine embed-token S3 downloads to the app's own keys/outputs

download_s3_file_from_app authorized any authenticated caller for any S3 key
(opt_authed.is_some() bypass). A sandboxed app's embed token carries the
viewer's identity, so app-authored JS could fetch arbitrary S3 keys readable by
the on-behalf identity, beyond the app's own declared keys or outputs.

Route app embed tokens through the same allowlist as anonymous viewers — the
app's declared allowed_s3_keys, or files produced by this app's own component
runs — instead of the authed bypass. The produced-files check is parameterized
by created_by (the embed viewer for a token, else anonymous) so a sandboxed
app's own S3 outputs still render while arbitrary keys are denied.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): let embed tokens cancel their own jobs; gate cancel to launcher

A sandboxed low-code app supersedes an in-flight component run on re-run by
canceling it, but the embed token only had jobs:read, so cancellation silently
failed and prior jobs ran to completion.

- Permit the by-id jobs_u/queue/cancel POST for app_embed tokens at the route
  layer (the only write reachable through the existing by-id allowlist).
- Gate cancel_job_api: an app_embed token may cancel ONLY jobs it launched
  (created_by == viewer). cancel_job_api had no other per-job ownership check,
  so this also confines the token instead of letting it cancel any job by id.
- /app_embed now sets workspaceStore so cancellation targets the right
  workspace instead of an empty/stale one in the cookieless iframe.

Add a shared has_app_embed_sentinel helper; cover cancel in the route matrix
and the jobs_read_auth integration test (own job cancelable, foreign denied).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): drop get_root_job_id from the embed-token job allowlist

Audit of the embed token's reachable job routes: get_root_job (jobs_u/
get_root_job_id) has no access check in its handler at all — it returns any
job's root-job id by id — and the app runtime never calls it. Remove it from
the by-id allowlist so the embed token can't probe a foreign job's flow lineage;
add a denied-route assertion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(apps): scope sandboxed-app localStorage per app

Sandboxed apps shared one localStorage store (one key on the real origin), so an
app could read or clobber another app's keys — and, with job ids stashed there,
reuse its embed token to read another app's job. Scope the backing store per app.

The embed-token endpoints now return the resolved app_path (EmbedTokenResponse;
not a new disclosure — the viewer already receives the path when it loads the
app). PublicAppFrame (low-code) and RawAppPreview (raw) key their backing store
by it: wm_apps_localstorage:<app_path>. Same app shares one store across its
public and in-workspace surfaces; different apps are isolated. Unsandboxed apps
are unaffected (real same-origin localStorage, as before).

Bump ee-repo-ref for the companion custom-path change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): scope embed access checks to embed tokens + key app storage by workspace

- Apply the path-scoped read/run checks on the public-by-secret read and the
  component run path only when the caller is an app embed token, so other
  caller types keep their prior access.
- Key the sandboxed app's backing client storage by workspace + path instead
  of path alone, and return the resolved workspace from the embed-token
  endpoints so the custom-path viewer can derive it.
- Show a clear message instead of an indefinite loader when the viewer route
  is opened outside its embedder.

Bumps ee-repo-ref to 5b8476b.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(apps): mint embed tokens only from the trusted embedder caller

An app embed token must not reach the embed-token mint endpoints; refresh
minting stays with the embedder session/JWT. Enforced at the scope route
layer and at the mint chokepoint, with a route-matrix regression test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(apps): support S3 upload and frontend-script S3 download in sandboxed apps

Sandboxed apps run with a scoped embed token (no cookie). Let the app's
S3 file-input upload and the frontend-script download({s3}) helper work in
that context: upload is reachable with apps:run and re-checked per-app at the
handler; the script download routes through the app-scoped apps_u endpoint
with the embed token instead of the cookie-authed job_helpers path. Default
(unsandboxed) apps are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: update ee-repo-ref to b0cb761bf9852974e571b2978032d310cc998517

This commit updates the EE repository reference after PR #600 was merged in windmill-ee-private.

Previous ee-repo-ref: e673c714a4618fdb72353a475f49c748e6016642

New ee-repo-ref: b0cb761bf9852974e571b2978032d310cc998517

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2026-06-23 10:05:00 +02:00
Ruben Fiszel 924f9c7e8d fix(backend): strip NUL bytes from draft values on write (#9673) 
draft.value is a json column (not jsonb), so a client could store a U+0000
escape in it. Any later text extraction (`->>` / `to_jsonb`) on such a value
raises 22P05 "unsupported Unicode escape sequence" — one poisoned draft 500'd
the whole GET /drafts/list, silently hiding the home-page "This workspace has N
drafts" banner (and breaking the global drafts page).

Prevent it at the source: sanitize the value in update_draft (the only path that
writes client-supplied draft content) so a NUL never reaches the column.
strip_json_nul does a single backslash-parity-aware byte pass that removes real
NUL escapes (values and keys alike) while leaving a legitimate escaped backslash
intact — O(n) with no serde_json::Value tree to allocate, important because the
slow path is also hit by any value legitimately containing the text after a
backslash (e.g. script source). The clean path is a single substring check.

A SQL migration scrubs rows written before this, gated to genuinely-poisoned
rows (a real NUL makes value::jsonb raise, distinguishing it from a legitimately
escaped backslash). With the data clean, no read-side query needs to change.

Tests: unit tests for the strip helper (escaped-backslash no-op, real+literal
collision, odd-backslash-run parity, nested keys/values) and an integration test
that POSTs a NUL-bearing draft and asserts it is stored and listed NUL-free
(fails without the strip).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-19 13:33:36 +00:00
Ruben Fiszel 066d7a4726 block operators from inline preview job execution (#9572) 
`POST /api/w/{workspace}/jobs/run_inline/preview` ran request-supplied
code inline (in-process, e.g. DuckDB) but was missing the operator
authorization guard that its sibling `/jobs/run/preview` enforces. An
authenticated operator — the most restricted role, which must not run
preview jobs — could execute arbitrary code in a single request
(file read/write, and OS command execution via the DuckDB `shellfs`
extension when worker egress is available).

This is the incomplete-fix residual of CVE-2026-22683 / GHSA-9q9g-rp9x-244h,
whose v1.615.0 patch covered the entity-CRUD endpoints but left this
direct inline-exec sink uncovered.

Add the same `is_operator` guard from `run_preview_script`. Audited the
rest of the preview/inline arbitrary-code endpoints (run_preview_script,
run_bundle_preview_script, run_preview_flow_job, the wait_result
wrappers, run_dynamic_select inline variant, dependency jobs) — all
already carry the guard. The `run_inline_script_by_path`/`by_hash`
endpoints run deployed scripts (operator-allowed, scope-checked) and
correctly remain ungated.

Fixes WIN-2043 (GHSA-pp5h-96x3-3wqq).

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-15 00:07:46 +02:00
Ruben Fiszel 317a8629d1 fix(ai): enforce resource authz when loading MCP tools in agent worker (#9571) 
* fix(ai): enforce resource authz when loading MCP tools in agent worker

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test: describe MCP authz test invariants instead of drafting history

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:50:18 +02:00
Ruben Fiszel d0aaba0f16 require pinned sha for inline raw_code in viewer app run mode (#9570) 
* fix: require pinned sha for inline raw_code in viewer app run mode

* fix: gate rd_string import behind parquet feature to fix oss build

* fix: also require pin for raw_code with app_script id in viewer run mode
 2026-06-14 23:48:03 +02:00
Ruben Fiszel 8053266f88 fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url (#9428) 
* fix(mcp): resolve MCP resource token via caller RLS + SSRF-guard url

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(mcp): clone user_db for oauth2 refresh and drop advisory ids from comments

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(mcp): disable redirects on MCP client to prevent SSRF bypass

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-03 09:00:43 +00:00
Ruben Fiszel 89a7a37776 fix(backend): authorize single-job read endpoints by job/flow visibility (#9416) 
* fix(backend): authorize single-job read endpoints by job/flow visibility

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(jobs): share read links + cached access checks for run visibility

- Cache the job read-access RLS probe (size-bounded LRU keyed by the caller's
  authz-relevant identity + job id; no TTL since job-side inputs are immutable).
- Inherit visibility along the full parent_job chain so any flow you can see lets
  you read its (deeply nested) steps.
- Share read links: GET /jobs/job_view_token/{id} mints a stateless
  HMAC(workspace_key, job_id) token (only if the caller can read the job); the
  token grants an authenticated member read of that job and its flow subtree via a
  ?view_token query param or X-View-Token header. Run page gains a Share button and
  honors a ?view_token link.
- Denied-but-existing reads now return 403 with guidance to request a share link
  (vs 404 for non-existent), and the run page renders that case with instructions.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): address PR review — scope-tag check on mint, constant-time view-token verify

- P1 (Codex): get_job_view_token now enforces the caller's if_jobs:filter_tags
  scope before minting, so a tag-scoped token can't mint a transferable link for a
  job outside its tags. Adds a scoped-token regression test (allowed + denied).
- Constant-time view-token verification (HmacSha256::verify_slice) instead of
  comparing hex strings (Claude/Pi nit).
- get_completed_job_result: an authed reader passing an invalid suspended-secret
  triple now falls through to the normal visibility gate instead of erroring out
  (Claude nit); unauthenticated callers still rejected.
- Length-prefix the read-access cache key fields so no input values can collide.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(api): add job_view_token to openapi spec; use generated client in run page

Addresses Codex review nit: the new GET /jobs/job_view_token/{id} endpoint was
missing from openapi.yaml (the source the frontend client is generated from). Adds
the path + operationId getJobViewToken, and switches the run page's Share button
from a raw fetch to JobService.getJobViewToken.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): carry view_token on share-link downloads

Addresses Codex review: download actions bypass the request interceptor that adds
X-View-Token (downloadViaClient uses raw fetch; cookie-mode downloads use plain
hrefs), so a share-link viewer got 403 downloading logs/results/args. Append the
view_token query param to the job download paths (result/logs/args/flow-all-logs)
via a new appendViewToken() helper, covering both client-fetch and href modes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): enforce tag scope in require_job_read_access (view-token use side)

Addresses Codex P1: the view_token use-side bypassed if_jobs:filter_tags on
handlers that don't tag-filter their data query (result_by_id,
get_flow_job_debug_info, get_otel_traces) — a tag-scoped token could use someone
else's valid share token to read out-of-scope job data. Move the tag-scope check
into require_job_read_access (runs before any created_by/view_token/RLS grant), so
it applies uniformly to every gated handler; removes the now-redundant explicit
check in get_job_view_token. Adds a use-side regression test (scoped token + valid
out-of-scope view_token denied on otel/result_by_id; in-scope still allowed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(frontend): include workspace in share read link

Addresses Codex P1: the copied share URL omitted the workspace. The token is
signed with the run's workspace key and the logged layout only switches
$workspaceStore when the URL carries workspace=, so a recipient whose persisted
active workspace differs would open the link against the wrong workspace and the
token would fail validation. Pin workspace= alongside view_token in the link.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(jobs): authorize get_result_maybe get_started branch for queued jobs

Addresses Codex P1: get_completed_job_result_maybe only gated when a completed row
existed; with ?get_started=true a non-reader reached the fallback branch and got
started:true for a running private job. Now fetches created_by and authorizes
(created_by/view_token/RLS, or anonymous for unauth) before disclosing
running-state; a non-existent job still returns started:false (leaks nothing).
Adds a regression test with a queued (no completed row) private job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-02 22:10:16 +00:00
Ruben Fiszel 4b1bea8aed fix: enforce auth guards on app component preview execution (#9235) 
* fix: enforce auth guards on app component preview execution

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: guard previewed runnable path and worker tag in app preview

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: validate app_script id ownership and keep root push isolation

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: scope app preview guards to operator check + referenced runnables

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: require jobs:run scope and tag check on app preview (apps:run escalation)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-19 17:40:55 +00:00
Ruben Fiszel dfeed9c5c2 fix: actionable error when a custom_path is taken by an app in another workspace (#9190) 
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-16 08:29:53 +00:00
Diego Imbert 4427a3d37f feat: add workspace-specific flag for resources and variables (#8836) 
* feat: add workspace-specific flag for resources and variables

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove set_ws_specific endpoint and fix rust-client compilation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fall back to workspace name for ws_specific file naming

When wsNameForFiles is not set (no wmill.yaml workspace config),
ws_specific items would not get workspace-suffixed filenames during
pull. Now falls back to workspace.name/workspaceId.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use workspace ID instead of CLI name for ws_specific file naming

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: pass workspace ID fallback to elementsToMap for ws_specific push

Without this, workspace-specific files (e.g., a.admins.resource.yaml)
were not recognized during push when no wmill.yaml or git branch was
available, causing spurious deletions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* ui nits

* nit

* Fix variable edit when only editing ws_specific

* mark_linked_variables_ws_specific

* Helper label

* Support json format alongside yaml

* Fix file naming push/pull asymetry & ws_specific orphans

* Revert all CLI diffs

* CLI now appends the remote ws_specific list to the local specificItems

* UI for Env switcher

* Refactor Resource/Variable editors to use dumb component

* Refactor side effects

* Editor works with multi workspaces

* Fix can_save

* Fix As JSON

* nit

* UI nits

* list_ws_specific_versions as pl sql function to avoid round trips

* UI Nits

* Per-workspace version read-only check

* fix: reset session context in list_ws_specific_versions to prevent RLS leakage

The function calls set_session_context() in a loop. Although SET LOCAL is
transaction-scoped (so settings revert at autocommit), defending against
the function being invoked inside a longer outer transaction:

- wrap the loop in a sub-block with EXCEPTION WHEN OTHERS that resets
  the session to a deny-default (windmill_user, empty session.* GUCs)
  before re-raising,
- on the happy path, reset to the same deny-default at the end of the
  function.

* feat: audit auto-marked ws_specific variables

When a resource is saved as ws_specific, every variable referenced via
$var: inside its value is auto-INSERTed into ws_specific. Previously
this happened silently. Now:

- mark_linked_variables_ws_specific takes the authed user,
- the INSERT uses RETURNING path so we know exactly which variables
  were freshly flipped (not the ones already ws_specific),
- each newly flipped variable gets a 'variables.set_ws_specific' audit
  entry pointing at the resource that triggered it.

* perf: skip mark_linked_variables_ws_specific when nothing relevant changed

update_resource was calling mark_linked_variables_ws_specific on every
save when the resource was ws_specific, even on a description-only or
label-only edit. Gate the call on `ns.value.is_some() || ns.ws_specific
== Some(true)` so we only re-mark when the $var: refs could actually
have changed or ws_specific was freshly enabled.

* docs: explain asymmetric ws_specific toggle in resource tooltip

Enabling the resource's 'Workspace specific' toggle silently marks
every variable referenced via $var: inside the value as ws_specific,
but disabling it does not un-mark those variables (they may be
referenced by other resources). Surface this in the tooltip so users
know what to expect.

* fix: surface non-404 errors when fetching ws_specific items in CLI sync

mergeWsSpecificFromServer was catching every error from listWsSpecific
and logging it at debug. That's correct for old servers without the
endpoint (404), but a 401/403/network failure would silently produce an
incomplete sync. Now distinguish 404 (debug, expected) from everything
else (warn with status + message) so users notice when the merge fails
for real reasons.

* perf: collapse compare_two_variables presence checks into one round-trip

The early-return path was issuing four sequential EXISTS queries
(ws_specific × {source, fork}, variable × {source, fork}). Combine
them into a single SELECT so the per-variable diff cost drops ~4x.

* sqlx prepare

* docs: clarify has_sql_updates invariant in update_variable

The else branch of the npath resolution is only reachable for non-rename
edits (labels-only, ws_specific-only) because ns.path being Some always
forces has_sql_updates=true at the top of the function. Add a debug_assert
and a comment explaining the invariant so a future change that decouples
ns.path from has_sql_updates trips immediately. Also use `path` directly
instead of unwrap_or_default-ing ns.path, since we know it's None here.

* chore: drop redundant ws_specific type augmentations

ListableResource and ListableVariable from $lib/gen now include
`ws_specific?: boolean` after the openapi.yaml additions in this
branch were regenerated. The intersection types in resources/+page
and variables/+page were duplicating the field — drop them.

* Put WsSpecificVersions toggle in top drawer bar

* nit size

* feat: detect local-only ws_specific items on sync push

When wmill.yaml lists a resource/variable in specificItems but the
remote isn't yet marked ws_specific for that item, sync push silently
dropped the flag because:

1. file-content diff alone never noticed (ws_specific is metadata, not
   YAML body) — push{Resource,Variable} were never called for those
   items;
2. even when called, isSuperset(local, remote) returned true and the
   early-return skipped the API call.

Now:

- mergeWsSpecificFromServer returns the raw server list alongside the
  merged config so push can compare 'in local' vs 'in server';
- a new computeWsSpecificFlagOnlyPushes helper walks the local file map,
  finds ws_specific-flagged paths absent from the server list, and the
  push function injects them as synthetic 'edited' changes (same before
  and after content) so the standard display + apply pipeline picks
  them up;
- push{Resource,Variable} no longer early-return when content matches
  but the ws_specific flag differs.

Pull is unaffected — only the push-side caller of mergeWsSpecific takes
the new (merged, serverItems) tuple.

* getDeployTo for selected ws

* refactor: ws_specific kind handling, support .json files

The ws_specific helpers had two warts:

1. computeWsSpecificFlagOnlyPushes hardcoded `.resource.{yaml,json}` /
   `.variable.{yaml,json}` magic strings, even though the existing
   getTypeStrFromPath / removeType helpers already do that work and
   already cover both extensions.
2. isSpecificItem / isItemTypeConfigured only matched `.yaml` paths,
   so users with opts.json local files got no specificItems coverage
   at all — patterns from wmill.yaml (and from mergeWsSpecificFromServer)
   are expressed with `.yaml`, and a `.json` file never matched.

Changes:

- Replace WS_SPECIFIC_KIND_MAP (a closed enum of resource+variable)
  with configKeyForItemKind, a generic kind→SpecificItemsConfig key
  mapping. Triggers fold into 'triggers' via the `_trigger` suffix,
  so adding a kind to the backend's list_ws_specific_versions doesn't
  require a CLI change.
- mergeWsSpecificFromServer now appends `${item.path}.${item.item_kind}.yaml`
  through the same helper.
- computeWsSpecificFlagOnlyPushes uses getTypeStrFromPath + removeType,
  gated by configKeyForItemKind. No more magic strings.
- isSpecificItem and isItemTypeConfigured normalize trailing `.json` to
  `.yaml` once at the entry, so a single set of patterns covers both
  extensions for the same logical item.

* refactor: dedicated change type for ws_specific flag-only pushes

Previously the sync push code injected a synthetic 'edited' Change with
before === after to nudge the apply loop into calling pushResource /
pushVariable for ws_specific-flag-only diffs, and a guard inside those
two functions skipped the early-return when the flag differed. The
contract was implicit and easy to break — any future 'skip identical
edits' optimization in the change pipeline would silently drop these
pushes.

Replace with an explicit Change variant:

  type WsSpecificFlag = {
    name: 'ws_specific_flag';
    path: string;
    kind: string;
    wsSpecific: boolean;
  };

The push apply loop now has a dedicated branch for it that calls
wmill.updateResource / updateVariable with just the ws_specific flag.
prettyChanges renders it on its own line. The dry-run JSON output picks
it up via the existing change.name / change.path passthrough.

The defensive wsSpecificMatches check inside push{Resource,Variable} is
no longer needed (sync push doesn't go through them for flag-only
diffs) and is reverted.

* drop folders

* feat(cli): warn on remote ws_specific items missing from local config

When 'wmill sync pull' fetches the server's ws_specific list, items the
server marks as ws_specific but that aren't matched by the local
wmill.yaml's specificItems patterns now produce a warning. The merge
already preserves correctness (those items are still treated as ws_specific
during this pull), but the user's config drifts from the remote — and a
later push from another machine without that config would push the item
as non-ws_specific. Surface the drift so the user can update wmill.yaml.

Also filter ws_specific_flag changes out before preCheckPermissionedAs
(it expects added/edited/deleted only and they have no content payload
so on_behalf_of resolution doesn't apply).

* fix(cli): scope ws_specific drift warning to items in this pull's changes

Previously the warning iterated every ws_specific item the server returned,
producing log spam for items unrelated to the current pull (items that
exist locally with no change, or items the user has nothing to do with
this round). Move the loop after compareDynFSElement and only warn for
items whose path appears in the changes list — i.e., items the user is
actually pulling right now.

* fix: clean up linked-side ws_specific rows on resource/variable delete

Three places left orphaned ws_specific rows behind:

1. delete_resource deleted the resource's own ws_specific row and the
   linked variables, but never the ws_specific 'variable' rows that
   mark_linked_variables_ws_specific had auto-inserted for those
   variable paths.
2. delete_variable deleted its own ws_specific row and the linked
   resource at the same path, but never a ws_specific 'resource' row at
   that path.
3. delete_resources_bulk didn't even cascade to linked variables, let
   alone clean up their ws_specific rows.

A new resource or variable later created at one of those paths would
silently inherit a stale ws_specific flag — list_ws_specific would
report it as workspace-specific, workspace diffs would treat it as
'no changes', and CLI sync would skip it.

Fix:

- delete_resource: DELETE FROM ws_specific WHERE item_kind = 'variable'
  AND path = ANY(linked_var_paths) before the linked-variable delete.
- delete_variable: DELETE FROM ws_specific WHERE item_kind = 'resource'
  AND path = path before the linked-resource delete.
- delete_resources_bulk: collect $var: refs from each bulk-deleted
  resource (mirror of single delete), then delete ws_specific 'variable'
  rows AND the variable rows themselves. Brings bulk delete in line with
  single delete semantics, including the orphan cleanup.

* fix: gate list_ws_specific by resource/variable RLS

The endpoint queried ws_specific directly under user_db, but ws_specific
itself has no per-item RLS — only a workspace-level column. Any workspace
member could enumerate every ws_specific path including those in folders
they lack read access to (e.g. f/finance/prod_db_creds), revealing path
existence that list_resources / list_variables would have hidden.

Add EXISTS clauses against resource and variable so the same path-based
RLS policies that govern those tables (see_own / see_member /
see_extra_perms_user / see_extra_perms_groups / see_folder_extra_perms_user)
also gate visibility here. The user transaction already establishes the
session context; the joins make the policies apply.

* only resources and variables

* fix(cli): make workspace-specific path mapping handle .json files

isSpecificItem() was extended to normalize .json -> .yaml so .json
files could be matched against patterns, but the surrounding helpers
remained yaml-only:

- toWorkspaceSpecificPath only mapped folder.meta.yaml / settings.yaml
  / .X.yaml — a foo.resource.json went through unchanged, so the
  workspace-specific filename was never produced.
- fromWorkspaceSpecificPath only matched .yaml extensions — pushing
  foo.dev.resource.json could not map back to foo.resource.json.
- isCurrentWorkspaceFile / isWorkspaceSpecificFile regexes ended in
  \.yaml$, missing every branch-specific .json file.

Replace the literal '.yaml' anchors with '(yaml|json)' alternations,
preserve the actual extension on round-trips, and rename the helper
buildYamlTypePattern -> buildItemTypePattern (it never had anything
extension-specific in it). getFileTypeSuffix now returns the matching
suffix for either extension. Changed:

- getFileTypeSuffix
- toWorkspaceSpecificPath / fromWorkspaceSpecificPath
- isCurrentWorkspaceFile / isWorkspaceSpecificFile
- isTriggerFile / isScheduleFile

isItemTypeConfigured / isSpecificItem don't need touching — their
checks run after normalizeJsonToYaml(), which already collapses both
extensions to .yaml at the entry.

* fix: create_resource?update_if_exists=true honors ws_specific=false

The upsert path matched on `unwrap_or(false)`, so an explicit
`ws_specific: false` and an absent flag were indistinguishable — both
fell through with no DELETE on the existing ws_specific row. Callers
trying to clear the flag via PUT-with-update_if_exists silently saw
their request ignored.

Mirror update_resource's three-way handling:

  Some(true)  -> INSERT (+ mark linked variables)
  Some(false) -> DELETE (only when update_if_exists, since a pure
                 create has no existing row anyway)
  None        -> leave the existing flag alone

create_variable doesn't have an upsert path (no ON CONFLICT), so the
same bug doesn't apply there.

* sqlx prepare

* test: cover ws_specific cleanup, RLS filtering, upsert clearing, and CLI .json paths

Backend (backend/tests/ws_specific.rs + fixture):

- test_linked_delete_cleanup: creates a ws_specific resource that
  references a variable via $var:, deletes the resource, asserts the
  cross-kind ws_specific row for the auto-marked variable is also
  removed. Then does the inverse for delete_variable, verifying the
  ws_specific 'resource' row at the same path is cleaned by variable
  delete.
- test_list_ws_specific_filters_by_rls: admin creates ws_specific items
  in u/test-user/ and u/test-user-2/; verifies admin sees both via
  list_ws_specific while a non-admin (test-user-2) only sees their own
  path — the RLS see_own policy on the joined resource/variable tables
  hides the other.
- test_create_resource_upsert_clears_ws_specific: covers the three-way
  Option<bool> handling on the upsert path: Some(true) inserts,
  Some(false) clears the existing row, None leaves it alone.

CLI:

- specific_items_unit.test.ts: add 14 tests covering toWorkspaceSpecificPath
  / fromWorkspaceSpecificPath / isWorkspaceSpecificFile /
  isCurrentWorkspaceFile / isSpecificItem / isItemTypeConfigured for
  .json files (variable, resource, trigger, schedule, folder.meta,
  settings).
- ws_specific_flag_only_unit.test.ts (new): covers
  computeWsSpecificFlagOnlyPushes — emits flag-only changes only for
  resource and variable kinds (the backend's list_ws_specific_versions
  scope), does not emit for schedules or triggers, returns empty when
  serverItems is null (older server), respects existing server entries,
  preserves .json extension on filePath.
- Export computeWsSpecificFlagOnlyPushes so it can be unit-tested.

* perf: index workspace_settings.deploy_to for the recursive CTE

list_ws_specific_versions's recursive CTE probes WHERE ws.deploy_to =
r.ws_id every iteration; without an index on workspace_settings.deploy_to
each iteration seq-scans the table — at 10M workspaces with the depth
cap of 32 that's up to 320M row reads per call.

deploy_to is sparse (most workspaces don't deploy anywhere), so a
partial index WHERE deploy_to IS NOT NULL stays small while still
covering every probe. Tucked into the existing migration since the
function and the index ship together.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-07 13:35:54 +00:00
hugocasa e74f06cb56 fix: handle singlestepflow zombies and stop filtering them from runs page (#9055) 
* fix: handle singlestepflow zombies and stop filtering them from runs page

* fix: support singlestepflow in batch_rerun_jobs

Previous PR added singlestepflow to list_selected_job_groups so the BatchReRun
pane shows them, but batch_rerun_jobs_inner still joined on kind = 'script' /
'flow' with j.runnable_id (which is NULL for SingleStepFlow), so the rows were
silently filtered out — user sees the option, click Re-run, gets zero successes.

Mirror the norm_kind CTE projection from list_selected_job_groups inside
batch_rerun_jobs_inner: pull the wrapped runnable type and pinned script hash
from raw_flow.modules[id='a'], cast back to JOB_KIND so the existing handler
dispatch works unchanged. Path-based schema fallback so input_transforms still
resolve at rerun time.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: project singlestepflow in batch-rerun schema lookups

Codex review pointed out two follow-on regressions from the previous fix:

(1) list_selected_job_groups returned schemas with script_hash=null and
schema=null for singlestepflow rows because the inner schemas subquery still
joined runnable metadata via j.runnable_id (NULL for SingleStepFlow). The
BatchReRun pane consumes every selected.schemas entry through
mergeSchemasForBatchReruns / buildExtraLibForBatchReruns, both of which
assume real schema objects.

(2) When use_latest_version=true, batch_rerun_handle_job re-fetched
latest_schema from v2_job filtering jb.kind='script' or 'flow' — neither
matched singlestepflow, so schema came back NULL and every input_transforms
entry silently no-op'd.

Both queries now project singlestepflow rows via raw_flow.modules[id='a'] —
norm_kind for dispatch and effective_hash for the schemas join, plus a
path-based latest-schema fallback so flow-wrapped SSF (no version pinning)
and any SSF whose pinned hash has been deleted still resolve.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test: add batch_rerun integration tests, fix SSF hash hex parsing

Adds 11 integration tests against /jobs/run/batch_rerun_jobs and
/jobs/list_selected_job_groups (both endpoints had zero CI coverage).
Tests cover the full 4-kind × 3-mode matrix: regular Script and Flow
(baseline regression for the SQL refactor), script-wrapped and flow-
wrapped SingleStepFlow (regression for the bugs this PR fixes), and a
mixed-kind batch.

Writing the tests caught a real bug in the previous commit: ScriptHash
serializes as a 16-char hex string in raw_flow.modules[a].value.hash
(per the custom Serialize impl in windmill-types/scripts.rs), not as
an integer. The earlier `(m->'value'->>'hash')::bigint` cast worked
on the hand-inserted SQL fixture I'd used for live testing (which
embedded the hash as a raw integer) but failed in production where
all SSF jobs are pushed via JobPayload::SingleStepFlow's serialized
form. Replaced with `('x' || lpad(hex, 16, '0'))::bit(64)::bigint` —
preserves the twos-complement bit pattern so both positive and
negative i64 hashes round-trip correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Update SQLx metadata

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-05-07 08:00:38 +00:00
Ruben Fiszel 581658d881 fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor (#8951) 
* fix(wac): recognize @workflow main, list WAC in scripts/list, run preprocessor

Three workflow-as-code bug fixes:

- #8945: Python WAC template with `@workflow async def main(...)` was not
  detected as `auto_kind = "wac"`. The detection only ran when no `main`
  function was found. Hoist the heuristic so it runs whether or not `main`
  is the entrypoint.

- #8946: `scripts/list?kinds=script` filtered out WAC scripts because they
  set `auto_kind = 'wac'` and the SQL hid everything that wasn't NULL.
  Allow both NULL and 'wac' (still excluding 'lib' library scripts).

- #8947: Preprocessor functions defined alongside a WAC workflow were
  ignored. Inject the preprocessor invocation into the Python WAC wrapper
  so it runs before the workflow on the first iteration, then plumb the
  preprocessed args through `handle_wac_v2_output` so inline child
  re-runs see the post-preprocessor args via `checkpoint.input_args`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(wac): integration tests for #8946 (scripts/list) and #8947 (preprocessor)

- test_scripts_list_includes_wac: hit GET /scripts/list?kinds=script and
  assert WAC scripts are in the response (would have failed pre-#8946 fix
  because of the auto_kind IS NULL filter).
- test_python_wac_v2_with_preprocessor: deploy a Python WAC script with a
  preprocessor, run with raw event args, assert the workflow saw the
  preprocessed shape and v2_job.args/preprocessed were updated.
- New wac_preprocessor.sql fixture with auto_kind = 'wac' set explicitly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(wac): address PR review feedback

Five review fixes:

- python_executor.rs: WAC preprocessor now runs inside the wrapper's
  `try:` block so failures route through the same `result.json` error
  serializer as workflow failures. Switched async-coroutine handling
  from deprecated `asyncio.get_event_loop().run_until_complete(...)` to
  `asyncio.run(...)` (the recommended primitive on 3.10+).

- bun_executor.rs: when copying preprocessed args into
  `checkpoint.input_args`, surface JSON parse failures via `?` instead
  of silently coercing to `Value::Null` (which would persist a corrupted
  arg into every child re-run). Also collapsed the redundant double
  iteration into a single pass.

- windmill-api-scripts/scripts.rs: switched the runnable-script filter
  from an allow-list (`auto_kind IS NULL OR = 'wac'`) to a deny-list
  (`<> 'lib'`), so future `auto_kind` values aren't silently filtered
  from triggers/dropdowns.

- windmill-parser-py: aligned the parser's WAC heuristic with the
  runtime detector `is_wac_v2_py` — `@task` is now optional, matching
  the runtime which says workflows that only use inline `step()` are
  still WAC. Added a regression test `test_parse_python_wac_step_only`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-27 13:38:49 +00:00
Ruben Fiszel 64c58c824f feat: add deploy restriction rule and fork review requests (#8804) 
* feat: add deploy restriction rule and fork review requests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for fork review requests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments on fork review requests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: rename fork review requests to deployment requests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for deployment request rename

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: inline deployment request panel into deploy layout

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: place Request deployment button to the left of Deploy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: inline fork triggers into main deploy list

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: open real trigger detail drawer for inline fork triggers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: email notifications for merge completion and reply pings

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update deployment_request + protection_rule tables on workspace id rename

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 972893c3870e4c4a70a35748abed282d88904805

This commit updates the EE repository reference after PR #528 was merged in windmill-ee-private.

Previous ee-repo-ref: 5684d1c17d930b17849c1e5d7577891e64682d45

New ee-repo-ref: 972893c3870e4c4a70a35748abed282d88904805

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-04-13 17:10:37 +00:00
Ruben Fiszel 60211c1d19 feat: folder default_permissioned_as rules for ownership defaults on deploy (#8801) 
* feat: add folder default_permissioned_as rules for ownership defaults on deploy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unnecessary auth guard on default_permissioned_as — rules are advisory only

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts with new CLI commands

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CI review findings — TOCTOU, race condition, email validation, type coercion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add sqlx offline cache for test queries (fixes cargo_test CI)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining review findings — incomplete request bodies, dead code, redundant import

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address remaining review findings — full script fields, reactive stores, catch-all validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: app/schedule/trigger set-permissioned-as fetch remote first to avoid data loss

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: app set-permissioned-as avoid creating redundant app version

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: compact user/group toggle + select for folder default_permissioned_as rules

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: collapse default_permissioned_as section by default in folder editor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: include default_permissioned_as in FolderFile CLI type for YAML round-trip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: process folder.meta changes before items in push to apply new rules immediately

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: clone default_permissioned_as on fork/rename + add full lifecycle tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add no-op guarantee test — folder without rules behaves like before

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: rename cliBehavior to syncBehavior — more accurate scope

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-12 20:14:07 +00:00
hugocasa bffa61e33f fix: dedicated worker dispatch, cross-workspace deps, UI improvements (#8689) 
* feat: restore bun as default runtime for dedicated workers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add context comment for bun dedicated worker nodejs migration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker dispatch for flows + add E2E tests

- Add workspace_id prefix to dedicated worker map lookup keys
- Update ee-repo-ref for dedicated worker path handling fix
- Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers
- Add 6 E2E tests for dedicated workers:
  - test_dedicated_flow_rawscript (regression for "Script not found" bug)
  - test_dedicated_flow_workspace_script
  - test_dedicated_flow_multiple_steps
  - test_dedicated_standalone_script
  - test_dedicated_runner_group
  - test_dedicated_flow_runners
- Add dedicated_flows.sql fixture with scripts, flows, and worker config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always run dependency job for dedicated worker scripts

When a script with dedicated_worker=true is deployed with a pre-computed
lock (e.g. via wmill sync push), no dependency job was created, so the
dedicated worker never detected the update and kept running the old version.

Now dedicated worker scripts always generate a dependency job regardless
of whether a lock is provided. The dependency job runs on the dedicated
worker and triggers a restart so it picks up the new script version.

Fixes #8638

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races

Dedicated worker tests need non-default worker tags in the global
WORKER_CONFIG. When run in parallel (CI uses --test-threads=10),
multiple tests clobber each other's config. Use #[serial] to ensure
dedicated worker tests run sequentially.

Also load worker config from DB via load_worker_config() instead of
manually setting WORKER_CONFIG fields, ensuring consistency with the
monitor's reload path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests

Fix script_path shadowing in bun_executor nodejs branch where the wrapper
file path was passed to handle_dedicated_process instead of the logical
path, causing "Script not found" for all //nodejs dedicated workers.

Add E2E tests for dedicated flows in all supported languages:
- test_dedicated_flow_deno
- test_dedicated_flow_python
- test_dedicated_flow_bunnative (V8 PrewarmedIsolate path)
- test_dedicated_flow_bun_nodejs (//nodejs annotation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify dedicated worker dispatch + add serialization and E2E tests

- Unified lookup: always use {workspace}:{runnable_path} for dedicated
  worker dispatch, replacing the flow_step_id iteration approach
- Added serialization_semaphore parameter to executor start_worker fns
- Added E2E tests: cross-workspace isolation, conflicting flow step IDs,
  preprocessor on dedicated worker
- Added workspace field to RunJob for cross-workspace test support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: cross-workspace workspace dependencies on workers page

Add two new instance-level endpoints to the configs router:
- GET /configs/list_all_workspace_dependencies
- GET /configs/list_all_dedicated_with_deps

Both require devops role and return data across all workspaces,
enabling the workers page to show a consistent view of which
workspace dependencies exist regardless of which workspace the
user is browsing.

Update DedicatedWorkersSelector to use the new cross-workspace
endpoints with fallback to per-workspace calls for non-devops users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to include dedicated worker lookup simplification

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update ee-repo-ref.txt with new reference

* sqlx

* fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings

- Remove serialization_semaphore from executor start_worker signatures
- Remove serialization test and fixtures
- Fix DedicatedWorkersSelector to preserve tags from other workspaces
  when toggling in the picker
- Track workspace deps per-workspace for conflict detection
- Show warning when dep exists in another workspace but not the script's
- Group runner groups per-workspace to prevent cross-workspace merging
- Add workspace to dep badge link URL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify exec protocol — execd: for single-script, exec: for runner groups

Add execd:/execd_preprocess: commands to bun/deno/python wrappers for
single-script dedicated workers (no path needed). Runner groups keep
exec:/exec_preprocess: with path for multi-script disambiguation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add unit tests for execd:/exec: wrapper protocol

Verify generate_multi_script_wrapper produces both execd: (single-script)
and exec: (runner group) protocol handlers, including preprocessor variants.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* fix: remove beta badge from squash loop, keep tooltip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update protocol tests to use execd: for single-script wrappers

Deno and bun single-script protocol tests now send execd:{args} instead
of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script
(runner group) tests continue to use exec:{path}:{args}.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused TEST_SCRIPT_PATH in deno protocol tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback — down migration, push_as workspace, UI improvements

- Use regexp_replace in down migration for positional accuracy
- Fix push_as() to use self.workspace_id instead of hardcoded value
- Remove per-workspace API fallbacks, use cross-workspace endpoints only
- Skip devops-only API calls when user is not devops (disabled prop)
- Fix duplicate key error for cross-workspace runner groups
- Add workspace to RunnerGroup for unique keying
- Reuse tagRow snippet for standalone items with expand/collapse
- Fix picker alignment: remove empty column for non-expandable items

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: comprehensive dedicated worker test coverage, fix Python execd_preprocess

- Add Python execd_preprocess: handler (was missing for single-script dedicated workers)
- Add 10 E2E tests: flow+standalone conflict, mixed lang fallback, unsupported lang
  flow runners, python runner group, bun/python/deno/bunnative preprocessors,
  runner group preprocessors, branchone flow
- Add 4 Python unit tests for execd:/execd_preprocess: protocol
- Update EE ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review feedback — migration escaping, deno try/catch, loadRunnables guard

- Down migration: use E'...' so \n matches actual newlines
- Up migration: anchor regex with ^ to avoid mid-content matches
- Deno execd_preprocess: move JSON.parse inside try/catch
- DedicatedWorkersSelector: skip devops-only API calls when disabled

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add dedicated worker relative import tests for bun and python

Verifies that build_loader's CURRENT_PATH correctly resolves workspace-
relative imports when running on a dedicated worker subprocess.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker dispatch for nested flow structures (branches/loops)

- Add extract_flow_root() to strip nesting segments from runnable_path
- Dispatch uses flow_root/flow_step_id for nested paths, runnable_path
  for flat paths — deterministic, O(1)
- Fix assert_ran_on_dedicated_worker to BFS all descendants
- Fix python mode labels (python vs python3 for runner groups)
- Add tests: simple forloop, multi-step forloop, whileloop, branchall,
  nested branch-in-loop, mixed lang fallback, unsupported lang runners

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: fix ee-repo-ref SHA

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: hide picker and skip API calls for read-only users, hide empty runner badge

- Hide "Add more scripts/flows" section when disabled (read-only)
- Skip per-runnable API calls (getScriptByPath, getFlowByPath) for
  disabled users — just show path info
- Hide "0 runners" badge on flows with no eligible steps

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 9422b189762ae27edfc346541ae668a4ad728325

This commit updates the EE repository reference after PR #503 was merged in windmill-ee-private.

Previous ee-repo-ref: 4c6ba214bfc23fff05d1dc3200ac59e650af3f4f

New ee-repo-ref: 9422b189762ae27edfc346541ae668a4ad728325

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-04-03 17:50:07 +00:00
hugocasa 61a867f086 Revert "feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)" (#8687) 
This reverts commit 619ebb65ce.
 2026-04-02 23:09:38 +00:00
hugocasa 619ebb65ce feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645) 
* feat: restore bun as default runtime for dedicated workers

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: add context comment for bun dedicated worker nodejs migration

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: dedicated worker dispatch for flows + add E2E tests

- Add workspace_id prefix to dedicated worker map lookup keys
- Update ee-repo-ref for dedicated worker path handling fix
- Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers
- Add 6 E2E tests for dedicated workers:
  - test_dedicated_flow_rawscript (regression for "Script not found" bug)
  - test_dedicated_flow_workspace_script
  - test_dedicated_flow_multiple_steps
  - test_dedicated_standalone_script
  - test_dedicated_runner_group
  - test_dedicated_flow_runners
- Add dedicated_flows.sql fixture with scripts, flows, and worker config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always run dependency job for dedicated worker scripts

When a script with dedicated_worker=true is deployed with a pre-computed
lock (e.g. via wmill sync push), no dependency job was created, so the
dedicated worker never detected the update and kept running the old version.

Now dedicated worker scripts always generate a dependency job regardless
of whether a lock is provided. The dependency job runs on the dedicated
worker and triggers a restart so it picks up the new script version.

Fixes #8638

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races

Dedicated worker tests need non-default worker tags in the global
WORKER_CONFIG. When run in parallel (CI uses --test-threads=10),
multiple tests clobber each other's config. Use #[serial] to ensure
dedicated worker tests run sequentially.

Also load worker config from DB via load_worker_config() instead of
manually setting WORKER_CONFIG fields, ensuring consistency with the
monitor's reload path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests

Fix script_path shadowing in bun_executor nodejs branch where the wrapper
file path was passed to handle_dedicated_process instead of the logical
path, causing "Script not found" for all //nodejs dedicated workers.

Add E2E tests for dedicated flows in all supported languages:
- test_dedicated_flow_deno
- test_dedicated_flow_python
- test_dedicated_flow_bunnative (V8 PrewarmedIsolate path)
- test_dedicated_flow_bun_nodejs (//nodejs annotation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify dedicated worker dispatch + add serialization and E2E tests

- Unified lookup: always use {workspace}:{runnable_path} for dedicated
  worker dispatch, replacing the flow_step_id iteration approach
- Added serialization_semaphore parameter to executor start_worker fns
- Added E2E tests: cross-workspace isolation, conflicting flow step IDs,
  preprocessor on dedicated worker
- Added workspace field to RunJob for cross-workspace test support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: cross-workspace workspace dependencies on workers page

Add two new instance-level endpoints to the configs router:
- GET /configs/list_all_workspace_dependencies
- GET /configs/list_all_dedicated_with_deps

Both require devops role and return data across all workspaces,
enabling the workers page to show a consistent view of which
workspace dependencies exist regardless of which workspace the
user is browsing.

Update DedicatedWorkersSelector to use the new cross-workspace
endpoints with fallback to per-workspace calls for non-devops users.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to include dedicated worker lookup simplification

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update ee-repo-ref.txt with new reference

* sqlx

* fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings

- Remove serialization_semaphore from executor start_worker signatures
- Remove serialization test and fixtures
- Fix DedicatedWorkersSelector to preserve tags from other workspaces
  when toggling in the picker
- Track workspace deps per-workspace for conflict detection
- Show warning when dep exists in another workspace but not the script's
- Group runner groups per-workspace to prevent cross-workspace merging
- Add workspace to dep badge link URL

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: simplify exec protocol — execd: for single-script, exec: for runner groups

Add execd:/execd_preprocess: commands to bun/deno/python wrappers for
single-script dedicated workers (no path needed). Runner groups keep
exec:/exec_preprocess: with path for multi-script disambiguation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test: add unit tests for execd:/exec: wrapper protocol

Verify generate_multi_script_wrapper produces both execd: (single-script)
and exec: (runner group) protocol handlers, including preprocessor variants.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* fix: remove beta badge from squash loop, keep tooltip

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update protocol tests to use execd: for single-script wrappers

Deno and bun single-script protocol tests now send execd:{args} instead
of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script
(runner group) tests continue to use exec:{path}:{args}.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused TEST_SCRIPT_PATH in deno protocol tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-02 19:37:02 +00:00
hugocasa efb4a27d51 fix: replace email with permissioned_as for triggers/schedules (#8439) 
* refactor: replace email with permissioned_as for triggers/schedules

Add a new `permissioned_as` column (format: `u/{username}`, `g/{group}`,
or raw email) to all trigger tables and schedule. This value is used
directly for job permission checks, removing the need for email lookups
when creating/updating triggers.

- Migration: add permissioned_as to all 9 trigger tables + schedule,
  drop email from trigger tables (schedule keeps it for backwards compat)
- Backend: resolve_email() (async, DB) -> resolve_permissioned_as() (sync)
- Email cache: get_email_from_permissioned_as() with quick_cache for
  places that still need email (fetch_api_authed, schedule backwards compat)
- Frontend: rename email/preserve_email -> permissioned_as/preserve_permissioned_as
  in deploy data and OpenAPI schemas
- Tests updated for new field names and u/{username} format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix sqlx/build

* update ee ref

* refactor: simplify resolve_edited_by to always use authed username

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix compile + migration

* update ref

* test: add trigger trait method tests for permissioned_as queries

Add tests that call TriggerCrud and Listener trait methods directly
to verify dynamic SQL correctly references the permissioned_as column.
Covers get_trigger_by_path, list_triggers, set_trigger_mode, and
fetch_enabled_unlistened_triggers for all trigger types.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update sqlx

* fix: use permissioned_as directly for schedules and fix audit RLS for groups

- Schedule: permissioned_as only set on create, not on edit/set_enabled
- Schedule: stop reading email column, use get_email_from_permissioned_as
- Triggers: use fetch_api_authed_from_permissioned_as instead of edited_by
- Triggers: rename listener fields for clarity (username -> edited_by)
- Fix audit author username for group permissioned_as (g/test -> group-test)
  to match session.user, preventing RLS policy violations on audit_partitioned
- OpenAPI: remove permissioned_as/preserve_permissioned_as from EditSchedule
- Add backwards-compat comments for schedule email writes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build

* refactor: generalize onBehalfOf naming, add permissioned_as to EditSchedule

- Frontend: rename onBehalfOfPermissionedAs -> onBehalfOf with comments
  explaining it carries emails for flows/scripts and permissioned_as for
  triggers/schedules
- Frontend: rename getOnBehalfOfEmail -> getOnBehalfOf,
  getOnBehalfOfPermissionedAsForDeploy -> getOnBehalfOfForDeploy,
  customOnBehalfOfEmails -> customOnBehalfOf
- Backend: add optional permissioned_as/preserve_permissioned_as to
  EditSchedule with COALESCE (only updates when provided)
- Backend: add on_behalf_of audit log for schedule edit
- Backend: remove unused resolve_on_behalf_of_permissioned_as
- Tests: remove email assertions from schedule update test (email is
  just backwards compat, only permissioned_as matters)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: preserve email column when permissioned_as is preserved on schedule edit

Derive email from the preserved permissioned_as via cache lookup instead
of always writing authed.email. This keeps the email column consistent
with the old behavior for backwards compat with old workers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: update deploy UI labels from "edited by" to "run as" for triggers

Triggers now use permissioned_as (not edited_by) for permissions, so
update the deploy UI wording to reflect this. Also update wm_deployers
group description to mention schedules and permissioned_as.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use u/username format for custom trigger/schedule deploy selection

When picking a custom user for trigger/schedule deployment, store
u/${username} (permissioned_as format) instead of the email. Flows/scripts
continue to use email format for on_behalf_of_email.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show u/username format for "me" option in trigger deploy selector

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: simplify OnBehalfOfSelector to return the right format per kind

OnBehalfOfSelector now handles the email vs permissioned_as format
internally based on kind:
- triggers: returns u/username, displays u/username in all options
- flows/scripts/apps: returns email, displays username

The onSelect callback now takes (choice, value?) where value is already
in the correct format. Parent components just store it directly without
needing to know about the format difference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always show u/username format in OnBehalfOfSelector for all kinds

Display is now consistent: all kinds show u/username in the selector.
The returned value still differs (email for flows/scripts, u/username
for triggers) since the backend APIs expect different formats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: replace email with permissioned_as in http_trigger test insert

The email column was dropped from trigger tables in the migration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: review fixes — migration, app policy, capture cleanup, naming

- Migration: remove DEFAULT '', use nullable → populate → SET NOT NULL
- App policy: set both on_behalf_of and on_behalf_of_email for all choices
- OnBehalfOfSelector: return OnBehalfOfDetails {email, permissionedAs} instead of ambiguous value
- Remove unused email field from Capture struct and query
- Rename getSourceEmail/getTargetEmail → getSourceOnBehalfOf/getTargetOnBehalfOf
- Rename test functions from preserve_email to preserve_permissioned_as

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to all test schedule INSERTs

Since the migration no longer uses DEFAULT '', all INSERTs must
explicitly provide permissioned_as. Updated test fixtures and
schedule_push tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: strip permissioned_as from exports/sync, fix OpenAPI required field

- Add permissioned_as to workspace export strip list (like edited_by)
- Add permissioned_as to CLI TriggerFile Omit list
- Fix TriggerExtraProperty.required: email → permissioned_as
- Regenerate frontend and CLI types

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove accidentally committed generated files

These directories are gitignored and should not be tracked.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: regenerate system prompts for permissioned_as schema changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove permissioned_as from CLI TriggerFile Omit list

Already stripped in workspace export, no need to also omit from the type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: optimize email cache key and revert TriggerFile Omit change

- Use single concatenated string for cache key instead of (String, String) tuple
- Remove permissioned_as from CLI TriggerFile Omit (already stripped in export)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: zero-allocation email cache lookups using Equivalent trait

Use a borrowed EmailCacheKey(&str, &str) for cache lookups via
quick_cache's Equivalent support. Only allocates (String, String)
on cache miss for insert. This is called on every trigger fire
and schedule push.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add permissioned_as to Schedule required fields in OpenAPI spec

The backend always returns permissioned_as (non-optional String),
so the schema should reflect that.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: handle group- prefix in migration UPDATE statements

edited_by can be 'group-{name}' for group-owned triggers/schedules.
The migration now correctly maps these to 'g/{name}' format instead
of incorrectly producing 'u/group-{name}'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "fix: handle group- prefix in migration UPDATE statements"

This reverts commit 0971392b38.

* fix: use superadmin email to resolve permissioned_as in schedule migration

For users upgrading from older versions where edited_by may not reflect
the actual schedule owner, check if the email belongs to a superadmin
and look up their username. Otherwise fall back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fall back to superadmin email when not in workspace usr table

If the superadmin isn't a member of the workspace, use their email
as raw permissioned_as instead of falling back to edited_by.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: always update permissioned_as and email on schedule edit

Consistent with pre-refactor behavior where email and edited_by
were always updated on every edit. permissioned_as is now always
set (to editing user or preserved value), removing the COALESCE
that previously preserved it when not provided.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add schedule permission tests and centralize group prefix constants

Tests: schedule create/update for normal user, workspace admin, and
superadmin not in workspace. Verifies schedule fields (email,
permissioned_as, edited_by) and pushed job fields (permissioned_as,
permissioned_as_email).

Constants: centralize "u/", "g/", "group-" as PERMISSIONED_AS_USER_PREFIX,
PERMISSIONED_AS_GROUP_PREFIX, USERNAME_GROUP_PREFIX.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: use @unknown.windmill.dev for synthetic email fallback

Prevents privilege escalation: a user with username like
'superadmin_secret' would get superadmin via the synthetic
email matching SUPERADMIN_SECRET_EMAIL. Using a different
subdomain avoids any collision with hardcoded @windmill.dev emails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* update ee ref

* sqlx

* chore: regenerate system prompts after main merge

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to bda51bc33bcb573659e7ff07d0a23ff6e23b8148

This commit updates the EE repository reference after PR #468 was merged in windmill-ee-private.

Previous ee-repo-ref: 8cf1802f8fe183f430830590b4f3172a50207843

New ee-repo-ref: bda51bc33bcb573659e7ff07d0a23ff6e23b8148

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-03-20 16:28:38 +00:00
Ruben Fiszel f2be625348 feat: store hashed tokens instead of plaintext (#8217) 
* feat: store hashed tokens in the token table instead of plaintext

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review issues in token hash migration

- Update all base.sql fixtures to include token_hash/token_prefix columns
- Keep plaintext token for webhook tokens (needed for URL reconstruction)
- Restore get_token_by_prefix to query DB for webhook tokens
- Fix down migration to delete NULL-token rows before restoring NOT NULL
- Update parser fixture standalone schema
- Update EE dedicated_worker_ee.rs to use token_hash/token_prefix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore sqlx offline cache (only add new query files)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: keep writing plaintext token column for backward compat

Write to token column alongside token_hash until MIN_VERSION_SUPPORTS_TOKEN_HASH
(1.649.0) is reached. This ensures older workers can still authenticate
during rolling upgrades. Remove the separate UPDATE in new_webhook_token
since create_token_internal now writes plaintext directly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: branch on MIN_VERSION to write plaintext token or null

Check MIN_VERSION_SUPPORTS_TOKEN_HASH at runtime: write plaintext to
token column while old workers exist, switch to NULL once all workers
are >= 1.649.0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: set MIN_VERSION_SUPPORTS_TOKEN_HASH to 1.650.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use token_hash for email lookup and expiry notifications

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rotate webhook tokens instead of recovering plaintext from DB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use token_hash for native trigger token lookups and deletes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* sqlx

* refactor: drop webhook_token_prefix from native_trigger table

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: backward compat for token rotation and make webhook_token_hash NOT NULL

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prevent panic on short superadmin secret token prefix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prevent panic on short superadmin secret token prefix

Replace all `token[0..TOKEN_PREFIX_LEN]` slicing with
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent
panics when a token shorter than 10 chars is provided (e.g.
malformed Authorization header, short superadmin secret).

Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>

* fix: prevent panic on short token prefix slicing

Replace all `token[0..TOKEN_PREFIX_LEN]` with safe
`token.get(..TOKEN_PREFIX_LEN).unwrap_or(token)` to prevent panics
on malformed tokens shorter than 10 characters.

Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: prevent panic on short superadmin secret token prefix"

This reverts commit 37ec2e5ad5.

* revert: remove unnecessary defensive token prefix slicing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add token_hash to end_user_email test fixture

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: add integration tests for token hash migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: correct token_hash test assertions for cache and version

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add plaintext column removal reminder to test fixtures

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: log count of orphaned triggers deleted during migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: preserve orphaned triggers with error instead of deleting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: rename token_expiry_notification.token to token_hash and copy owner/expiration in rotate

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: hash existing plaintext values before renaming token_expiry_notification column

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove unnecessary length check in token_expiry_notification migration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* update dates and version

* updat ee ref + sqlx

* improve mcp migration

* fix: atomic token rotation with rollback on trigger update failure

rotate_webhook_token now atomically inserts the new token and deletes
the old one in a single transaction, preventing token leaks.

Returns new_token_hash so callers can clean up the new token if their
subsequent trigger update fails (which involves external HTTP calls
and cannot be in the same DB transaction).

- Handler: wraps post-rotation work; deletes new token on failure
- Google renewal: deletes new token if service_config update fails
- Tests updated to match new atomic semantics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* higher min version

* fix: defer old token deletion to avoid breaking triggers on update failure

rotate_webhook_token now keeps the old token alive and returns
old_token_hash. Callers delete it only after the trigger row has been
successfully updated. If the external service call or DB update fails,
the trigger keeps working with the old token.

Worst case: if the best-effort delete fails, the old token leaks as an
extra DB row — harmless compared to breaking the trigger.

Also update summarized_schema.txt for renamed columns.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to 2d0823a471014e2bc2d898c63518323946b7474f

This commit updates the EE repository reference after PR #437 was merged in windmill-ee-private.

Previous ee-repo-ref: 7aef8b06cb6f54c2bc89dd57b70947deed72553c

New ee-repo-ref: 2d0823a471014e2bc2d898c63518323946b7474f

Automated by sync-ee-ref workflow.

* fix: prevent panic on short tokens by using safe prefix extraction

Add safe_token_prefix() helper that uses .get(..TOKEN_PREFIX_LEN).unwrap_or(token)
instead of direct slice indexing, preventing panics when tokens are shorter than
10 characters (e.g., short superadmin secrets or malformed Bearer tokens).

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
 2026-03-17 01:15:38 +00:00
hugocasa dc0e59f432 feat: add preprocessor support for dedicated workers and bunnative scripts (#8284) 
* feat: add preprocessor support for dedicated workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: extract transform_and_run helper in python dedicated wrapper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add preprocessor support for bunnative scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: resolve unused postprocessor variable in python wrapper

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* chore: add workflow_dispatch trigger to backend integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: bunnative fixture lock format and PrewarmedIsolate::spawn callers

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: update generate_dedicated_worker_wrapper callers in bun_jobs test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use non-dedicated workers in preprocessor integration tests

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: prewarm preprocessor isolate for bunnative dedicated workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: flatten bunnative dedicated worker preprocessing into single result path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use labeled block instead of async block for EE compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update commit reference in ee-repo-ref.txt

* chore: update ee-repo-ref to e36945b987f7904fa984181baf3124e7b2722bd1

This commit updates the EE repository reference after PR #445 was merged in windmill-ee-private.

Previous ee-repo-ref: 8a2625833452aadb8907242bf502b24ca2dffd73

New ee-repo-ref: e36945b987f7904fa984181baf3124e7b2722bd1

Automated by sync-ee-ref workflow.

* Fix merge conflict in ee-repo-ref.txt

Resolve merge conflict in ee-repo-ref.txt

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2026-03-10 18:26:02 +00:00
Pyra baf2bcf14d feat: make WM_END_USER_EMAIL display users from different workspaces (#8208) 
Signed-off-by: pyranota <pyra@duck.com>
 2026-03-04 11:50:59 +00:00
wendrul 7ac93f6ee3 feat: option to preserve on_behalf_of and edited_by for admins and users in the new wm_deployers group (#8079) 2026-02-25 12:05:22 +00:00
Ruben Fiszel 4fedfdfd11 feat(cli): add consistent get/list/new subcommands for all item types (#8047) 
* feat(cli): add consistent get/list/new subcommands for all item types

Make the CLI consistent so every item type (script, flow, app, resource,
resource-type, variable, schedule, folder, trigger) supports get/list/new
subcommands, enabling the CLI to be used as a full API client in bash
scripts with jq piping.

- Add --json flag to all list commands for machine-readable output
- Register explicit "list" subcommand alongside default action
- Add "get <path> [--json]" subcommand to fetch single items from API
- Rename "bootstrap" to "new" for script/flow, keep "bootstrap" as alias
- Add "new" subcommand for resource, resource-type, variable, schedule,
  folder, and trigger to create local template YAML files
- Update cli-commands skill documentation for wmill init
- Add integration tests for all new commands

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* feat: install wmill CLI in Docker images and use it for bash variable/resource access

- Install windmill-cli via bun in all Dockerfiles that include bun
- DockerfileCli: switch from node:slim to oven/bun:slim
- CLI: auto-configure from WM_WORKSPACE/WM_TOKEN/BASE_INTERNAL_URL env vars
  as last-resort fallback when no workspace is configured
- Frontend: replace curl-based bash snippets with wmill variable/resource get
- Add backend integration tests for wmill CLI in bash scripts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): install windmill-cli in backend test workflow

Ensures wmill is available on PATH for bash integration tests
that use `wmill variable get` and `wmill resource get`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(cli): replace @std/* Deno dependencies with Node.js equivalents

Replace @std/log with a lightweight custom logger (core/log.ts),
@std/path with node:path, and @std/yaml with the yaml npm package.
Also fix process hang on exit, add --node option to install_dev.sh,
and add missing hasRequiredPermissions to NpmProvider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* refactor(cli): replace @ayonli/jsext and @std/encoding with lightweight alternatives

Replace @ayonli/jsext (8.4MB) with tar-stream (32kB) for tar creation,
replace @std/encoding with Node.js Buffer.toString("hex"), and fix
@windmill-labs/shared-utils to use direct npm instead of JSR mirror.
Also resolve merge conflicts in sync.ts and fix pre-existing type errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(cli): use singleQuote YAML output and pass yamlOptions in gitsync pull

The yaml library defaults to double quotes, but the codebase (and tests)
expect single-quoted strings. Add singleQuote: true to yamlOptions and
pass yamlOptions to gitsync-settings pull writeFile calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* fix(cli): address code review feedback

- Install CLI from source in backend tests instead of npm
- Fix script bootstrap catch block to re-throw "File already exists"
- Add type-safe local variable after trigger kind validation
- Use created_by instead of policy.on_behalf_of for app get output
- Note --kind is recommended for faster trigger lookup in help text
- Document node symlink purpose in Dockerfiles

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): use /usr/bin for wmill wrapper to ensure it's in PATH

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(ci): install wmill to ~/.local/bin to avoid permission issues

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci(backend): switch to Blacksmith runner and add cargo caching

- Switch from ubicloud-standard-16 to blacksmith-16vcpu-ubuntu-2404 for faster NVMe-backed builds
- Add stickydisk for cargo target directory (persistent NVMe cache across runs)
- Add cache for cargo registry and git dependencies
- Upgrade DuckDB FFI cache from actions/cache@v3 to useblacksmith/cache@v1
- Enable CARGO_INCREMENTAL=1 to benefit from persistent target cache

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix ci

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-22 07:53:28 +00:00
Ruben Fiszel ff70a4e9d1 fix: parse Python datetime.datetime and datetime.date type annotations (#7856) 
* fix: parse Python datetime.datetime and datetime.date type annotations correctly

The Python parser only matched ExprKind::Name for type annotations, so
`datetime.datetime` (an Attribute expression) silently fell through to
Typ::Unknown and no datetime picker was shown in the UI.

- Extend parse_expr to resolve `datetime.*` attribute access (alongside
  the existing `wmill.*` handling)
- Add Typ::Date variant for `datetime.date` → JSON schema format "date"
- Update python worker to import and convert `date.fromisoformat()`
- Update argSigToJsonSchemaType, AI types, schema validation, and SQL
  datatype wasm for the new Date variant

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-09 15:17:26 +00:00
Ruben Fiszel dc5e69481d test: add end-to-end trigger integration tests and DB CRUD tests 
Add 7 #[ignore] e2e tests (one per trigger type) that fire real messages
to external services and verify job creation in v2_job. Also add 9 DB-level
CRUD tests for MQTT, GCP, and Email triggers.

Includes helper shell scripts in tests/fixtures/ to start/stop each
external service (MQTT, WebSocket, Postgres replication, Kafka, NATS,
SQS via LocalStack, GCP Pub/Sub emulator).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-09 12:28:25 +00:00
Ruben Fiszel 84d40ba8fd test: expand workspace endpoint coverage to ~40 endpoints 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 22:06:53 +00:00
Ruben Fiszel 862ccbbac9 test: expand endpoint coverage with workspaces, global users, and unauthed routes 
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-08 21:57:06 +00:00
Ruben Fiszel c8f539bb57 test: add integration tests for schedules, groups, folders, users, and drafts 
Also extends existing tests with additional endpoint coverage:
- scripts: archive/h, delete/h
- flows: get/v/:version
- apps: get/v/:version, custom_path_exists
- resources: list_names/:type
- base fixture: add password entry for whoami LEFT JOIN

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 21:25:35 +00:00
Ruben Fiszel c787ac9a9c add back mcp_tools endpoints 2026-02-08 20:50:58 +00:00
Ruben Fiszel 6cd1d3e050 chore: remove obsolete debouncing tests 
The test file used old JobPayload/FlowValue field names that were
refactored into DebouncingSettings/ConcurrencySettings structs.
Remove the test file, fixture, feature flag, and cfg gate in monitor.rs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-08 17:39:27 +00:00
Ruben Fiszel e655a06563 fix: prevent schedule pool connection exhaustion (#7821) 
* fix: prevent schedule pool connection exhaustion

Schedules were disabling themselves after upgrading from v1.605.0 to
v1.614.0 due to pool connection deadlock. The root cause was
fetch_authed_from_permissioned_as acquiring a pool connection inside
push() while a transaction already held one, exhausting the pool under
pressure.

Fix: pre-compute Authed before db.begin() for the normal path, and
reuse the transaction connection via fetch_authed_from_permissioned_as_conn
for the on_behalf_of_email path. Peak pool usage drops from 2 to 1 for
all schedule push paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace pool exhaustion tests with comprehensive schedule push tests

Replace the 16 pool-specific tests with 19 tests covering all schedule
push code paths: script/flow scheduling, on_behalf_of_email (script and
flow), retry wrapping, duplicate detection, invalid timezone/cron/args,
script/flow not found, paused schedules, clock shift detection, disabled
schedule, path mismatch, push failure disabling schedule, and trigger
metadata.

Also simplify the obo_authed pattern in push_scheduled_job to use a
single match assignment instead of two bindings with .or() chaining.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-02-05 23:25:42 +00:00
hugocasa ef89a51f3a feat: upgrade bun to v1.3.8 with regression tests (#7761) 
* test: add bun executor tests with minimal production code changes

- Add comprehensive bun job tests (bun_jobs.rs) covering:
  - Basic execution, error handling, annotation modes
  - Relative imports, deeply nested imports
  - Dedicated worker protocol for both Node.js and Bun runtimes
  - Builder tests for lockfile generation (import scanning)

- Minimize changes to bun_executor.rs by exposing:
  - RELATIVE_BUN_LOADER and RELATIVE_BUN_BUILDER constants
  - build_loader() function and LoaderMode enum
  - BUN_DEDICATED_WORKER_ARGS constant
  - generate_dedicated_worker_wrapper() function

- Tests call production code directly (build_loader) instead of
  duplicating script generation logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* nit

* fix: reuse BUN_PATH/NODE_BIN_PATH from windmill-worker, add node to CI

- Tests now use exported BUN_PATH and NODE_BIN_PATH constants instead
  of duplicating env var logic
- Update backend-test.yml:
  - Upgrade bun to v1.3.8
  - Add setup-node action
  - Add NODE_BIN_PATH to cargo test command

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* add private repo test

* fix private repo test

* try fix again

* fix

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
 2026-02-04 11:38:15 +00:00
Ruben Fiszel 6ec4b92999 test: add integration tests for instance group and workspace auto-add interaction 
Add tests verifying the interaction between instance groups and workspace
auto-add functionality:
- Users in instance groups get auto-added to configured workspaces
- Role assignment (admin/operator/developer) works correctly
- Role precedence when user belongs to multiple groups
- User removal when removed from instance group
- Cleanup when instance groups removed from workspace config
- added_via field tracking

Tests are ignored by default in CI and can be run locally with:
cargo test -p windmill --test instance_group_auto_add --features private,enterprise -- --ignored

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-26 23:46:27 +00:00
Ruben Fiszel 865ab70c89 add comprehensive permissions tests and restrict operator script archiving 
- Add permissions test suite covering scripts, flows, apps, resources,
  variables, schedules, and folder-based permissions
- Add test for operator restrictions (cannot create/update/archive)
- Restrict operators from archiving scripts (archive_script_by_path,
  archive_script_by_hash) for security consistency with flows/apps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-25 23:33:54 +00:00
Ruben Fiszel 1b9d1c56c7 feat: add HashiCorp Vault secret storage integration (#7599) 
* feat: add HashiCorp Vault secret storage integration

- Create SecretBackend trait abstraction for secret storage
- Add VaultBackend implementation with CRUD operations
- Integrate secret backend into variable CRUD operations
- Add migration functions (DB → Vault and Vault → DB)
- Add frontend configuration UI for secret backend
- Add test connection and migration endpoints
 2026-01-18 23:08:29 +00:00
Pyra d38c96db36 feat: workspace dependencies (#7124) 
* commit raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* implement `parse_annotation`

Signed-off-by: pyranota <pyra@duck.com>

* more progress on wdeps

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* cli improvements + raw deps

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix python versions

Signed-off-by: pyranota <pyra@duck.com>

* progress

Signed-off-by: pyranota <pyra@duck.com>

* update :)

Signed-off-by: pyranota <pyra@duck.com>

* add MaybeLock

Signed-off-by: pyranota <pyra@duck.com>

* go WIP

Signed-off-by: pyranota <pyra@duck.com>

* fix python ignoring py version from requirements

Signed-off-by: pyranota <pyra@duck.com>

* optimize php

Signed-off-by: pyranota <pyra@duck.com>

* require admin to alter

Signed-off-by: pyranota <pyra@duck.com>

* fix(cli): flow generateLocks raw deps

Signed-off-by: pyranota <pyra@duck.com>

* progress in checklist

Signed-off-by: pyranota <pyra@duck.com>

* fix agent workers

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nit: remove default features

Signed-off-by: pyranota <pyra@duck.com>

* oh-wow

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* add indexes

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* add debug flag

Signed-off-by: pyranota <pyra@duck.com>

* cli: remove noise

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* trigger deps correctly

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend again

Signed-off-by: pyranota <pyra@duck.com>

* finally fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* more fixes...

Signed-off-by: pyranota <pyra@duck.com>

* remove test

Signed-off-by: pyranota <pyra@duck.com>

* Update backend-test.yml

* comment out legacy test

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix ci?

Signed-off-by: pyranota <pyra@duck.com>

* comment out thing

Signed-off-by: pyranota <pyra@duck.com>

* ignore test

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* base fixture

Signed-off-by: pyranota <pyra@duck.com>

* fix regression

Signed-off-by: pyranota <pyra@duck.com>

* fix docs links

Signed-off-by: pyranota <pyra@duck.com>

* update min version

Signed-off-by: pyranota <pyra@duck.com>

* simplify

* implement cache for get_latest

Signed-off-by: pyranota <pyra@duck.com>

* move to workspace settings

Signed-off-by: pyranota <pyra@duck.com>

* sqlx + migration

Signed-off-by: pyranota <pyra@duck.com>

* more migrations

Signed-off-by: pyranota <pyra@duck.com>

* use box pin

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2025-11-28 16:00:11 +00:00
hugocasa cf86881528 fix(backend): add locks to inline preprocessor/failure/tool modules in flows (#6825) 
* fix(backend): add locks to inline preprocessor/failure/tool modules in flows

* fix merge

* fix tests

* fix tests

* add test for ai agent tool

* nit

* fix build

* fix locking
 2025-10-30 10:33:36 +00:00
Pyra edece035f8 feat: job debouncing (#6878) 
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
 2025-10-23 20:18:08 +00:00
Pyra defb6c9694 feat: dependency job debouncing (#6769) 
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

* feat: dependency job debouncing

* checkpoint

Signed-off-by: pyranota <pyra@duck.com>

* more improvements

Signed-off-by: pyranota <pyra@duck.com>

* refactor: clean up dependency job debouncing implementation

- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
  - extract_to_relock_from_args() for extracting nodes/components
  - accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* better error handling for helper

Signed-off-by: pyranota <pyra@duck.com>

* update ee-ref

Signed-off-by: pyranota <pyra@duck.com>

* test: add race condition test for dependency job debouncing

- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment

The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.

This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* test: implement race condition test for dependency job debouncing

- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* implement fallback

Signed-off-by: pyranota <pyra@duck.com>

* make it mostly work

* all tests are almost working

Signed-off-by: pyranota <pyra@duck.com>

* add comments a bit

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat: improve debouncing documentation, tracing, and error handling

- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* do some work for future improvements

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* flag tests behind the feature, add timeout

Signed-off-by: pyranota <pyra@duck.com>

* fix timeout + cleanup

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* row lock debounce_key

Signed-off-by: pyranota <pyra@duck.com>

* addressing TODOs

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2025-10-16 16:49:35 +00:00
Pyra ed806bf9d0 fix(backend): rework dependency_map handling (#6598) 
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
 2025-09-24 13:40:39 +00:00
pyranota 26222539e6 feat(python): inline script metadata (PEP 723) (#5712) 
* make resolver

* more updates

* fix build

* fix raw_dependencies job type

* compat with http agent workers

* refactor

* rename

* more refactor

* cleanup

* more tests

* fix s3

* small fixes

* more fixing

* fix endpoint

* nit: update comment

* update ee ref

* update ee ref

* update ee ref

* implement safer `list_available_python_versions`

* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix build

* update ee ref

* ignore versions <=3.9

* fix windows build

* correct versions filter

* fix windows build (this time for real)

* inject error to debug CI

* update CI

* undo debug of CI

* fix tests

* remove outdated comment

* update ee repo ref

* Update ee-repo-ref.txt

* Update backend/parsers/windmill-parser-py-imports/src/lib.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update InstanceSetting.svelte

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
 2025-05-24 02:13:47 +02:00
pyranota 0b6d017fed feat(python): per import requirement pin (#5520) 
* implement single line pin

* make panic-safe

* use pin even if multiple modules imported withing single statement

* add repins and make imports respect pins

* keep all pins

* Allow multiple pins

* add comments + handle stuff more safely

* fix fully qualified imports

* remove ignore

* sort nested

* apply unique to output requirements list

* fix typo

* remove mut

* update sqlx

* sort imports

* sort imports

* fix formatter and format

* refactor

* fix comptime error

* write tests

* perf: do not capture if string is empty
 2025-04-11 21:31:51 +00:00
Lucas Abel b1f358d4a2 backend: finalize v2 migration (v2 phase 4 - final) (#5155) 
* backend: finalize v2 migration (v2 phase 4 - final)

* update migration

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
 2025-02-13 17:10:30 +01:00
Lucas Abel 4849af04de backend: fix workflow as code (#5239) 
* backend: improve `/get_job_update` after v2

* backend: insert missing `workflow_as_code_status` on completion

also insert `flow_status` from so we can remove the query when `_duration` is above 500

* backend: fix workflow_as_code after v2

* backend: add `workflow_as_code` worker test
 2025-02-07 12:27:43 +01:00
Lucas Abel d51c3080f7 backend: use v2 tables through views where possible (v2 phase 3) (#5119) 2025-02-06 12:43:24 +01:00
Lucas Abel 3e8e0c653e backend: migrate job tables to v2 schema (v2 phase 1) (#5084) 2025-02-06 12:37:11 +01:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866) 
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
 2024-12-16 17:10:16 +01:00
HugoCasa 6924370f11 refactor sql result formatting (#4760) 2024-11-20 14:53:39 +01:00
HugoCasa 7f2289d4d5 feat: flow versioning (#4009) 
* feat: flow versioning

* fix: sqlx

* fix: update schedule test for flow versioning

* fix: with_deployment_msg + UI nits

* fix: nit

* fix: improve down migration

* patch: keep latest flow version in flow table for backward compat

* fix: app deployments in list view

* chore: update ee ref

* fix: merge

* fix: tests
 2024-07-03 15:53:45 +02:00
HugoCasa 35166883e8 feat: bun absolute/relative imports + tests (#2286) 
* feat: bun relative imports + tests

* fix: add missing .sqlx
 2023-09-15 13:19:38 +02:00