Compare commits

...
Author SHA1 Message Date
Ruben Fiszelandrubenfiszel fafa809670 chore(main): release 1.649.0 (#8198)
* chore(main): release 1.649.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-04 07:14:00 +00:00
hugocasaandClaude Opus 4.6 c97d8b4715 feat(frontend): add script recorder for offline replay (#8200)
* feat(frontend): add script recorder for offline replay of script test executions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): use Video icon for recording instead of Circle

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): use Disc icon for recording

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(frontend): improve script recorder replay and recording privacy

- Record schema at capture time in ScriptRecording (lockfile unavailable for previews)
- Read schema from recording instead of job object in replay view
- Remove lockfile tab (not available via normal job API for preview jobs)
- Use text-xs for code/schema views, remove max-height limits
- Disable log download button in replay (endpoint won't work without real job)
- Truncate UUIDs in downloaded recordings (last 8 chars) for privacy
- Make activeReplay a $state so $derived(isReplay) in FlowStatusViewerInner
  updates reactively, preventing stale reads that caused API calls during replay
- Use JSON round-trip instead of structuredClone to unwrap $state proxies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 18:41:11 +00:00
wendrul f6ceb2e366 Remove edit in fork button for app.windmill.dev (#8213)
* Remove edit in fork button for app.windmill.dev

* remove duplicate import
2026-03-03 18:39:24 +00:00
Ruben Fiszel ef7b2ec81c sqlx 2026-03-03 16:48:40 +00:00
Ruben FiszelandClaude Opus 4.6 ee01acd9a6 feat: move index management out of /srch/, add storage size reporting (#8169)
* feat: move index management endpoints out of /srch/, add storage size reporting

- Mount management_service() at /api/indexer (authenticated)
- Add management_service() OSS stub in indexer_oss.rs
- Update OpenAPI: /indexer/delete/{idx_name} and /indexer/storage
- Show disk + S3 storage sizes in IndexerMemorySettings UI

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add index storage section with refresh button

Move storage sizes into a dedicated "Index storage" section with a
refresh button to reload sizes after clearing an index.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add indexer status endpoint with liveness detection and improve settings UI

Add GET /indexer/status endpoint that combines lock-based liveness
detection with storage sizes. Frontend now shows running/stopped
indicators with last-active timestamps for each indexer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* update ee ref

* fix

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 16:46:10 +00:00
Ruben Fiszel 7b6f1deeb1 update ee ref 2026-03-03 16:25:05 +00:00
Henri Courdent f331e1f0ad Error frontend links (#8210) 2026-03-03 16:11:54 +00:00
centdix aafe716823 chore: add env config for wmdev (#8209)
* add wmdev startup envs

* name
2026-03-03 15:25:34 +00:00
GuilhemandClaude Opus 4.5 e97da86067 fix(frontend): prevent subflow expansion from hiding all insertion points (#8203)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-03 13:26:29 +00:00
Ruben FiszelandClaude Opus 4.6 26f4f2b399 fix: clean up slow-load toast interval on component destroy (#8207)
The slowStreamIntervalId (which fires "Loading is taking a long time..."
toasts every 15s) was not cleared in onDestroy, causing it to keep
firing after navigating away from the runs page.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 13:25:17 +00:00
Ruben FiszelandClaude Opus 4.6 cac4bdd54f fix: gracefully handle malformed OAuth entries in instance config (#8205)
When an OAuth provider entry in instance settings has unexpected types
(e.g. `"true"` instead of `true` for req_body_auth), the entire
/api/settings/instance_config endpoint would fail with a deserialization
error, preventing access to any instance settings.

Introduce OAuthClientEntry enum that tries typed OAuthClient
deserialization first and falls back to raw JSON, logging the
deserialization error. This allows the settings page to load even when
individual OAuth entries are malformed.

Also show a user-visible error toast in SaveButton on save failure
instead of only logging to console.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 12:37:37 +00:00
Ruben FiszelandClaude Opus 4.6 4a14e9436e prevent async lock gen race condition in mixed case path tests (#8202)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:19:16 +00:00
Ruben FiszelandClaude Opus 4.6 e6f7775d4d fix: skip stop_after_if evaluation for skipped (identity) flow steps (#8201)
* fix: skip stop_after_if evaluation for skipped (identity) flow steps

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: relax is_identity_job guard to only require skip_if

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 07:00:27 +00:00
Ruben Fiszel c5b440e569 cli tests nit 2026-03-03 06:09:25 +00:00
Ruben FiszelandClaude Opus 4.6 2b2be38f12 fix: use exact matching for python requirements directive parsing (#8199)
* fix: use exact matching for python requirements directive parsing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: apply same exact matching fix to CLI parser

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:21:28 +00:00
Ruben FiszelandClaude Opus 4.6 50defdded1 perf: use two-step query in input history to leverage v2_job index (#8197)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 21:05:35 +00:00
Ruben FiszelandClaude Opus 4.5 759eb68a7f use polling loop in schedule integration tests to avoid CI flakes (#8196)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 18:49:32 +00:00
Ruben Fiszel 3e6b1bee59 sqlx 2026-03-02 18:07:30 +00:00
lubu0 f412fbc3b7 add top-level get_job wrapper function (#8192) 2026-03-02 18:01:56 +00:00
Diego Imbert cf3ddce68a Fix data tables not working with non-secret pg variables (#8195) 2026-03-02 18:01:18 +00:00
Ruben Fiszelandrubenfiszel e906818982 chore(main): release 1.648.0 (#8182)
* chore(main): release 1.648.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 16:09:50 +00:00
18552046c2 feat: add right-click context menu to ObjectViewer (#8181)
* feat: add right-click context menu to ObjectViewer

Add a contextual menu to ObjectViewer.svelte that appears on right-click
with three actions:
- Copy value: copies the field's value to clipboard
- Copy object key: copies the property key name
- Copy entire object: copies the parent object as JSON

Uses setContext/getContext to share the context menu handler across
recursive ObjectViewer instances, rendering a single menu at the root
level via Portal. Reuses existing contextMenuStyles for visual consistency.

Closes #8177

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* Fix popover closing

* Use existing ContextMenuItem patterns

* hover style

* close contextmenu on pointerdown outside

* try catch for circular objects

* Fix copying undefined not working

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diego@windmill.dev>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
2026-03-02 16:03:06 +00:00
hugocasaandClaude Opus 4.5 a111653c6d fix: don't insert underscore after digit in PascalCase to snake_case conversion (#8184)
* fix: don't insert underscore after digit in PascalCase to snake_case conversion (#7934)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* update parsers

* remove unused wasms + fix build

* update cli lock

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-02 16:02:48 +00:00
centdixandClaude Opus 4.6 e0d4a4b38e chore(workmux): add name field to config (#8186)
* chore(workmux): add name field to config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Update .workmux.yaml

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:02:06 +00:00
Ruben FiszelandClaude Opus 4.6 9e92445fae fix: preserve debouncing settings for post-preprocessing arg accumulation (#8191)
* fix: preserve debouncing settings for post-preprocessing arg accumulation

After preprocessing completes, store the flow's debouncing settings in
runnable_settings_handle on v2_job_queue so that maybe_apply_debouncing
can find them when the surviving job is pulled. Without this, the handle
is NULL and arg accumulation silently does nothing for flows with
preprocessors.

Also adds a debouncing badge in flow settings and 4 focused accumulation
tests covering scripts, flows without preprocessor, flows with
preprocessor (with and without the fix).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx prepared query for worker_flow.rs change

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 16:01:36 +00:00
Ruben Fiszel 5faeae9486 nit copy license key on workmux creation 2026-03-02 15:21:45 +00:00
Ruben FiszelandClaude Opus 4.6 cfd9541ab1 fix(frontend): preserve keycloak realm url between instance settings saves (#8189)
* fix(frontend): preserve keycloak realm url between instance settings saves

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(backend): preserve provider-specific oauth fields through round-trip

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 14:26:34 +00:00
centdixandClaude Opus 4.6 b121f4388b docs: move autonomous-mode reference to system prompt (#8173)
* docs: move autonomous-mode reference from CLAUDE.md to system prompt

Remove the autonomous-mode.md bullet from CLAUDE.md and instead reference
it via the workmux system prompt, matching the workmux-web pattern. Also
remove the duplicated "Dev Environment (tmux)" section from
autonomous-mode.md since that info is already in the system prompt.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add autonomous-mode.md reference to wmdev sandbox system prompt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:43:19 +00:00
HugoCasa 5ebaa43aa1 internal(workmux): allow cloning main db using WM_CLONE_DB or --clone-db 2026-03-02 11:18:36 +01:00
GuilhemandClaude Opus 4.6 7a5e487878 feat(frontend): add drag-and-drop node movement in flow editor (#8076)
* feat: add drag-and-drop node movement in flow editor

Replace the 2-step click-based move with drag-and-drop: grab a node's
Move icon, drag it near an insert point, see a visual drop indicator,
and drop to move. Click-based move is preserved as fallback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: hide insert buttons on edges during drag-and-drop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: unify drop zone and legacy move target styles

Use consistent dot indicator for both drag-and-drop and click-based
move targets. Use text-accent theming, hide insert buttons during drag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: render real SvelteFlow graph in drag ghost for subflows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: center drag ghost on the dragged node instead of the whole subflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: pass isSubflow prop through drag system and improve move UX

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: fade entire subflow during legacy move and drag-and-drop

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: use text-secondary for move and drop target indicators

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: improve drag-and-drop visual feedback with proximity cues

Ghost opacity reacts to drop zone proximity (dims when far, brightens
when near). Add move icon badge near cursor that highlights on valid
drop target. Switch hit detection from circular radius to axis-aligned
bounding box matching the node gap dimensions.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: unify DragGhost to always use MiniFlowGraph

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* feat: scale drag ghost using flow viewport zoom instead of fixed width

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: register drop zone positions from BaseEdge instead of recomputing from node data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: hide node UI clutter during drag and polish drag ghost

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fade all deeply nested nodes when dragging a subflow

Previously only immediate children of a dragged subflow would fade —
deeply nested nodes (e.g. steps inside a forloop inside a branchall)
stayed at full opacity. Store the full set of dragged node IDs on
DragManager and check set membership instead of single-parent comparison.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: rename DragManager to MoveManager and eliminate moving prop drilling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: unify subflow node computation for both move modes

Extract getSubflowNodeIds() to moveManager.svelte.ts and populate
draggedNodeIds via a single $effect in DragCoordinator for both legacy
click-to-move and drag-and-drop. Consumers (MapItem, NodeWrapper) now
only check draggedNodeIds set membership instead of dual-checking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: clean up drag-and-drop code review issues

Fix toggle risk in DragCoordinator by using forceSetMoving instead of
the toggle-based setMoving. Remove dead code (DragInfo unused fields,
parentSubflowId, GHOST_ZOOM_FACTOR, debug log), extract duplicated
expressions to $derived variables, and add missing type annotations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clear click-to-move when drag starts to prevent dual mode activation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: centralize draggedNodeIds cleanup in $effect

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: adjust insertion index when moving node forward in same array

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: address PR review feedback for node move feature

- Snapshot drag ghost once at drag start using untrack() to avoid
  recomputing on every nodes/edges change during drag
- Rename setMoving/forceSetMoving to toggleMoving/setMoving for clarity
- Add capture: true to DragCoordinator's Escape handler for consistency
- Rename MOVE_BTN_OFFSET to DRAG_HANDLE_OFFSET with descriptive comment
- Move misplaced import to top of moveManager.svelte.ts
- Replace (n.data as any).offset with typed nodeOffset() helper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: register asset/AI node types in MiniFlowGraph for drag ghost

MiniFlowGraph was missing asset, assetsOverflowed, aiTool, and
newAiTool node types, so these nodes rendered as invisible elements
that inflated the drag ghost bounding box. Register them so the
ghost renders all node types correctly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve relative positions to absolute for xyflow child nodes in drag ghost

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use initialViewport instead of fitView so drag ghost matches flow zoom

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: format BaseEdge.svelte

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fade asset and AI tool nodes when their parent is being moved

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: include child nodes of edge-matched nodes in subflow ID collection

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: hide +Tool button when moving nodes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: address PR review feedback (listener cleanup, set iteration, dead code)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* style: position cancel move button on top of node instead of above it

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: compute draggedNodeIds eagerly via callback instead of reactive effect

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove redundant parentModuleId from NodeWrapper

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: address PR review comments for drag ghost and move manager

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 10:06:23 +00:00
Ruben Fiszelandrubenfiszel cfc8ab5b2d chore(main): release 1.647.2 (#8180)
* chore(main): release 1.647.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 09:45:06 +00:00
Ruben FiszelandClaude Opus 4.6 758b35f8eb fix: update oracle instant client arm64 download url (#8179)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:40:00 +00:00
Ruben FiszelandClaude Opus 4.6 b34ba965c1 chore: bump Bun to v1.3.10 (#8178)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 09:39:43 +00:00
Ruben Fiszelandrubenfiszel 889c98b38b chore(main): release 1.647.1 (#8171)
* chore(main): release 1.647.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-02 07:40:38 +00:00
Ruben FiszelandClaude Opus 4.6 db44b8be74 fix: add missing display_name and tenant fields to instance config OAuthClient (#8176)
* fix: add missing grant_types field to instance config OAuth structs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing display_name and tenant fields to instance config OAuthClient

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:35:40 +00:00
Ruben FiszelandClaude Opus 4.6 fca94f88dd fix: add missing grant_types field to instance config OAuth structs (#8175)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 07:30:36 +00:00
Ruben FiszelandClaude Opus 4.5 c70307d3f2 fix: show sync endpoint timeout setting on all instances (#8170)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-01 23:21:06 +00:00
centdixandClaude Opus 4.6 89f835727b chore: use Nix profiles in sandbox Docker image (#8140)
* feat: use Nix profiles in sandbox Docker image

Replace manual tool installs (rustup, nodesource, curl installers) in
sandbox-image/Dockerfile.sandbox with a single `nix profile install .#sandbox`.
All tools (Rust, Node, Bun, Deno, Go, gh, sqlx-cli, cargo-watch, Chromium,
Playwright, etc.) are now managed declaratively via flake.nix.

- Add `packages.sandbox` and `packages.sandbox-full` buildEnv outputs to flake.nix
- Add `sandbox-env` helper script for browser tooling env vars
- Update playwrightWrapper to export PLAYWRIGHT_BROWSERS_PATH
- Rewrite Dockerfile.sandbox: Nix replaces ~50 lines of manual installs
- Update entrypoint.sh to source Nix profile PATH
- Delete deprecated root Dockerfile.sandbox

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sandbox image runs as non-root user with wmdev

- Rewrite entrypoint.sh to start PostgreSQL as current user (no
  chown/su needed), fixing "Operation not permitted" when wmdev
  runs containers with --user
- Add chmod -R 777 /root and passwd entry for UID 1000 so non-root
  containers can access bashrc, nix-profile, and tool configs
- Remove apt postgresql server (Nix profile provides it)
- Fix bash history expansion errors from literal `!` in system prompt
- Fix asciinema path reference (available on PATH, not hardcoded)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: wrap pkg-config in sandbox profiles to bake in Nix search path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add openssh-client and sudo to sandbox image for full root access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use useradd instead of manual passwd entry for sandbox agent user

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 17:42:33 +00:00
Ruben FiszelandClaude Opus 4.6 6eca08480a chore: remove legacy wmill_pg python client (#8155)
The wmill_pg package (psycopg2 wrapper for running PostgreSQL queries)
has been fully replaced by Windmill's native PostgreSQL support.
Remove the package directory and all references from build, publish,
install, version, LSP, and dependabot configs.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 08:52:35 +00:00
Ruben Fiszelandrubenfiszel 36353359f6 chore(main): release 1.647.0 (#8127)
* chore(main): release 1.647.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-03-01 07:00:37 +00:00
Fred Reimer 7d6f4fdabb chore: bump Go in container images to 1.26.0 (#8135) 2026-03-01 06:53:33 +00:00
Ruben FiszelandClaude Opus 4.6 7a32abec96 feat: slow stream warnings, batch size control, and fix result/skipped filters (#8154)
- Show recurring toast every 15s (8s duration) when loading takes long, with stop button
- When streaming by batches of 25 and a batch takes >4s, offer to stream 1 by 1
- Expose batch size in progress bar with editable input to customize on the fly
- Make stop button more prominent (destructive Button component)
- Fix list_jobs UNION: exclude queue jobs when filtering by result or is_skipped=true
- Add "Show skipped" preset to runs filter

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-01 06:39:24 +00:00
Ruben FiszelandClaude Opus 4.6 4f5a804091 perf: batch large job list requests and fix loadExtraJobs cursor (#8151)
* perf: batch large job list requests and fix loadExtraJobs cursor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: replace timeout toast with batch progress banner for large job lists

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: show loading indicator on Load more buttons

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: distinguish load-more vs auto-refresh loading indicators

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: offer to stream by batches of 25 when loading is slow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove refreshing text on auto-refresh and clean up unused loading prop

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: batch progress race condition when restreaming with small batches

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 22:57:06 +00:00
Ruben Fiszel faf190f12d fix: sync flow on_behalf_of_email on load (#8149) 2026-02-28 22:30:15 +00:00
Ruben FiszelandClaude Opus 4.5 86182ed2e9 fix: validate tarball URL host against registry to prevent SSRF and token exfiltration (#8153)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-28 22:03:19 +00:00
Ruben Fiszel 7f6e9fec0c bun-types 2026-02-28 21:47:41 +00:00
Ruben FiszelandClaude Opus 4.6 13daebf88a fix: restore email domain (MX) setting in instance settings UI (#8152)
The email_domain setting was accidentally removed from the frontend
instance settings in a recent onboarding cleanup. The backend still
fully supports it. This restores the setting in the Core section.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 21:46:01 +00:00
Ruben Fiszel c98db016b6 nit claude settings 2026-02-28 21:39:15 +00:00
Ruben FiszelandClaude Opus 4.6 d4673c2e91 fix: add partial index for fast failure filtering on runs page (#8150)
When failures are sparse (<1%), filtering by failure status on the runs
page required scanning millions of success rows. Add a partial index on
v2_job_completed (workspace_id, completed_at DESC) WHERE status IN
('failure', 'canceled') and switch ORDER BY to completed_at when
filtering failures, so Postgres walks the small partial index directly.

Benchmarked at 5.2M rows / 1% failure rate:
- LIMIT 30:   800ms -> 0.4ms (2000x faster)
- LIMIT 1000: 550ms -> 21ms  (26x faster)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 18:18:34 +00:00
Ruben Fiszel 59e51ac097 nit workmux cli 2026-02-28 18:06:39 +00:00
Ruben FiszelandClaude Opus 4.6 278983c4fd fix: process deletes before adds in CLI sync push to avoid conflicts (#8148)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 18:05:33 +00:00
Ruben Fiszel d933446a9e .npmrc nit 2026-02-28 09:16:55 +00:00
Ruben FiszelandClaude Opus 4.6 ba48d70157 perf: lazy-load heavy deps (graphql, openapi-parser, sha256) (#8145)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:11:00 +00:00
Ruben FiszelandClaude Opus 4.6 cd2cf0c39e copy .npmrc in Dockerfiles so npm ci resolves legacy-peer-deps (#8146)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 09:08:03 +00:00
Ruben FiszelandClaude Opus 4.6 bd9ff03010 perf: lazy-load markdown in Tooltip components (#8143)
* perf: lazy-load markdown in Tooltip to reduce stores2 chunk by 335KB

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: migrate TooltipInner to Svelte 5 runes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* perf: remove markdown rendering from Tooltip components

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use HTML tables for date format tooltips to preserve formatting

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 08:52:10 +00:00
Ruben FiszelandClaude Opus 4.6 c424b1a961 chore: update vite to 8, vite-plugin-svelte to 7 (#8141)
* chore: update vite to 8 beta, vite-plugin-svelte to 7, vitest to 4.1 beta

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add .npmrc with legacy-peer-deps for vite 8 beta

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:50:31 +00:00
Ruben FiszelandClaude Opus 4.6 0776de6b21 fix: copy deps and remove user auto-add on workspace fork (#8142)
* fix: copy deps and remove user auto-add on workspace fork

Clone workspace_dependencies to forked workspaces and remove
automatic workspace_invite creation for parent workspace users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update sqlx offline cache

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:39:27 +00:00
Diego ImbertandRuben Fiszel 762fd3d993 Fix python datatable client requiring explicit types (#8086)
* Support arg type decl in postgres

* Python datatable client no longer requires explicit arg typing

* compilation fix

* Set correct type in statement exec

* reset to main

* Explicit pg arg types

* remove code duplication

* update parser js

* FLOAT8 doesn't have space

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-28 07:08:02 +00:00
83aee49978 add Google triggers doc link in workspace native triggers settings (#8091)
Add docsUrl to the Google service config in WorkspaceIntegrations so a
"Docs" button appears next to the Google integration, linking to
https://www.windmill.dev/docs/core_concepts/native_triggers#google-triggers.
This follows the same pattern already used for Nextcloud.

Closes #8090

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Henri Courdent <122811744+hcourdent@users.noreply.github.com>
2026-02-28 07:04:40 +00:00
Diego Imbert 095505136c fix: Handle CTEs and local tables in SQL asset parser (#8131)
* Handle CTEs and local tables in SQL asset parser

* also handle CREATE VIEW

* Update package regex version
2026-02-28 07:04:19 +00:00
claude[bot] 257734b9ab prevent dropdown from switching to top when less space is available above (#8126) 2026-02-28 07:03:44 +00:00
hugocasaandClaude Opus 4.6 5d58a87a7f feat: populate baseUrl and userId in Nextcloud resource from OAuth (#8132)
When connecting Nextcloud via workspace integration OAuth, the resource
now includes baseUrl (from OAuth config) and userId (fetched from
Nextcloud OCS API) alongside the token, making it immediately usable
by scripts. Falls back to token + baseUrl if user info fetch fails.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:03:15 +00:00
Diego Imbert b68ff965dd fix: fix custom TS Monaco worker not reloading on file uri change (#8130) 2026-02-28 07:01:23 +00:00
centdixandClaude Opus 4.6 ff180de4de refactor: slim down claude instructions for lean context and fast iteration (#8136)
* refactor: slim down claude instructions for lean context and fast iteration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add private and license feature flags to enterprise validation docs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add /refine skill for end-of-session doc evolution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove architecture.md overview doc per research findings

General codebase overviews distract agents and trigger unnecessary
exploration. Keep only operational docs (validation, enterprise).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add autonomous mode doc for bypass permission workflows

Covers: plan-first requirement, tmux pane usage for checking
backend/frontend logs, manual testing via Playwright MCP,
Playwright gotchas, and end-of-task summary expectations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add mermaid, playwright, and asciinema tools to autonomous mode doc

Claude should use mmdc for diagrams during planning, playwright CLI for
screenshots of frontend changes, and asciinema for terminal recordings
of CLI changes. All attached to the PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use pastebin for screenshot/recording uploads

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review findings

- Remove stale docs/architecture.md reference from /refine skill
- Fix script name: ./update-sqlx -> ./update_sqlx.sh
- Remove .claude/settings.local.json mention from enterprise doc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 07:00:19 +00:00
centdixandClaude Opus 4.6 7728475fc9 refactor: rewrite flake.nix for clarity and modularity (#8137)
* refactor: rewrite flake.nix from scratch for clarity and modularity

Rewrite the Nix flake with clean separation of concerns, organized
let-bindings, and 4 purpose-specific devShells instead of a monolithic
default shell with broken package outputs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add CLI tools to default devShell (gh, aws, playwright, mermaid, asciinema)

Add tools needed for AI agent workflows and dev tooling:
- gh (GitHub CLI)
- awscli2
- asciinema (terminal recording)
- playwright-driver with Nix-managed browsers
- mermaid-cli (diagram generation)

Playwright browsers are provided via nixpkgs' playwright-driver.browsers.
Mermaid/Puppeteer reuses the headless_shell from the same browser set.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move wm-minio scripts to default devShell

MinIO (local S3) is needed for regular development, not just the full
profile.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use playwright wrapper + chromium for browser tools

Replace playwright-driver (library, no CLI) with:
- A `playwright` wrapper script that calls the Nix playwright-core CLI
  (version-matched to its own Nix-provided browsers)
- pkgs.chromium for Mermaid/Puppeteer (which respects PUPPETEER_EXECUTABLE_PATH)

This fixes playwright screenshot and mermaid diagram generation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: auto-load .env.local from main worktree in all devShells

Gitignored files like .env.local don't exist in git worktrees.
Add a shared shellHook that resolves back to the main tree via
git-common-dir and sources .env.local if present. This ensures
AWS credentials and other secrets are available in worktrees.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace deprecated pkgs.hostPlatform with stdenv.hostPlatform

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove AWS CLI from flake and sandbox images

Pastebin is sufficient for screenshot sharing; AWS credentials
add unnecessary complexity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review — ruby mismatch, quoting, shell dedup

- Fix pkgs.ruby → pkgs.ruby_3_4 in extraRuntimeVars to match extraRuntimes
- Replace $* with "$@" in all helper scripts (wm, wm-build, wm-caddy,
  wm-bench, wm-cli) to correctly preserve argument boundaries
- Extract coreBuildInputs, browserVars, and playwrightWrapper as shared
  let-bindings to eliminate duplication between default and full shells

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove .env.local auto-loading from devShells

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-28 06:51:02 +00:00
7d9d16a6a3 feat: runScript inline for path and hash (#8019)
* runScript inline for path and hash

* Update backend/windmill-api/src/jobs.rs

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* refactor: unify inline script param structs and deduplicate closures

- Replace RunInlineScriptByPathFnParams and RunInlineScriptByHashFnParams
  with a single RunInlineScriptFnParams using InlineScriptTarget enum
- Collapse two nearly-identical closures in worker.rs into one
- Merge duplicate InlineByPath/InlineByHash into InlineScriptArgs
- Extract shared run_inline_script_inner helper in API handler
- Add missing check_scopes to run_inline_script_by_hash endpoint
- Fix duplicate lines from prior commit in run_inline_script_by_path
- Change tag from "inline_preview" to "inline" for deployed scripts

Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>

* Integration tests

* rm

* rename feature to run_inline

* Run inline integration tests

* Fix tests

* check path scope

* openapi fix

* nits

* remove register_potential_assets_on_inline_execution

* unused variable

* refactor

* Pass user_db to check script permission

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Diego Imbert <diegoimbert@users.noreply.github.com>
2026-02-27 13:59:14 +01:00
HugoCasaandClaude Opus 4.6 cdc0543747 fix: remove review comments from discord notifications and support comment edits
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-27 13:32:12 +01:00
HugoCasaandClaude Opus 4.5 b9e3e053e4 fix: prevent wm-cursor from hanging on stale cursor IPC sockets
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-27 13:09:30 +01:00
HugoCasa 3a552c5b95 nit button text override slack interactive approval request 2026-02-27 09:59:46 +01:00
c8d99d7fc9 replace SELECT * with explicit columns in teams command query (#8129)
* fix: replace SELECT * with explicit columns in teams command query

- Update sqlx offline cache for the changed query
- Fix write_latest_ee_ref.sh to prefer matching EE worktree branch
- Update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 8ffae1f43b31dc8136714fa612d22b6301773e27

This commit updates the EE repository reference after PR #434 was merged in windmill-ee-private.

Previous ee-repo-ref: da1f8bf8676f85cac2b6fa2705246e1819d4b6f0

New ee-repo-ref: 8ffae1f43b31dc8136714fa612d22b6301773e27

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-27 06:54:33 +00:00
f1d8568831 fix:: persist show schedules and show future jobs toggles in local storage (#8125)
Store the runs page 'show schedules' and 'show future jobs' filter toggles
in local storage using useLocalStorageValue so they persist across page
navigations. URL parameters remain dominant - local storage values are only
applied when URL params are undefined.

Closes #8123

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-02-27 06:24:49 +00:00
Ruben Fiszelandrubenfiszel ef84ce24ab chore(main): release 1.646.0 (#8116)
* chore(main): release 1.646.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2026-02-26 18:42:50 +00:00
Ruben FiszelandClaude Opus 4.6 99c01bca38 fix: remove duplicate job loading on chart zoom (#8121)
- Remove explicit loadJobs(true) calls from onZoom handlers in RunChart
  and ConcurrentJobsChart — setting _timeframe.val already triggers the
  effect which calls onParamChanges
- Fix debounce by hoisting promise ref outside effect closure so cleanup
  can properly cancel in-flight requests

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 18:32:36 +00:00
247 changed files with 6432 additions and 7899 deletions
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
# PreToolUse hook: block destructive git operations when on the main branch.
# Non-git tool calls and read-only git commands pass through silently.
set -euo pipefail
input="$(cat)"
tool_name="$(echo "$input" | jq -r '.tool_name // empty')"
# Only care about Bash tool calls
[[ "$tool_name" == "Bash" ]] || exit 0
command="$(echo "$input" | jq -r '.tool_input.command // empty')"
# Only care about git write commands
if [[ "$command" =~ ^git\ (push|reset|revert|checkout|merge|rebase|commit|add) ]]; then
branch="$(git rev-parse --abbrev-ref HEAD 2>/dev/null || true)"
if [[ "$branch" == "main" ]]; then
echo "BLOCK: You are on the main branch. Create or switch to a feature branch first."
fi
fi
+22 -8
View File
@@ -30,7 +30,15 @@
"Bash(cargo check:*)",
"mcp__ide__getDiagnostics",
"Bash(npm run generate-backend-client:*)",
"Bash(npm run check:*)"
"Bash(npm run check:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)",
"Bash(git add:*)",
"Bash(git commit:*)"
],
"deny": [
"Read(.env)",
@@ -55,17 +63,23 @@
"Bash(chown:*)",
"Bash(truncate:*)",
"Bash(shred:*)",
"Bash(unlink:*)",
"Bash(git push:*)",
"Bash(git reset:*)",
"Bash(git revert:*)",
"Bash(git checkout:*)",
"Bash(git merge:*)",
"Bash(git rebase:*)"
"Bash(unlink:*)"
]
},
"enableAllProjectMcpServers": true,
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/.claude/hooks/guard-main-branch.sh",
"timeout": 5
}
]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write",
+39
View File
@@ -0,0 +1,39 @@
---
name: refine
user_invocable: true
description: End-of-session reflection. Reviews friction encountered during the session and proposes updates to docs/ to capture lessons learned.
---
# Refine Skill
Reflect on the current session and update documentation with lessons learned.
## Instructions
1. **Identify friction**: Review what happened in this session:
- Run `git diff main...HEAD --stat` to see what files were touched
- Think about: what was slow, what failed, what required multiple attempts, what information was missing or hard to find
2. **Read current docs**: Read the docs that were relevant to this session:
- `docs/validation.md`
- `docs/enterprise.md`
- `docs/autonomous-mode.md`
- Any skills that were invoked
3. **Propose updates**: For each piece of friction, decide if it warrants a doc update:
- **Missing knowledge**: Information you had to discover that should be documented
- **Wrong guidance**: Instructions that led you astray
- **Missing validation rule**: A check that should be in the validation matrix
- **New pattern**: A codebase pattern worth capturing for next time
4. **Apply updates**: Edit the relevant `docs/` files. Keep changes minimal and specific — add only what would have saved time this session.
5. **Report**: Summarize what was added/changed and why.
## Rules
- Only add knowledge confirmed by this session — no speculative additions
- Keep docs concise — add a line or two, not a paragraph
- If a whole new doc is needed, create it in `docs/` and add a pointer in `CLAUDE.md`
- Don't update skills unless a coding pattern was genuinely wrong
- Don't add things Claude already knows — only Windmill-specific knowledge
+49 -437
View File
@@ -3,493 +3,105 @@ name: rust-backend
description: Rust coding guidelines for the Windmill backend. MUST use when writing or modifying Rust code in the backend directory.
---
# Rust Backend Coding Guidelines
# Windmill Rust Patterns
Apply these patterns when writing or modifying Rust code in the `backend/` directory.
## Data Structure Design
Choose between `struct`, `enum`, or `newtype` based on domain needs:
- Use `enum` for state machines instead of boolean flags or loosely related fields
- Model invariants explicitly using types (e.g., `NonZeroU32`, `Duration`, custom enums)
- Consider ownership of each field:
- Use `&str` vs `String`, slices vs vectors
- Use `Arc<T>` when sharing across threads
- Use `Cow<'a, T>` for flexible ownership
```rust
// State machine with enum
enum JobState {
Pending { scheduled_for: DateTime<Utc> },
Running { started_at: DateTime<Utc>, worker: String },
Completed { result: JobResult, duration_ms: i64 },
Failed { error: String, retries: u32 },
}
// Avoid multiple booleans
struct Job {
is_pending: bool, // Don't do this
is_running: bool,
is_completed: bool,
}
```
## Impl Block Organization
Place `impl` blocks immediately below the struct/enum they modify. Group methods logically:
```rust
struct JobQueue {
jobs: Vec<Job>,
capacity: usize,
}
impl JobQueue {
// Constructors first
pub fn new(capacity: usize) -> Self { ... }
pub fn with_jobs(jobs: Vec<Job>) -> Self { ... }
// Getters
pub fn len(&self) -> usize { ... }
pub fn is_empty(&self) -> bool { ... }
// Mutation methods
pub fn push(&mut self, job: Job) -> Result<()> { ... }
pub fn pop(&mut self) -> Option<Job> { ... }
// Domain logic
pub fn next_scheduled(&self) -> Option<&Job> { ... }
}
```
## Iterator Chains Over For-Loops
Prefer functional iterator chains (`.filter().map().collect()`) over imperative for-loops:
```rust
// Preferred
let results: Vec<_> = items
.iter()
.filter(|item| item.is_valid())
.map(|item| item.transform())
.collect();
// Avoid
let mut results = Vec::new();
for item in items.iter() {
if item.is_valid() {
results.push(item.transform());
}
}
```
Apply these Windmill-specific patterns when writing Rust code in `backend/`.
## Error Handling
Use the `Error` type from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>` for fallible functions:
Use `Error` from `windmill_common::error`. Return `Result<T, Error>` or `JsonResult<T>`:
```rust
use windmill_common::error::{Error, Result};
// Use ? operator for propagation
pub async fn get_job(db: &DB, id: Uuid) -> Result<Job> {
let job = sqlx::query_as!(Job, "SELECT ... WHERE id = $1", id)
sqlx::query_as!(Job, "SELECT id, workspace_id FROM v2_job WHERE id = $1", id)
.fetch_optional(db)
.await?
.ok_or_else(|| Error::NotFound("job not found".to_string()))?;
Ok(job)
}
```
Prefer `if let` for optional handling. Use `let...else` when early return makes code clearer:
Never panic in library code. Reserve `.unwrap()` for compile-time guarantees.
## SQLx Patterns
**Never use `SELECT *`** — always list columns explicitly. Critical for backwards compatibility when workers lag behind API version:
```rust
let Some(config) = get_config() else {
return Err(Error::MissingConfig);
};
// Correct
sqlx::query_as!(Job, "SELECT id, workspace_id, path FROM v2_job WHERE id = $1", id)
// Wrong — breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", id)
```
Never panic in library code. Reserve `.unwrap()` for cases with compile-time guarantees. Keep functions short to help lifetime inference and clarity.
## Early Returns
Return early to avoid deep nesting. Handle error cases and edge conditions first:
Use batch operations to avoid N+1:
```rust
// Preferred - early returns
fn process_job(job: Option<Job>) -> Result<Output> {
let Some(job) = job else {
return Ok(Output::default());
};
if !job.is_valid() {
return Err(Error::InvalidJob);
}
if job.is_cached() {
return Ok(job.cached_result());
}
// Main logic at the end, not nested
execute_job(job)
}
// Avoid - deep nesting
fn process_job(job: Option<Job>) -> Result<Output> {
if let Some(job) = job {
if job.is_valid() {
if !job.is_cached() {
execute_job(job)
} else {
Ok(job.cached_result())
}
} else {
Err(Error::InvalidJob)
}
} else {
Ok(Output::default())
}
}
// Preferred — single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
## Variable Shadowing
Shadow variables instead of creating new names with prefixes:
```rust
// Preferred
let data = fetch_raw_data();
let data = parse(data);
let data = validate(data)?;
// Avoid
let raw_data = fetch_raw_data();
let parsed_data = parse(raw_data);
let validated_data = validate(parsed_data)?;
```
## Minimal Comments
- No inline comments explaining obvious code
- No TODO/FIXME comments in committed code
- Doc comments (`///`) only on public items
- Let code be self-documenting through clear naming
## Type Safety
Use enums over boolean flags for clarity:
```rust
// Preferred
enum JobStatus {
Pending,
Running,
Completed,
}
// Avoid
struct Job {
is_running: bool,
is_completed: bool,
}
```
## Pattern Matching
Prefer explicit matching. Use wildcards strategically for fallback cases or ignored fields:
```rust
// Explicit matching preferred
match status {
JobStatus::Pending => handle_pending(),
JobStatus::Running => handle_running(),
JobStatus::Completed => handle_completed(),
}
// Wildcards OK for fallback
match result {
Ok(value) => process(value),
Err(_) => return default_value(),
}
// Wildcards OK for ignoring fields in destructuring
let Point { x, y, .. } = point;
```
## Destructuring in Function Signatures
Destructure structs directly in function parameters:
```rust
// Preferred
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> {
// ...
}
// Avoid
async fn process_job(
db_ext: Extension<DB>,
path: Path<(String, Uuid)>,
query: Query<Pagination>,
) -> Result<Json<Job>> {
let Extension(db) = db_ext;
let Path((workspace, job_id)) = path;
// ...
}
```
## Trait Implementations
Use standard trait implementations to simplify conversions and reduce boilerplate:
```rust
// Implement From/Into for type conversions
impl From<DbJob> for ApiJob {
fn from(db: DbJob) -> Self {
ApiJob {
id: db.id,
status: db.status.into(),
}
}
}
// Use TryFrom for fallible conversions
impl TryFrom<String> for JobKind {
type Error = Error;
fn try_from(s: String) -> Result<Self, Self::Error> { ... }
}
```
Apply `derive` macros to reduce boilerplate:
```rust
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Job { ... }
```
## Module Structure
- Use `pub(crate)` instead of `pub` when possible; expose only what needs exposing
- Keep APIs small and expressive; avoid leaking internal types
- Organize code into modules reflecting ownership and domain boundaries
```rust
// Prefer restricted visibility
pub(crate) fn internal_helper() { ... }
// Only pub for external API
pub fn create_job(...) -> Result<Job> { ... }
```
## Code Navigation
Always use rust-analyzer LSP for:
- Go to definition
- Find references
- Type information
- Import resolution
Do not guess at module paths or type definitions.
Use transactions for multi-step operations. Parameterize all queries.
## JSON Handling
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when:
- Storing JSON in the database (JSONB columns)
- Passing JSON through without modification
- The JSON structure doesn't need inspection
Prefer `Box<serde_json::value::RawValue>` over `serde_json::Value` when storing/passing JSON without inspection:
```rust
// Preferred - avoids parsing/serialization overhead
pub struct Job {
pub id: Uuid,
pub args: Option<Box<serde_json::value::RawValue>>,
}
// Only use Value when you need to inspect/modify JSON
let value: serde_json::Value = serde_json::from_str(&json)?;
if let Some(field) = value.get("field") {
// modify or inspect
}
```
## Serde Optimizations
Only use `serde_json::Value` when you need to inspect or modify the JSON.
Use serde attributes to optimize serialization:
## Serde Optimizations
```rust
#[derive(Serialize, Deserialize)]
pub struct Job {
#[serde(rename = "jobId")]
pub id: Uuid,
#[serde(default)]
pub priority: i32,
#[serde(skip_serializing_if = "Option::is_none")]
pub parent_job: Option<Uuid>,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub tags: Vec<String>,
#[serde(default)]
pub priority: i32,
}
```
Prefer borrowing for zero-copy deserialization when lifetimes allow:
## Async & Concurrency
Never block the async runtime. Use `spawn_blocking` for CPU-intensive work:
```rust
#[derive(Deserialize)]
pub struct JobInput<'a> {
#[serde(borrow)]
pub workspace_id: Cow<'a, str>,
#[serde(borrow)]
pub script_path: &'a str,
}
let result = tokio::task::spawn_blocking(move || expensive_computation(&data)).await?;
```
## SQLx Patterns
**Mutex selection**: Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) for data protection. Only use `tokio::sync::Mutex` when holding locks across `.await` points.
**Never use `SELECT *`** - always list columns explicitly. This is critical for backwards compatibility when workers run behind the API server version:
Use `tokio::sync::mpsc` (bounded) for channels. Avoid `std::thread::sleep` in async contexts.
## Module Structure & Visibility
- Use `pub(crate)` instead of `pub` when possible
- Place new code in the appropriate crate based on functionality
- API endpoints go in `windmill-api/src/` organized by domain
- Shared functionality goes in `windmill-common/src/`
## Code Navigation
Always use rust-analyzer LSP for go-to-definition, find-references, and type info. Do not guess at module paths.
## Axum Handlers
Destructure extractors directly in function signatures:
```rust
// Preferred - explicit columns
sqlx::query_as!(
Job,
"SELECT id, workspace_id, path, created_at FROM v2_job WHERE id = $1",
job_id
)
// Avoid - breaks when columns are added
sqlx::query_as!(Job, "SELECT * FROM v2_job WHERE id = $1", job_id)
async fn process_job(
Extension(db): Extension<DB>,
Path((workspace, job_id)): Path<(String, Uuid)>,
Query(pagination): Query<Pagination>,
) -> Result<Json<Job>> { ... }
```
Use batch operations to minimize round trips:
```rust
// Preferred - single query with multiple values
sqlx::query!(
"INSERT INTO job_logs (job_id, logs) VALUES ($1, $2), ($3, $4)",
id1, log1, id2, log2
)
// Avoid N+1 queries
for id in ids {
sqlx::query!("SELECT ... WHERE id = $1", id).fetch_one(db).await?;
}
// Preferred - single query with IN clause
sqlx::query!("SELECT ... WHERE id = ANY($1)", &ids[..]).fetch_all(db).await?
```
Use transactions for multi-step operations and parameterize all queries.
## Async & Tokio Patterns
Never block the async runtime. Use `spawn_blocking` for CPU-intensive or blocking I/O:
```rust
// Preferred - offload blocking work
let result = tokio::task::spawn_blocking(move || {
expensive_computation(&data)
}).await?;
// Avoid - blocks the runtime
let result = expensive_computation(&data); // Don't do this in async
```
Use tokio primitives for sleep and channels:
```rust
use tokio::sync::mpsc;
use tokio::time::sleep;
// Avoid in async contexts
use std::thread::sleep; // Blocks the runtime
```
Use bounded channels for backpressure:
```rust
// Preferred - bounded channel prevents overwhelming
let (tx, rx) = tokio::sync::mpsc::channel(100);
// Be careful with unbounded
let (tx, rx) = tokio::sync::mpsc::unbounded_channel();
```
## Mutex Selection in Async Code
**Prefer `std::sync::Mutex` (or `parking_lot::Mutex`) over `tokio::sync::Mutex`** for protecting data in async code. The async mutex is more expensive and only needed when holding locks across `.await` points.
```rust
// Preferred for data protection - std mutex is faster
use std::sync::Mutex;
struct Cache {
data: Mutex<HashMap<String, Value>>,
}
impl Cache {
fn get(&self, key: &str) -> Option<Value> {
self.data.lock().unwrap().get(key).cloned()
}
fn insert(&self, key: String, value: Value) {
self.data.lock().unwrap().insert(key, value);
}
}
```
**Use `tokio::sync::Mutex` only when you must hold the lock across `.await` points**, typically for IO resources like database connections:
```rust
use tokio::sync::Mutex;
use std::sync::Arc;
// Async mutex for IO resources held across await points
let conn = Arc::new(Mutex::new(db_connection));
async fn execute_query(conn: Arc<Mutex<DbConn>>, query: &str) {
let mut lock = conn.lock().await;
lock.execute(query).await; // Lock held across .await
}
```
**Common pattern**: Wrap `Arc<Mutex<...>>` in a struct with non-async methods that lock internally, keeping lock scope minimal:
```rust
struct SharedState {
inner: std::sync::Mutex<StateInner>,
}
impl SharedState {
fn update(&self, value: i32) {
self.inner.lock().unwrap().value = value;
}
fn get(&self) -> i32 {
self.inner.lock().unwrap().value
}
}
```
**Alternative for IO resources**: Spawn a dedicated task to manage the resource and communicate via message passing:
```rust
let (tx, mut rx) = tokio::sync::mpsc::channel(32);
tokio::spawn(async move {
while let Some(cmd) = rx.recv().await {
handle_io_command(&mut resource, cmd).await;
}
});
```
## Build & Tooling
Build speed tips:
- Use `cargo check` during rapid iteration over `cargo build`
- Minimize unnecessary dependencies and feature flags
+43 -281
View File
@@ -3,316 +3,78 @@ name: svelte-frontend
description: Svelte coding guidelines for the Windmill frontend. MUST use when writing or modifying code in the frontend directory.
---
# Svelte 5 Best Practices
# Windmill Svelte Patterns
This guide outlines best practices for developing with Svelte 5, incorporating the new Runes API and other modern Svelte features. These rules MUST NOT be applied on svelte 4 files unless explicitly asked to do so.
Apply these Windmill-specific patterns when writing Svelte code in `frontend/`. For general Svelte 5 syntax (runes, snippets, event handling), use the Svelte MCP server.
## Reactivity with Runes
## Windmill UI Components (MUST use)
Svelte 5 introduces Runes for more explicit and flexible reactivity.
Always use Windmill's design-system components. Never use raw HTML elements.
1. **Embrace Runes for State Management**:
* Use `$state` for reactive local component state.
```svelte
<script>
let count = $state(0);
function increment() {
count += 1;
}
</script>
<button onclick={increment}>
Clicked {count} {count === 1 ? 'time' : 'times'}
</button>
```
* Use `$derived` for computed values based on other reactive state.
```svelte
<script>
let count = $state(0);
const doubled = $derived(count * 2);
</script>
<p>{count} * 2 = {doubled}</p>
```
* Use `$effect` for side effects that need to run when reactive values change (e.g., logging, manual DOM manipulation, data fetching). Remember `$effect` does not run on the server.
```svelte
<script>
let count = $state(0);
$effect(() => {
console.log('The count is now', count);
if (count > 5) {
alert('Count is too high!');
}
});
</script>
```
2. **Props with `$props`**:
* Declare component props using `$props()`. This offers better clarity and flexibility compared to `export let`.
```svelte
<script>
// ChildComponent.svelte
let { name, age = $state(30) } = $props();
</script>
<p>Name: {name}</p>
<p>Age: {age}</p>
```
* For bindable props, use `$bindable`.
```svelte
<script>
// MyInput.svelte
let { value = $bindable() } = $props();
</script>
<input bind:value />
```
## Event Handling
* **Use direct event attributes**: Svelte 5 moves away from `on:` directives for DOM events.
* **Do**: `<button onclick={handleClick}>...</button>`
* **Don't**: `<button on:click={handleClick}>...</button>`
* **For component events, prefer callback props**: Instead of `createEventDispatcher`, pass functions as props.
```svelte
<!-- Parent.svelte -->
<script>
import Child from './Child.svelte';
let message = $state('');
function handleChildEvent(detail) {
message = detail;
}
</script>
<Child onCustomEvent={handleChildEvent} />
<p>Message from child: {message}</p>
<!-- Child.svelte -->
<script>
let { onCustomEvent } = $props();
function emitEvent() {
onCustomEvent('Hello from child!');
}
</script>
<button onclick={emitEvent}>Send Event</button>
```
## Snippets for Content Projection
* **Use `{#snippet ...}` and `{@render ...}` instead of slots**: Snippets are more powerful and flexible.
```svelte
<!-- Parent.svelte -->
<script>
import Card from './Card.svelte';
</script>
<Card>
{#snippet title()}
My Awesome Title
{/snippet}
{#snippet content()}
<p>Some interesting content here.</p>
{/snippet}
</Card>
<!-- Card.svelte -->
<script>
let { title, content } = $props();
</script>
<article>
<header>{@render title()}</header>
<div>{@render content()}</div>
</article>
```
* Default content is passed via the `children` prop (which is a snippet).
```svelte
<!-- Wrapper.svelte -->
<script>
let { children } = $props();
</script>
<div>
{@render children?.()}
</div>
```
## Component Design
1. **Create Small, Reusable Components**: Break down complex UIs into smaller, focused components. Each component should have a single responsibility. This also aids performance by limiting the scope of reactivity updates.
2. **Descriptive Naming**: Use clear and descriptive names for variables, functions, and components.
3. **Minimize Logic in Components**: Move complex business logic to utility functions or services. Keep components focused on presentation and interaction.
## State Management (Stores)
1. **Segment Stores**: Avoid a single global store. Create multiple stores, each responsible for a specific piece of global state (e.g., `userStore.js`, `themeStore.js`). This can help limit reactivity updates to only the parts of the UI that depend on specific state segments.
2. **Use Custom Stores for Complex Logic**: For stores with related methods, create custom stores.
```javascript
// counterStore.js
import { writable } from 'svelte/store';
function createCounter() {
const { subscribe, set, update } = writable(0);
return {
subscribe,
increment: () => update(n => n + 1),
decrement: () => update(n => n - 1),
reset: () => set(0)
};
}
export const counter = createCounter();
```
3. **Use Context API for Localized State**: For state shared within a component subtree, consider Svelte's context API (`setContext`, `getContext`) instead of global stores when the state doesn't need to be truly global.
## Performance Optimizations (Svelte 5)
When generating Svelte 5 code, prioritize frontend performance by applying the following principles:
### General Svelte 5 Principles
- **Leverage the Compiler:** Trust Svelte's compiler to generate optimized JavaScript. Avoid manual DOM manipulation (`document.querySelector`, etc.) unless absolutely necessary for integrating third-party libraries that lack Svelte adapters.
- **Keep Components Small and Focused:** Reinforcing from Component Design, smaller components lead to less complex reactivity graphs and more targeted, efficient updates.
### Reactivity & State Management
- **Optimize Computations with `$derived`:** Always use `$derived` for computed values that depend on other state. This ensures the computation only runs when its specific dependencies change, avoiding unnecessary work compared to recomputing derived values in `$effect` or less efficient methods.
- **Minimize `$effect` Usage:** Use `$effect` sparingly and only for true side effects that interact with the outside world or non-Svelte state. Avoid putting complex logic or state updates *within* an `$effect` unless those updates are explicitly intended as a reaction to external changes or non-Svelte state. Excessive or complex effects can impact rendering performance.
- **Structure State for Fine-Grained Updates:** Design your `$state` objects or variables such that updates affect only the necessary parts of the UI. Avoid putting too much unrelated state into a single large object that gets frequently updated, as this can potentially trigger broader updates than necessary. Consider normalizing complex, nested state.
### List Rendering (`{#each}`)
- **Mandate `key` Attribute:** Always use a `key` attribute (`{#each items as item (item.id)}`) that refers to a unique, stable identifier for each item in a list. This is critical for allowing Svelte to efficiently update, reorder, add, or remove list items without destroying and re-creating unnecessary DOM elements and component instances.
### Component Loading & Bundling
- **Implement Lazy Loading/Code Splitting:** For routes, components, or modules that are not immediately needed on page load, use dynamic imports (`import(...)`) to split the code bundle. SvelteKit handles this automatically for routes, but it can be applied manually to components using helper patterns if needed.
- **Be Mindful of Third-Party Libraries:** When incorporating external libraries, import only the necessary functions or components to minimize the final bundle size. Prefer libraries designed to be tree-shakeable.
### Rendering & DOM
- **Use CSS for Animations/Transitions:** Prefer CSS animations or transitions where possible for performance. Svelte's built-in `transition:` directive is also highly optimized and should be used for complex state-driven transitions, but simple cases can often use plain CSS.
- **Optimize Image Loading:** Implement best practices for images: use optimized formats (WebP, AVIF), lazy loading (`loading="lazy"`), and responsive images (`<picture>`, `srcset`) to avoid loading unnecessarily large images.
### Server-Side Rendering (SSR) & Hydration
- **Ensure SSR Compatibility:** Write components that can be rendered on the server for faster initial page loads. Avoid relying on browser-specific APIs (like `window` or `document`) in the main `<script>` context. If necessary, use `$effect` or check `if (browser)` inside effects to run browser-specific code only on the client.
- **Minimize Work During Hydration:** Structure components and data fetching such that minimal complex setup or computation is required when the client-side Svelte code takes over from the server-rendered HTML. Heavy synchronous work during hydration can block the main thread.
## General Clean Code Practices
1. **Organized File Structure**: Group related files together. A common structure:
```
/src
|-- /routes // Page components (if using a router like SvelteKit)
|-- /lib // Utility functions, services, constants (SvelteKit often uses this)
| |-- /stores
| |-- /utils
| |-- /services
| |-- /components // Reusable UI components
|-- App.svelte
|-- main.js (or main.ts)
```
2. **Scoped Styles**: Keep CSS scoped to components to avoid unintended side effects and improve maintainability. Avoid `:global` where possible.
3. **Immutability**: With Svelte 5 and `$state`, direct assignments to properties of `$state` objects (`obj.prop = value;`) are generally fine as Svelte's reactivity system handles updates. However, for non-rune state or when interacting with other systems, understanding and sometimes preferring immutable updates (creating new objects/arrays) can still be relevant.
4. **Use `class:` and `style:` directives**: For dynamic classes and styles, use Svelte's built-in directives for cleaner templates and potentially optimized updates.
```svelte
<script>
let isActive = $state(true);
let color = $state('blue');
</script>
<div class:active={isActive} style:color={color}>
Hello
</div>
```
5. **Stay Updated**: Keep Svelte and its related packages up to date to benefit from the latest features, performance improvements, and security fixes.
## Windmill UI Component Rules (MUST follow)
Always use Windmill's own design-system components instead of raw HTML elements. Using raw HTML elements produces inconsistent styling and breaks the design language.
### Icons — use `lucide-svelte`
**Never** write inline SVGs. Import icons from `lucide-svelte`.
```svelte
<script>
import { ChevronLeft, ChevronRight, X } from 'lucide-svelte'
</script>
<ChevronLeft size={16} />
```
### Buttons — use `<Button>`
**Never** use `<button>`. Import and use `Button` from `$lib/components/common`.
### Buttons — `<Button>`
```svelte
<script>
import { Button } from '$lib/components/common'
import { ChevronLeft, ChevronRight } from 'lucide-svelte'
import { ChevronLeft } from 'lucide-svelte'
</script>
<!-- Regular button -->
<Button variant="default" onclick={handleClick}>Label</Button>
<!-- Icon-only button (no label) -->
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prevMonth} />
<Button startIcon={{ icon: ChevronRight }} iconOnly onclick={nextMonth} />
<Button startIcon={{ icon: ChevronLeft }} iconOnly onclick={prev} />
```
Key `Button` props:
- `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`
- `unifiedSize?: 'sm' | 'md' | 'lg'`
- `startIcon?: { icon: SvelteComponent }` — renders an icon before the label
- `iconOnly?: boolean` — renders icon with no surrounding label text
- `disabled?: boolean`
Props: `variant?: 'accent' | 'accent-secondary' | 'default' | 'subtle'`, `unifiedSize?: 'sm' | 'md' | 'lg'`, `startIcon?: { icon: SvelteComponent }`, `iconOnly?: boolean`, `disabled?: boolean`
### Text inputs — use `<TextInput>`
**Never** use `<input>`. Import and use `TextInput` from `$lib/components/common`.
### Text inputs — `<TextInput>`
```svelte
<script>
import { TextInput } from '$lib/components/common'
let val = $state('')
</script>
<TextInput bind:value={val} placeholder="Enter value" />
```
Key `TextInput` props:
- `value?: string | number` (bindable)
- `placeholder?: string`
- `disabled?: boolean`
- `error?: string | boolean`
- `size?: 'sm' | 'md' | 'lg'`
- `inputProps?` — forwarded to the underlying `<input>`
Props: `value?: string | number` (bindable), `placeholder?: string`, `disabled?: boolean`, `error?: string | boolean`, `size?: 'sm' | 'md' | 'lg'`
### Selects — use `<Select>`
**Never** use `<select>`. Import and use `Select` from `$lib/components/select/Select.svelte`.
### Selects — `<Select>`
```svelte
<script>
import Select from '$lib/components/select/Select.svelte'
const monthItems = [
{ label: 'January', value: 1 },
{ label: 'February', value: 2 },
// ...
]
let selectedMonth = $state(1)
</script>
<Select items={monthItems} bind:value={selectedMonth} />
<Select items={[{ label: 'Jan', value: 1 }]} bind:value={selected} />
```
Key `Select` props:
- `items?: Array<{ label?: string; value: any; subtitle?: string; disabled?: boolean }>`
- `value` (bindable) — the currently selected `.value`
- `placeholder?: string`
- `clearable?: boolean`
- `disabled?: boolean`
- `size?: 'sm' | 'md' | 'lg'`
Props: `items?: Array<{ label?: string; value: any }>`, `value` (bindable), `placeholder?: string`, `clearable?: boolean`, `size?: 'sm' | 'md' | 'lg'`
### Icons — `lucide-svelte`
Never write inline SVGs. Import from `lucide-svelte`:
```svelte
<script>
import { ChevronLeft, X } from 'lucide-svelte'
</script>
<ChevronLeft size={16} />
```
## Form Components
Form components (TextInput, Toggle, Select, etc.) should use the unified size system when placed together.
## Styling
- Use Tailwind CSS for all styling — no custom CSS
- Use Windmill's theming classes for colors/surfaces (see `frontend/brand-guidelines.md`)
- Read component props JSDoc before using them
## Svelte MCP Server
Use the Svelte MCP tools when working on Svelte code:
1. **list-sections**: Call first to discover available docs
2. **get-documentation**: Fetch relevant sections based on use_cases
3. **svelte-autofixer**: MUST use on all Svelte code before finalizing — keep calling until no issues
4. **playground-link**: Only after user confirms and code was NOT written to project files
+1 -1
View File
@@ -42,7 +42,7 @@ RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VER
RUN /usr/local/bin/python3 -m pip install pip-tools
# Bun
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
# Install windmill CLI
RUN bun install -g windmill-cli \
-3
View File
@@ -15,11 +15,8 @@ sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescrip
sed -i '' -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i '' -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i '' -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" python-client/wmill_pg/pyproject.toml
sed -i '' -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i '' -E "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
-3
View File
@@ -16,11 +16,8 @@ sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/typescript-c
sed -i -e "/\"version\": /s/: .*,/: \"$VERSION\",/" ${root_dirpath}/frontend/package.json
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^windmill-api =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill/pyproject.toml
sed -i -e "/^version =/s/= .*/= \"$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^[[:space:]]*ModuleVersion[[:space:]]*=/s/= .*/= '$VERSION'/" ${root_dirpath}/powershell-client/WindmillClient/WindmillClient.psd1
# sed -i -e "/^wmill =/s/= .*/= \"\\^$VERSION\"/" ${root_dirpath}/python-client/wmill_pg/pyproject.toml
sed -i -e "/^wmill =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -e "/^wmill_pg =/s/= .*/= \">=$VERSION\"/" ${root_dirpath}/lsp/Pipfile
sed -i -zE "s/name = \"windmill\"\nversion = \"[^\"]*\"\\n(.*)/name = \"windmill\"\nversion = \"$VERSION\"\\n\\1/" ${root_dirpath}/backend/Cargo.lock
-6
View File
@@ -31,9 +31,3 @@ updates:
directory: "/python-client/wmill"
schedule:
interval: "weekly"
# Maintain dependencies for wmill_pg python client
- package-ecosystem: "pip"
directory: "/python-client/wmill_pg"
schedule:
interval: "weekly"
+2 -2
View File
@@ -55,7 +55,7 @@ jobs:
go-version: 1.21.5
- uses: oven-sh/setup-bun@v2
with:
bun-version: 1.3.8
bun-version: 1.3.10
- uses: actions/setup-node@v4
with:
node-version: "20"
@@ -238,4 +238,4 @@ jobs:
run: |
deno --version && bun -v && node --version && go version && python3 --version && php --version && ruby --version && pwsh --version && dotnet --version
cd windmill-duckdb-ffi-internal && ./build_dev.sh && cd ..
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) PHP_PATH=$(which php) COMPOSER_PATH=$(which composer) RUBY_PATH=$(which ruby) RUBY_BUNDLE_PATH=$(which bundle) RUBY_GEM_PATH=$(which gem) POWERSHELL_PATH=$(which pwsh) DOTNET_PATH=$(which dotnet) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp --all -- --nocapture --test-threads=10
DENO_PATH=$(which deno) BUN_PATH=$(which bun) NODE_BIN_PATH=$(which node) GO_PATH=$(which go) UV_PATH=$(which uv) PHP_PATH=$(which php) COMPOSER_PATH=$(which composer) RUBY_PATH=$(which ruby) RUBY_BUNDLE_PATH=$(which bundle) RUBY_GEM_PATH=$(which gem) POWERSHELL_PATH=$(which pwsh) DOTNET_PATH=$(which dotnet) cargo test --features enterprise,deno_core,duckdb,license,python,rust,scoped_cache,parquet,private,private_registry_test,csharp,php,ruby,mysql,quickjs,mcp,run_inline --all -- --nocapture --test-threads=10
+7 -12
View File
@@ -4,13 +4,13 @@ on:
push:
branches: [main]
paths:
- 'cli/**'
- '.github/workflows/cli-tests.yml'
- "cli/**"
- ".github/workflows/cli-tests.yml"
pull_request:
branches: [main]
paths:
- 'cli/**'
- '.github/workflows/cli-tests.yml'
- "cli/**"
- ".github/workflows/cli-tests.yml"
env:
CARGO_TERM_COLOR: always
@@ -26,7 +26,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -72,7 +72,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -126,7 +126,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: '20'
node-version: "20"
- name: Setup Bun
uses: oven-sh/setup-bun@v2
@@ -163,11 +163,6 @@ jobs:
NODE_BIN_PATH: ${{ steps.runtime-paths.outputs.NODE_BIN_PATH }}
run: bun test --timeout 120000 test/
- name: Keep runner alive for SSH debug
if: failure()
shell: pwsh
run: Start-Sleep -Seconds 3600
# Combined summary job for branch protection
test-summary:
runs-on: ubuntu-latest
+2 -20
View File
@@ -9,9 +9,7 @@ on:
issue_comment:
types:
- created
pull_request_review_comment:
types:
- created
- edited
jobs:
notify_discord_when_pr_opened:
@@ -53,23 +51,7 @@ jobs:
COMMENT_BODY: ${{ github.event.comment.body }}
COMMENT_AUTHOR: ${{ github.event.comment.user.login }}
COMMENT_URL: ${{ github.event.comment.html_url }}
DISCORD_CHANNEL_ID: "1372204995868491786"
DISCORD_GUILD_ID: "930051556043276338"
secrets:
DISCORD_BOT_TOKEN: ${{ secrets.DISCORD_AI_BOT_TOKEN }}
notify_discord_on_review_comment:
if: >
github.event_name == 'pull_request_review_comment'
&& github.event.comment.user.login != 'cloudflare-workers-and-pages[bot]'
&& github.event.comment.user.login != 'ellipsis-dev[bot]'
uses: ./.github/workflows/shareable-discord-notification.yml
with:
PR_STATUS: "comment"
PR_NUMBER: ${{ github.event.pull_request.number }}
COMMENT_BODY: ${{ github.event.comment.body }}
COMMENT_AUTHOR: ${{ github.event.comment.user.login }}
COMMENT_URL: ${{ github.event.comment.html_url }}
COMMENT_IS_EDIT: ${{ github.event.action == 'edited' }}
DISCORD_CHANNEL_ID: "1372204995868491786"
DISCORD_GUILD_ID: "930051556043276338"
secrets:
@@ -36,6 +36,10 @@ on:
description: "The comment URL"
type: string
default: ""
COMMENT_IS_EDIT:
description: "Whether this is an edit of an existing comment"
type: string
default: "false"
secrets:
DISCORD_WEBHOOK_URL:
description: "Discord Webhook URL"
@@ -135,7 +139,7 @@ jobs:
runs-on: ubuntu-latest
if: ${{ inputs.PR_STATUS == 'comment' }}
steps:
- name: Post comment to Discord thread
- name: Post or update comment in Discord thread
env:
BOT_TOKEN: ${{ secrets.DISCORD_BOT_TOKEN }}
CHANNEL_ID: ${{ inputs.DISCORD_CHANNEL_ID }}
@@ -144,6 +148,7 @@ jobs:
COMMENT_BODY: ${{ inputs.COMMENT_BODY }}
COMMENT_AUTHOR: ${{ inputs.COMMENT_AUTHOR }}
COMMENT_URL: ${{ inputs.COMMENT_URL }}
COMMENT_IS_EDIT: ${{ inputs.COMMENT_IS_EDIT }}
run: |
# 1) Find the thread by PR number
threads=$(curl -s -H "Authorization: Bot $BOT_TOKEN" \
@@ -172,10 +177,36 @@ jobs:
truncated_body="$COMMENT_BODY"
fi
# 3) Post the comment to the thread
message=$(printf '**%s** [commented](%s):\n%s' "$COMMENT_AUTHOR" "$COMMENT_URL" "$truncated_body")
# 3) Build the message content
if [ "$COMMENT_IS_EDIT" = "true" ]; then
message=$(printf '**%s** [edited comment](%s):\n%s' "$COMMENT_AUTHOR" "$COMMENT_URL" "$truncated_body")
else
message=$(printf '**%s** [commented](%s):\n%s' "$COMMENT_AUTHOR" "$COMMENT_URL" "$truncated_body")
fi
payload=$(jq -n --arg content "$message" '{content: $content, flags: 4, allowed_mentions: {parse: []}}')
# 4) If this is an edit, try to find and update the existing Discord message
if [ "$COMMENT_IS_EDIT" = "true" ]; then
# Search recent messages in the thread for one containing the comment URL
messages=$(curl -s -H "Authorization: Bot $BOT_TOKEN" \
"https://discord.com/api/v10/channels/${thread_id}/messages?limit=100")
existing_msg_id=$(echo "$messages" | jq -r \
--arg url "$COMMENT_URL" \
'[.[] | select(.content | contains($url))] | first | .id // empty')
if [ -n "$existing_msg_id" ]; then
echo "Updating existing Discord message $existing_msg_id"
curl -s -X PATCH \
-H "Authorization: Bot $BOT_TOKEN" \
-H "Content-Type: application/json" \
-d "$payload" \
"https://discord.com/api/v10/channels/${thread_id}/messages/${existing_msg_id}"
exit 0
fi
echo "Original Discord message not found, posting as new message"
fi
# 5) Post a new message to the thread
curl -s -X POST \
-H "Authorization: Bot $BOT_TOKEN" \
-H "Content-Type: application/json" \
+10 -5
View File
@@ -1,3 +1,8 @@
name: Windmill
startupEnvs:
CARGO_FEATURES: "quickjs"
services:
- name: BE
portEnv: BACKEND_PORT
@@ -53,12 +58,11 @@ profiles:
--endpoint-url "$(printenv R2_ENDPOINT)"
3) The public URL will be:
$(printenv R2_PUBLIC_URL)/<branch>/screenshot.png
4) Include screenshots in PR descriptions as markdown images:
![description]($(printenv R2_PUBLIC_URL)/<branch>/screenshot.png)
4) Include in PR descriptions using markdown image syntax.
--- Terminal Recordings (asciinema) ---
You can record terminal sessions and upload them for sharing.
asciinema is pre-installed at /usr/local/bin/asciinema.
asciinema is available on PATH.
1) Write a shell script with the commands to demo. Add sleep
delays for readable pacing:
@@ -98,8 +102,9 @@ profiles:
4) The public URL will be:
$(printenv R2_PUBLIC_URL)/<branch>/diagram.svg
5) Include in PR descriptions as markdown images:
![description]($(printenv R2_PUBLIC_URL)/<branch>/diagram.svg)
5) Include in PR descriptions using markdown image syntax.
IMPORTANT: Read docs/autonomous-mode.md before starting any work.
linkedRepos:
- repo: windmill-labs/windmill-ee-private
+2 -1
View File
@@ -55,7 +55,8 @@ panes:
- Pane 2: frontend (npm run dev)\n\n
To check logs, use: \`tmux capture-pane -t .1 -p -S -50\` (backend) or \`tmux capture-pane -t .2 -p -S -50\` (frontend).\n
When restarting backend or frontend, make sure to use the ports listed in .env.local.\n
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check."
Because we are running backend with cargo watch, to verify your changes, just check the logs in the backend pane. No need for cargo check.\n\n
IMPORTANT: Read docs/autonomous-mode.md before starting any work."
focus: true
- command: 'ROOT="$(git rev-parse --show-toplevel)"; [ -f "$ROOT/.env.local" ] && source "$ROOT/.env.local"; cd "$ROOT/backend" && PORT=${BACKEND_PORT:-8000} cargo watch -x "run ${CARGO_FEATURES:+--features $CARGO_FEATURES}"'
split: horizontal
+100
View File
@@ -1,5 +1,105 @@
# Changelog
## [1.649.0](https://github.com/windmill-labs/windmill/compare/v1.648.0...v1.649.0) (2026-03-03)
### Features
* **frontend:** add script recorder for offline replay ([#8200](https://github.com/windmill-labs/windmill/issues/8200)) ([c97d8b4](https://github.com/windmill-labs/windmill/commit/c97d8b4715f86ea83ab2c0223ba859ced690829a))
* move index management out of /srch/, add storage size reporting ([#8169](https://github.com/windmill-labs/windmill/issues/8169)) ([ee01acd](https://github.com/windmill-labs/windmill/commit/ee01acd9a6a2cd68a3f226988bfb46f6a6e64c08))
### Bug Fixes
* clean up slow-load toast interval on component destroy ([#8207](https://github.com/windmill-labs/windmill/issues/8207)) ([26f4f2b](https://github.com/windmill-labs/windmill/commit/26f4f2b399b828185b553289d6560e12261030a3))
* **frontend:** prevent subflow expansion from hiding all insertion points ([#8203](https://github.com/windmill-labs/windmill/issues/8203)) ([e97da86](https://github.com/windmill-labs/windmill/commit/e97da860672171e33054a77d71f4824bb09e540d))
* gracefully handle malformed OAuth entries in instance config ([#8205](https://github.com/windmill-labs/windmill/issues/8205)) ([cac4bdd](https://github.com/windmill-labs/windmill/commit/cac4bdd54f0c3ea80844ac31f7597f418ff7d8ae))
* skip stop_after_if evaluation for skipped (identity) flow steps ([#8201](https://github.com/windmill-labs/windmill/issues/8201)) ([e6f7775](https://github.com/windmill-labs/windmill/commit/e6f7775d4d9a052aefc37260c6ed161146841cd7))
* use exact matching for python requirements directive parsing ([#8199](https://github.com/windmill-labs/windmill/issues/8199)) ([2b2be38](https://github.com/windmill-labs/windmill/commit/2b2be38f129bbe58b6bb3815c4bd94aa03a3da90))
### Performance Improvements
* use two-step query in input history to leverage v2_job index ([#8197](https://github.com/windmill-labs/windmill/issues/8197)) ([50defdd](https://github.com/windmill-labs/windmill/commit/50defdded113b4d2cf0991b3fb642d1cd9a462b7))
## [1.648.0](https://github.com/windmill-labs/windmill/compare/v1.647.2...v1.648.0) (2026-03-02)
### Features
* add right-click context menu to ObjectViewer ([#8181](https://github.com/windmill-labs/windmill/issues/8181)) ([1855204](https://github.com/windmill-labs/windmill/commit/18552046c29878b5cf115b9364c2ce829ab7aa59))
* **frontend:** add drag-and-drop node movement in flow editor ([#8076](https://github.com/windmill-labs/windmill/issues/8076)) ([7a5e487](https://github.com/windmill-labs/windmill/commit/7a5e48787860c38aa3589c49ea9a70654d479c8a))
### Bug Fixes
* don't insert underscore after digit in PascalCase to snake_case conversion ([#8184](https://github.com/windmill-labs/windmill/issues/8184)) ([a111653](https://github.com/windmill-labs/windmill/commit/a111653c6d32fd1a3d2f45351eceb8d8d7df6f41))
* **frontend:** preserve keycloak realm url between instance settings saves ([#8189](https://github.com/windmill-labs/windmill/issues/8189)) ([cfd9541](https://github.com/windmill-labs/windmill/commit/cfd9541ab1daf635c7d801cd3a7788db57b98257))
* preserve debouncing settings for post-preprocessing arg accumulation ([#8191](https://github.com/windmill-labs/windmill/issues/8191)) ([9e92445](https://github.com/windmill-labs/windmill/commit/9e92445faed1a10b2406b97562e8df7a5b2dfd76))
## [1.647.2](https://github.com/windmill-labs/windmill/compare/v1.647.1...v1.647.2) (2026-03-02)
### Bug Fixes
* update oracle instant client arm64 download url ([#8179](https://github.com/windmill-labs/windmill/issues/8179)) ([758b35f](https://github.com/windmill-labs/windmill/commit/758b35f8ebbf78e1473a8fd83dbc795d58b23b80))
## [1.647.1](https://github.com/windmill-labs/windmill/compare/v1.647.0...v1.647.1) (2026-03-02)
### Bug Fixes
* add missing display_name and tenant fields to instance config OAuthClient ([#8176](https://github.com/windmill-labs/windmill/issues/8176)) ([db44b8b](https://github.com/windmill-labs/windmill/commit/db44b8be74e1709dbf759dd391bdb3861b3c711b))
* add missing grant_types field to instance config OAuth structs ([#8175](https://github.com/windmill-labs/windmill/issues/8175)) ([fca94f8](https://github.com/windmill-labs/windmill/commit/fca94f88dd796db66e0c5bd0225e23b92efce4a7))
* show sync endpoint timeout setting on all instances ([#8170](https://github.com/windmill-labs/windmill/issues/8170)) ([c70307d](https://github.com/windmill-labs/windmill/commit/c70307d3f2dfe61a0250dd12234470a25baf2d1b))
## [1.647.0](https://github.com/windmill-labs/windmill/compare/v1.646.0...v1.647.0) (2026-03-01)
### Features
* populate baseUrl and userId in Nextcloud resource from OAuth ([#8132](https://github.com/windmill-labs/windmill/issues/8132)) ([5d58a87](https://github.com/windmill-labs/windmill/commit/5d58a87a7f02c4f7775bd02c885071495a5f686d))
* runScript inline for path and hash ([#8019](https://github.com/windmill-labs/windmill/issues/8019)) ([7d9d16a](https://github.com/windmill-labs/windmill/commit/7d9d16a6a3357981e5692023982ca1e670acfaae))
* slow stream warnings, batch size control, and fix result/skipped filters ([#8154](https://github.com/windmill-labs/windmill/issues/8154)) ([7a32abe](https://github.com/windmill-labs/windmill/commit/7a32abec96124f96a1dbac11e03162cca68f3286))
### Bug Fixes
* : persist show schedules and show future jobs toggles in local storage ([#8125](https://github.com/windmill-labs/windmill/issues/8125)) ([f1d8568](https://github.com/windmill-labs/windmill/commit/f1d8568831bf69ee790def4f90df8f32c59a94e0)), closes [#8123](https://github.com/windmill-labs/windmill/issues/8123)
* add partial index for fast failure filtering on runs page ([#8150](https://github.com/windmill-labs/windmill/issues/8150)) ([d4673c2](https://github.com/windmill-labs/windmill/commit/d4673c2e91168dcdb0aca9d6c039df0d9c52bb28))
* copy deps and remove user auto-add on workspace fork ([#8142](https://github.com/windmill-labs/windmill/issues/8142)) ([0776de6](https://github.com/windmill-labs/windmill/commit/0776de6b2173075f533fd59a49efb111000da5df))
* fix custom TS Monaco worker not reloading on file uri change ([#8130](https://github.com/windmill-labs/windmill/issues/8130)) ([b68ff96](https://github.com/windmill-labs/windmill/commit/b68ff965dd4f67046fae7e8cf756c8b3e15c2643))
* Handle CTEs and local tables in SQL asset parser ([#8131](https://github.com/windmill-labs/windmill/issues/8131)) ([0955051](https://github.com/windmill-labs/windmill/commit/095505136c2b3e03f656ace20a5c1bbe142fa63f))
* prevent wm-cursor from hanging on stale cursor IPC sockets ([b9e3e05](https://github.com/windmill-labs/windmill/commit/b9e3e053e4914e753bbb806e6b748c791edb92d2))
* process deletes before adds in CLI sync push to avoid conflicts ([#8148](https://github.com/windmill-labs/windmill/issues/8148)) ([278983c](https://github.com/windmill-labs/windmill/commit/278983c4fd38d67a14a8c208178c04db05ee1880))
* remove review comments from discord notifications and support comment edits ([cdc0543](https://github.com/windmill-labs/windmill/commit/cdc0543747680267e30974037a2eb180a19062d9))
* restore email domain (MX) setting in instance settings UI ([#8152](https://github.com/windmill-labs/windmill/issues/8152)) ([13daebf](https://github.com/windmill-labs/windmill/commit/13daebf88ac1abcb833646490073f922ac7c050e))
* sync flow on_behalf_of_email on load ([#8149](https://github.com/windmill-labs/windmill/issues/8149)) ([faf190f](https://github.com/windmill-labs/windmill/commit/faf190f12d96cd75ba9eda10ab3e6f26d2eed813))
* validate tarball URL host against registry to prevent SSRF and token exfiltration ([#8153](https://github.com/windmill-labs/windmill/issues/8153)) ([86182ed](https://github.com/windmill-labs/windmill/commit/86182ed2e999f018fc72343308e7df8e9de6c189))
### Performance Improvements
* batch large job list requests and fix loadExtraJobs cursor ([#8151](https://github.com/windmill-labs/windmill/issues/8151)) ([4f5a804](https://github.com/windmill-labs/windmill/commit/4f5a8040912e18f34401a6e3a95dea6f97d1d24c))
* lazy-load heavy deps (graphql, openapi-parser, sha256) ([#8145](https://github.com/windmill-labs/windmill/issues/8145)) ([ba48d70](https://github.com/windmill-labs/windmill/commit/ba48d7015741eb6bbbe04088a957c37499cd8471))
* lazy-load markdown in Tooltip components ([#8143](https://github.com/windmill-labs/windmill/issues/8143)) ([bd9ff03](https://github.com/windmill-labs/windmill/commit/bd9ff03010f75557dcc315d10e9208b4e9cafece))
## [1.646.0](https://github.com/windmill-labs/windmill/compare/v1.645.0...v1.646.0) (2026-02-26)
### Features
* add force_branch parameter to git sync settings ([#8089](https://github.com/windmill-labs/windmill/issues/8089)) ([4e1ae27](https://github.com/windmill-labs/windmill/commit/4e1ae276b006992e06ae755ec9315dbfadf4f838))
* add wmill docs CLI command for querying documentation ([#8114](https://github.com/windmill-labs/windmill/issues/8114)) ([01c7270](https://github.com/windmill-labs/windmill/commit/01c7270cdaa0d5dbee2e15aa5dd08551cff60c70))
* Broad filters for search ([#8112](https://github.com/windmill-labs/windmill/issues/8112)) ([16a6d5e](https://github.com/windmill-labs/windmill/commit/16a6d5e7afe9323b2f2c7a93828518f5d924cc69))
* change on behalf selector to allow picking any user + select value in target by default if possible ([#8113](https://github.com/windmill-labs/windmill/issues/8113)) ([408c5af](https://github.com/windmill-labs/windmill/commit/408c5af6d8352f1e205e4543772ce5d060556ffc))
### Bug Fixes
* remove duplicate job loading on chart zoom ([#8121](https://github.com/windmill-labs/windmill/issues/8121)) ([99c01bc](https://github.com/windmill-labs/windmill/commit/99c01bca3863ac9b2882948bb5914f051a7716a4))
* runs page date picker query parameter handling ([#8120](https://github.com/windmill-labs/windmill/issues/8120)) ([427bc64](https://github.com/windmill-labs/windmill/commit/427bc6410be7fda132fc91991164e9b38b32c7e3))
## [1.645.0](https://github.com/windmill-labs/windmill/compare/v1.644.0...v1.645.0) (2026-02-26)
+22 -57
View File
@@ -1,68 +1,33 @@
# Windmill Development Guide
# Windmill
## Overview
Open-source platform for internal tools, workflows, API integrations, background jobs, and UIs. Rust backend + Svelte 5 frontend.
Windmill is an open-source developer platform for building internal tools, workflows, API integrations, background jobs, workflows, and user interfaces. See @windmill-overview.mdc for full platform details.
## Workflow
## New Feature Implementation Guidelines
1. **Understand**: Before coding, read relevant docs from `docs/` to understand the area you're changing
2. **Plan**: For non-trivial changes, use plan mode. For large features, break into reviewable stages
3. **Execute**: Follow coding patterns from skills (`rust-backend`, `svelte-frontend`)
4. **Validate**: After every change, run the appropriate checks per `docs/validation.md`
When implementing new features in Windmill, follow these best practices:
## Documentation
- **Clean Code First**: Write clean, readable, and maintainable code. Prioritize clarity over cleverness.
- **Avoid Duplication at All Costs**: Before writing new code, thoroughly search for existing implementations that can be reused or extended.
- **Adapt Existing Code**: Refactor and generalize existing code when necessary to avoid logic duplication. Extract common patterns into reusable utilities.
- **Follow Established Patterns**: Study existing code patterns in the codebase and maintain consistency with established conventions.
- **Single Responsibility**: Each function, component, and module should have a single, well-defined responsibility.
- **Incremental Implementation**: Break large features into smaller, reviewable chunks that can be implemented and tested incrementally.
## Language-Specific Guides
- Backend (Rust): see `backend/CLAUDE.md` and the `rust-backend` skill: `.claude/skills/rust-backend/SKILL.md`
- Frontend (Svelte 5): see `frontend/CLAUDE.md` and the `svelte-frontend` skill: `.claude/skills/svelte-frontend/SKILL.md`
- **Validation**: `docs/validation.md` — what checks to run based on what you changed
- **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
- **Backend patterns**: use the `rust-backend` skill when writing Rust code
- **Frontend patterns**: use the `svelte-frontend` skill when writing Svelte code
- **Domain guides**: `.claude/skills/native-trigger/` and `frontend/tutorial-system-guide.mdc`
- **Brand/UI guidelines**: `frontend/brand-guidelines.md`
## Dev Environment
- **Backend**: `cargo run` from `backend/` (API at http://localhost:8000)
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/`
- The `REMOTE` env var configures the Vite proxy target. Without it, API calls proxy to `https://app.windmill.dev` instead of the local backend.
- The dev server starts on port 3000 (or 3001+ if 3000 is in use).
- **Default login**: `admin@windmill.dev` / `changeme`
- **Instance settings**: navigate to `/#superadmin-settings` (opens the drawer overlay)
- **Frontend**: `REMOTE=http://localhost:8000 npm run dev` from `frontend/` (port 3000+)
- **DB**: `psql postgres://postgres:changeme@localhost:5432/windmill`
- **Login**: `admin@windmill.dev` / `changeme`
- **Instance settings**: navigate to `/#superadmin-settings`
## UI Testing with Playwright MCP
## Core Principles
When testing the frontend with the Playwright MCP tools:
1. **Start servers**: Launch backend (`cargo run`) and frontend (`REMOTE=http://localhost:8000 npm run dev`) as background tasks
2. **Wait for readiness**: Backend takes ~60s to compile; check output for `health check completed`. Frontend starts in ~5s.
3. **Login flow**: Navigate to `/user/login`, click "Log in without third-party", fill email/password, submit
4. **Instance settings drawer**: Navigate to `/#superadmin-settings` to open the drawer directly
5. **Toggle components**: The YAML toggle uses a custom `<Toggle>` component where the checkbox is visually hidden (`sr-only`). Click the wrapper `<label>` element (the parent container with `cursor=pointer`), not the checkbox ref directly.
6. **Console errors to ignore**: `critical_alerts` 404s are expected on CE builds (EE-only endpoint). VSCode worker 404s are dev-mode artifacts.
## Code Validation (MUST DO)
After making code changes, you MUST run the appropriate checks and fix all errors before considering the work done:
- **Backend**: Run `cargo check` from the `backend/` directory. Only enable the feature flags needed for the code you changed — check `backend/Cargo.toml` `[features]` section to identify which flags gate the crates/modules you modified. For example: `cargo check --features enterprise,parquet` if you only touched enterprise and parquet code.
- **Frontend**: Run `npm run check` from the `frontend/` directory.
## Querying the Database
`backend/summarized_schema.txt` provides a compact overview of all tables, columns, types, ENUMs, and foreign keys. Use it to quickly understand the data model and relationships. Note: this file is a simplified summary — it omits indexes, constraints details, and other metadata.
For exact table definitions (indexes, constraints, column defaults, etc.), query the database directly:
```bash
psql postgres://postgres:changeme@localhost:5432/windmill
```
Useful psql commands:
- `\d <table_name>` — full table definition with indexes and constraints
- `\di <table_name>*` — list indexes for a table
- `\d+ <table_name>` — extended table info including storage and descriptions
This is also helpful for:
- Inspecting database state during development
- Testing queries before implementing them in Rust
- Debugging data-related issues
- Search for existing code to reuse before writing new code
- Follow established patterns in the codebase
- Keep changes focused — don't refactor beyond what's asked
+3 -3
View File
@@ -58,7 +58,7 @@ FROM node:24-alpine as frontend
# install dependencies
WORKDIR /frontend
COPY ./frontend/package.json ./frontend/package-lock.json ./
COPY ./frontend/package.json ./frontend/package-lock.json ./frontend/.npmrc ./
COPY ./frontend/scripts/ ./scripts/
RUN npm ci
@@ -126,7 +126,7 @@ ARG POWERSHELL_DEB_VERSION=7.5.0-1
ARG KUBECTL_VERSION=1.28.7
ARG HELM_VERSION=3.14.3
# NOTE: If changing, also change go version in workspace dependencies template at WorkspaceDependenciesEditor.svelte
ARG GO_VERSION=1.25.0
ARG GO_VERSION=1.26.0
ARG APP=/usr/src/app
ARG WITH_POWERSHELL=true
ARG WITH_KUBECTL=true
@@ -256,7 +256,7 @@ COPY --from=windmill_duckdb_ffi_internal_builder /windmill-duckdb-ffi-internal/t
COPY --from=denoland/deno:2.2.1 --chmod=755 /usr/bin/deno /usr/bin/deno
COPY --from=oven/bun:1.3.8 /usr/local/bin/bun /usr/bin/bun
COPY --from=oven/bun:1.3.10 /usr/local/bin/bun /usr/bin/bun
# Install windmill CLI
RUN bun install -g windmill-cli \
-238
View File
@@ -1,238 +0,0 @@
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y --no-install-recommends \
curl \
ca-certificates \
git \
iptables \
gosu \
sudo \
unzip \
# Rust native build deps (for cargo check)
pkg-config \
cmake \
clang \
mold \
libtool \
libssl-dev \
libxml2-dev \
libxmlsec1-dev \
libxslt1-dev \
libffi-dev \
zlib1g-dev \
libcurl4-openssl-dev \
libclang-dev \
libkrb5-dev \
libsasl2-dev \
# PostgreSQL (for local DB during development)
postgresql \
postgresql-client \
# Node.js 22 (for npm run check / frontend dev)
&& curl -fsSL https://deb.nodesource.com/setup_22.x | bash - \
&& apt-get install -y --no-install-recommends nodejs \
&& rm -rf /var/lib/apt/lists/* \
# Container runs as arbitrary UIDs (--user uid:gid). These three lines make
# sudo work for any UID:
# 1) NOPASSWD rule so sudo never prompts for a password
# 2) Writable passwd/group so the entrypoint can register the dynamic UID
# 3) Writable shadow so unix_chkpwd can validate the account (without this,
# sudo fails with "account validation failure, is your account locked?")
&& echo "ALL ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/sandbox \
&& chmod 0440 /etc/sudoers.d/sandbox \
&& chmod 666 /etc/passwd /etc/group /etc/shadow
# ── GitHub CLI (for PR creation) ──────────────────────────────────────────────
RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
-o /usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
> /etc/apt/sources.list.d/github-cli.list \
&& apt-get update && apt-get install -y --no-install-recommends gh \
&& rm -rf /var/lib/apt/lists/*
# ── Rust toolchain ────────────────────────────────────────────────────────────
# Install under /usr/local/lib/ so bins are world-readable with default umask.
# CARGO_HOME is overridden to /tmp/.cargo at the end for mutable runtime state.
ENV RUSTUP_HOME=/usr/local/lib/rustup CARGO_HOME=/usr/local/lib/cargo
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | \
sh -s -- -y --default-toolchain stable --profile minimal && \
ln -s /usr/local/lib/cargo/bin/* /usr/local/bin/
RUN cargo install sqlx-cli --no-default-features --features native-tls,postgres && \
cargo install cargo-watch && \
cargo install --locked --git https://github.com/asciinema/asciinema && \
ln -sf /usr/local/lib/cargo/bin/sqlx /usr/local/bin/sqlx && \
ln -sf /usr/local/lib/cargo/bin/cargo-watch /usr/local/bin/cargo-watch && \
ln -sf /usr/local/lib/cargo/bin/asciinema /usr/local/bin/asciinema
# ── Register dynamic runtime users ───────────────────────────────────────────
RUN cat <<'SCRIPT' > /usr/local/bin/register-dynamic-user.sh
#!/bin/sh
set -eu
uid="${1:-}"
gid="${2:-}"
if [ -z "$uid" ] || [ -z "$gid" ]; then
echo "register-dynamic-user: usage: register-dynamic-user <uid> <gid>" >&2
exit 1
fi
if ! getent group "$gid" >/dev/null 2>&1; then
echo "sandbox:x:${gid}:" >> /etc/group
fi
if ! getent passwd "$uid" >/dev/null 2>&1; then
echo "sandbox:x:${uid}:${gid}:sandbox:/tmp:/bin/sh" >> /etc/passwd
fi
# Add a shadow entry ("*" = no password) so unix_chkpwd doesn't reject sudo.
if ! grep -q "^sandbox:" /etc/shadow 2>/dev/null; then
echo "sandbox:*:19000:0:99999:7:::" >> /etc/shadow
fi
SCRIPT
RUN chmod +x /usr/local/bin/register-dynamic-user.sh
# ── Network init script (iptables firewall + privilege drop) ──────────────────
RUN cat <<'SCRIPT' > /usr/local/bin/network-init.sh
#!/bin/bash
set -euo pipefail
if [ -n "${WM_PROXY_HOST:-}" ] && [ -n "${WM_PROXY_PORT:-}" ]; then
# Resolve hostnames to ALL IPs (multi-A records, round-robin DNS)
PROXY_IPS=$(getent ahostsv4 "$WM_PROXY_HOST" | awk '{print $1}' | sort -u)
RPC_HOST="${WM_RPC_HOST:-$WM_PROXY_HOST}"
RPC_IPS=$(getent ahostsv4 "$RPC_HOST" | awk '{print $1}' | sort -u)
if [ -z "$PROXY_IPS" ] || [ -z "$RPC_IPS" ]; then
echo "network-init: failed to resolve proxy/RPC host" >&2
exit 1
fi
# IPv4: default deny outbound
iptables -P OUTPUT DROP
iptables -A OUTPUT -o lo -j ACCEPT
iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow DNS (UDP/TCP 53) to configured nameservers.
if [ -f /etc/resolv.conf ]; then
grep '^nameserver' /etc/resolv.conf | awk '{print $2}' | while read -r ns; do
iptables -A OUTPUT -d "$ns" -p udp --dport 53 -j ACCEPT
iptables -A OUTPUT -d "$ns" -p tcp --dport 53 -j ACCEPT
done
fi
# Allow ALL resolved proxy IPs (handles multi-A DNS)
for ip in $PROXY_IPS; do
iptables -A OUTPUT -d "$ip" -p tcp --dport "$WM_PROXY_PORT" -j ACCEPT
done
# Allow ALL resolved RPC IPs
if [ -n "${WM_RPC_PORT:-}" ]; then
for ip in $RPC_IPS; do
iptables -A OUTPUT -d "$ip" -p tcp --dport "$WM_RPC_PORT" -j ACCEPT
done
fi
# Reject (not drop) everything else to fail fast instead of hanging
iptables -A OUTPUT -j REJECT
# IPv6: block entirely to prevent leaks (fail closed)
if ip6tables -L -n >/dev/null 2>&1; then
ip6tables -P OUTPUT DROP
ip6tables -A OUTPUT -o lo -j ACCEPT
ip6tables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
ip6tables -A OUTPUT -j REJECT
else
if ! sysctl -w net.ipv6.conf.all.disable_ipv6=1 2>/dev/null; then
echo "network-init: failed to block IPv6 (neither ip6tables nor sysctl available)" >&2
exit 1
fi
fi
fi
# Add sandbox user/group so sudo works after dropping privileges.
if [ -z "${WM_TARGET_UID:-}" ] || [ -z "${WM_TARGET_GID:-}" ]; then
echo "network-init: WM_TARGET_UID and WM_TARGET_GID are required" >&2
exit 1
fi
/usr/local/bin/register-dynamic-user.sh "${WM_TARGET_UID}" "${WM_TARGET_GID}"
# Fix PTY ownership so the unprivileged user can read/write the terminal.
if [ -t 0 ]; then
chown "${WM_TARGET_UID}:${WM_TARGET_GID}" "$(tty)"
fi
# Drop privileges and exec the user command.
exec gosu "${WM_TARGET_UID}:${WM_TARGET_GID}" env HOME=/tmp "$@"
SCRIPT
RUN chmod +x /usr/local/bin/network-init.sh
# ── workmux (sandbox RPC) ────────────────────────────────────────────────────
RUN curl -fsSL https://raw.githubusercontent.com/raine/workmux/main/scripts/install.sh | bash
# ── Claude Code ───────────────────────────────────────────────────────────────
RUN curl -fsSL https://claude.ai/install.sh | bash && \
target="$(readlink -f /root/.local/bin/claude)" && \
mv /root/.local/share/claude /usr/local/lib/claude && \
ln -s "/usr/local/lib/claude/versions/$(basename "$target")" /usr/local/bin/claude && \
mkdir -p /tmp/.local/bin && \
ln -s /usr/local/bin/claude /tmp/.local/bin/claude && \
chmod -R a+rwX /tmp/.local
# ── Codex ─────────────────────────────────────────────────────────────────────
RUN npm i -g @openai/codex
# ── Bun ───────────────────────────────────────────────────────────────────────
ENV BUN_INSTALL=/usr/local/lib/bun
RUN curl -fsSL https://bun.sh/install | bash && \
ln -s /usr/local/lib/bun/bin/bun /usr/local/bin/bun && \
ln -s /usr/local/lib/bun/bin/bunx /usr/local/bin/bunx
# ── Playwright + Chromium (for screenshots) ──────────────────────────────────
ENV PLAYWRIGHT_BROWSERS_PATH=/usr/local/lib/playwright-browsers
RUN bun add -g @playwright/test \
&& bunx playwright install chromium --with-deps \
&& chmod -R a+rwX /usr/local/lib/playwright-browsers \
&& chmod -R a+rwX /usr/local/lib/bun/install \
&& rm -rf /var/lib/apt/lists/* /tmp/bunx-*
# ── AWS CLI (for S3-compatible uploads to R2) ─────────────────────────────────
RUN curl -fsSL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o /tmp/awscliv2.zip \
&& unzip -q /tmp/awscliv2.zip -d /tmp \
&& /tmp/aws/install \
&& rm -rf /tmp/aws /tmp/awscliv2.zip
ENV AWS_DEFAULT_REGION=auto
# ── Runtime env for arbitrary UID ─────────────────────────────────────────────
# Mutable state goes to /tmp (writable by any UID). Toolchains stay read-only.
ENV CARGO_HOME=/tmp/.cargo BUN_TMPDIR=/tmp
# ── Entrypoint ────────────────────────────────────────────────────────────────
RUN cat <<'ENTRY' > /usr/local/bin/entrypoint.sh
#!/bin/sh
/usr/local/bin/register-dynamic-user.sh "$(id -u)" "$(id -g)"
# Start PostgreSQL (unix socket in /tmp, owned by postgres user)
mkdir -p /tmp/pgdata && sudo chown postgres:postgres /tmp/pgdata
if [ ! -f /tmp/pgdata/PG_VERSION ]; then
sudo -u postgres /usr/lib/postgresql/15/bin/initdb -D /tmp/pgdata --auth=trust
fi
sudo -u postgres /usr/lib/postgresql/15/bin/pg_ctl -D /tmp/pgdata -l /tmp/pg.log start -o "-k /tmp"
sudo -u postgres psql -h /tmp -c "CREATE ROLE sandbox SUPERUSER LOGIN" 2>/dev/null || true
sudo -u postgres createdb -h /tmp windmill 2>/dev/null || true
# Run database migrations so sqlx compile-time checks work
if [ -d "$PWD/backend/migrations" ]; then
DATABASE_URL="postgres://sandbox@localhost/windmill?host=/tmp" \
sqlx migrate run --source "$PWD/backend/migrations" 2>/dev/null || true
fi
# Install frontend dependencies and generate backend client
if [ -d "$PWD/frontend" ]; then
(cd "$PWD/frontend" && npm install && npm run generate-backend-client) 2>/dev/null || true
fi
exec "$@"
ENTRY
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
@@ -1,202 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT * FROM workspace_settings WHERE teams_team_id = $1 AND teams_command_script IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "slack_team_id",
"type_info": "Varchar"
},
{
"ordinal": 2,
"name": "slack_name",
"type_info": "Varchar"
},
{
"ordinal": 3,
"name": "slack_command_script",
"type_info": "Varchar"
},
{
"ordinal": 4,
"name": "slack_email",
"type_info": "Varchar"
},
{
"ordinal": 5,
"name": "customer_id",
"type_info": "Varchar"
},
{
"ordinal": 6,
"name": "plan",
"type_info": "Varchar"
},
{
"ordinal": 7,
"name": "webhook",
"type_info": "Text"
},
{
"ordinal": 8,
"name": "deploy_to",
"type_info": "Varchar"
},
{
"ordinal": 9,
"name": "ai_config",
"type_info": "Jsonb"
},
{
"ordinal": 10,
"name": "large_file_storage",
"type_info": "Jsonb"
},
{
"ordinal": 11,
"name": "git_sync",
"type_info": "Jsonb"
},
{
"ordinal": 12,
"name": "default_app",
"type_info": "Varchar"
},
{
"ordinal": 13,
"name": "default_scripts",
"type_info": "Jsonb"
},
{
"ordinal": 14,
"name": "deploy_ui",
"type_info": "Jsonb"
},
{
"ordinal": 15,
"name": "mute_critical_alerts",
"type_info": "Bool"
},
{
"ordinal": 16,
"name": "color",
"type_info": "Varchar"
},
{
"ordinal": 17,
"name": "operator_settings",
"type_info": "Jsonb"
},
{
"ordinal": 18,
"name": "teams_command_script",
"type_info": "Text"
},
{
"ordinal": 19,
"name": "teams_team_id",
"type_info": "Text"
},
{
"ordinal": 20,
"name": "teams_team_name",
"type_info": "Text"
},
{
"ordinal": 21,
"name": "git_app_installations",
"type_info": "Jsonb"
},
{
"ordinal": 22,
"name": "ducklake",
"type_info": "Jsonb"
},
{
"ordinal": 23,
"name": "slack_oauth_client_id",
"type_info": "Varchar"
},
{
"ordinal": 24,
"name": "slack_oauth_client_secret",
"type_info": "Varchar"
},
{
"ordinal": 25,
"name": "datatable",
"type_info": "Jsonb"
},
{
"ordinal": 26,
"name": "teams_team_guid",
"type_info": "Text"
},
{
"ordinal": 27,
"name": "auto_invite",
"type_info": "Jsonb"
},
{
"ordinal": 28,
"name": "error_handler",
"type_info": "Jsonb"
},
{
"ordinal": 29,
"name": "success_handler",
"type_info": "Jsonb"
},
{
"ordinal": 30,
"name": "public_app_execution_limit_per_minute",
"type_info": "Int4"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
true,
false,
true,
true,
true,
true,
true,
true,
true,
true,
true
]
},
"hash": "08f288d2781d823e109a9e5b8848234ca7d1efeee9661f3901f298da375e73f7"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, preprocessed)\n VALUES ($1, 'flow', 'flow', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Varchar",
"Bool"
]
},
"nullable": []
},
"hash": "181e6fca7e0d0fd88eccd79303f0339b1f2194c52f6bd1245dfa8ff3f0db4051"
}
@@ -1,23 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "SELECT value FROM variable WHERE workspace_id = $1 AND path = $2",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "value",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"Text",
"Text"
]
},
"nullable": [
false
]
},
"hash": "2c0ab7571e1a7c4290315bc3efccb4db9e0c9aee05596a594f81975a0cdb74d1"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag, running)\n VALUES ($1, $2, now(), 'flow', false)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar"
]
},
"nullable": []
},
"hash": "2c503e1e8ee0863b3a6274874ef9b9a10b31dbbe2a676a50d1bbfb2e9e0ab7e0"
}
@@ -0,0 +1,28 @@
{
"db_name": "PostgreSQL",
"query": "SELECT workspace_id, teams_command_script FROM workspace_settings WHERE teams_team_id = $1 AND teams_command_script IS NOT NULL",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "workspace_id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "teams_command_script",
"type_info": "Text"
}
],
"parameters": {
"Left": [
"Text"
]
},
"nullable": [
false,
true
]
},
"hash": "34721bce20aa8b2a2c6b9bd5455735f1a2270f23d73de95101e6350f6df40acc"
}
@@ -0,0 +1,14 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO global_settings (name, value) VALUES ('indexer_settings', $1)\n ON CONFLICT (name) DO UPDATE SET value = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Jsonb"
]
},
"nullable": []
},
"hash": "380ca9ebea53d5c016e4e76797cc103178ac4a25fc2842a13ce19b1ec4445c9d"
}
@@ -1,19 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "WITH job_result AS (\n SELECT result\n FROM v2_job_completed\n WHERE id = $1\n ),\n updated_queue AS (\n UPDATE v2_job_queue\n SET running = false,\n tag = COALESCE($3, tag),\n scheduled_for = COALESCE($6, scheduled_for)\n WHERE id = $2\n )\n UPDATE v2_job\n SET\n tag = COALESCE($3, tag),\n concurrent_limit = COALESCE($4, concurrent_limit),\n concurrency_time_window_s = COALESCE($5, concurrency_time_window_s),\n args = COALESCE(\n CASE\n WHEN job_result.result IS NULL THEN NULL\n WHEN jsonb_typeof(job_result.result) = 'object'\n THEN job_result.result\n WHEN jsonb_typeof(job_result.result) = 'null'\n THEN NULL\n ELSE jsonb_build_object('value', job_result.result)\n END,\n '{}'::jsonb\n ),\n preprocessed = TRUE\n FROM job_result\n WHERE v2_job.id = $2;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid",
"Varchar",
"Int4",
"Int4",
"Timestamptz"
]
},
"nullable": []
},
"hash": "79b437ad31ddab94310989b8fb6a1c130b9be1ab4b6a100fffffd687677b9c92"
}
@@ -0,0 +1,17 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, args)\n VALUES ($1, 'script', 'deno', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Varchar",
"Varchar",
"Jsonb"
]
},
"nullable": []
},
"hash": "9c76a980bf1e3b79ab26c79aee19e5552aa16eb3626618da4dbb44ed18efee60"
}
@@ -0,0 +1,34 @@
{
"db_name": "PostgreSQL",
"query": "SELECT id, last_locked_at, owner FROM concurrency_locks WHERE id = ANY($1)",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id",
"type_info": "Varchar"
},
{
"ordinal": 1,
"name": "last_locked_at",
"type_info": "Timestamp"
},
{
"ordinal": 2,
"name": "owner",
"type_info": "Varchar"
}
],
"parameters": {
"Left": [
"TextArray"
]
},
"nullable": [
false,
false,
true
]
},
"hash": "bcefd1ce47d05f2ce14493f0e7c4d4fea16c0cf71ddc233f6431cf624ecdfe60"
}
@@ -0,0 +1,20 @@
{
"db_name": "PostgreSQL",
"query": "WITH job_result AS (\n SELECT result\n FROM v2_job_completed\n WHERE id = $1\n ),\n updated_queue AS (\n UPDATE v2_job_queue\n SET running = false,\n tag = COALESCE($3, tag),\n scheduled_for = COALESCE($6, scheduled_for),\n runnable_settings_handle = COALESCE($7, runnable_settings_handle)\n WHERE id = $2\n )\n UPDATE v2_job\n SET\n tag = COALESCE($3, tag),\n concurrent_limit = COALESCE($4, concurrent_limit),\n concurrency_time_window_s = COALESCE($5, concurrency_time_window_s),\n args = COALESCE(\n CASE\n WHEN job_result.result IS NULL THEN NULL\n WHEN jsonb_typeof(job_result.result) = 'object'\n THEN job_result.result\n WHEN jsonb_typeof(job_result.result) = 'null'\n THEN NULL\n ELSE jsonb_build_object('value', job_result.result)\n END,\n '{}'::jsonb\n ),\n preprocessed = TRUE\n FROM job_result\n WHERE v2_job.id = $2;\n ",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Uuid",
"Varchar",
"Int4",
"Int4",
"Timestamptz",
"Int8"
]
},
"nullable": []
},
"hash": "c2a0605b07f5df8d972bc02cc23fe7def5e1ee8fdf6dfb68576d3b72aa03f666"
}
@@ -0,0 +1,15 @@
{
"db_name": "PostgreSQL",
"query": "UPDATE v2_job SET args = $2 WHERE id = $1",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Uuid",
"Jsonb"
]
},
"nullable": []
},
"hash": "c31cf6239044615e1cc3743aa1c82cce96e1a23ada28107ffffc8b5546d48101"
}
@@ -1,15 +0,0 @@
{
"db_name": "PostgreSQL",
"query": "INSERT INTO workspace_invite (workspace_id, email, is_admin, operator)\n SELECT $1, email, is_admin, operator\n FROM usr\n WHERE workspace_id = $2",
"describe": {
"columns": [],
"parameters": {
"Left": [
"Varchar",
"Text"
]
},
"nullable": []
},
"hash": "cd399a3a797d1733fb9071ebca3f5928a3c7eba2983431844581fd2393312a2e"
}
+7 -97
View File
@@ -1,98 +1,8 @@
# Backend Development (Rust)
# Backend (Rust)
## Project Structure
Windmill uses a workspace-based architecture with multiple crates:
- **windmill-api**: API server functionality
- **windmill-worker**: Job execution
- **windmill-common**: Shared code used by all crates
- **windmill-queue**: Job & flow queuing
- **windmill-audit**: Audit logging
- Other specialized crates (git-sync, autoscaling, etc.)
## Key References (MUST FOLLOW THESE)
- You MUST follow best-practices by using the `rust-backend` skill, everytime you write RUST code.
- When working with the database: read `summarized_schema.txt` before starting
- When working with the API routes: you can read `windmill-api/src/lib.rs` to get started
## Adding New Code
### Module Organization
- Place new code in the appropriate crate based on functionality
- For API endpoints, create or modify files in `windmill-api/src/` organized by domain
- For shared functionality, use `windmill-common/src/`
- Follow existing patterns for file structure and organization
### API Endpoints
- Follow existing patterns in the `windmill-api` crate
- Use axum's routing system and extractors
- Update `backend/windmill-api/openapi.yaml` after modifying API endpoints
### Database Changes
- Update database schema with migration if necessary
- Use `sqlx` for database operations with prepared statements
- Use transactions for multi-step operations
- To apply pending migrations: `sqlx migrate run` (never manually run .sql files)
- **Never use `SQLX_OFFLINE=true`** — a live database is always available for compilation
- After all code changes are done, run `./update-sqlx` to regenerate the offline query cache
## Enterprise Features
- Enterprise files use the `*_ee.rs` suffix
- Enterprise source is in `windmill-ee-private` folder (sibling directory at `../../windmill-ee-private` or `~/windmill-ee-private`), symlinked into each crate's `src/`
- The `_ee.rs` files are gitignored in the main repo — they are tracked only in the `windmill-ee-private` repo
- You can and should modify `windmill-ee-private` directly when needed (e.g., when creating new crates that need EE code, mirror the package structure there)
- Use feature flags: `#[cfg(feature = "enterprise")]`
- Isolate enterprise code in separate modules
### EE PR Workflow (MUST DO when modifying `*_ee.rs` files)
When you modify any `*_ee.rs` file and create a PR on the windmill repo, you **MUST** also:
1. **Create a matching branch** in the `windmill-ee-private` repo (use the same branch name). If using worktrees, the EE worktree is at `~/windmill-ee-private__worktrees/<branch-name>/`
2. **Commit and push** the `_ee.rs` changes in that branch
3. **Create a PR** on `windmill-ee-private` with a link to the companion windmill PR
4. **Update `ee-repo-ref.txt`**: Run `bash write_latest_ee_ref.sh` from `backend/` to write the latest EE commit hash. **Important**: the script may fall back to `~/windmill-ee-private` (main branch) instead of the worktree — verify it wrote the correct commit hash from your branch, not from main. If wrong, manually write the correct hash.
5. **Commit `ee-repo-ref.txt`** in the windmill repo so CI picks up the correct EE ref
## Code Validation (MUST DO)
After making backend changes, you MUST run `cargo check` and fix all errors and warnings before considering the work done.
Only enable the feature flags relevant to your changes — do NOT use `all_sqlx_features` as it compiles the entire codebase and is very slow. Check the `[features]` section in `Cargo.toml` to identify which flags gate the crates/modules you modified.
Examples:
```bash
# Changed core code (no feature-gated modules)
cargo check
# Changed code behind the enterprise feature
cargo check --features enterprise
# Changed kafka trigger code
cargo check --features kafka
```
## Git Workflow
- **Never push directly to main** — always create a branch and open a pull request
## Testing
- Write unit tests for core functionality
- Use the `#[cfg(test)]` module for test code
- For database tests, use the existing test utilities
## Common Crates
- **tokio**: Async runtime
- **axum**: Web server and routing
- **sqlx**: Database operations
- **serde**: Serialization/deserialization
- **tracing**: Logging and diagnostics
- **reqwest**: HTTP client
- **Coding patterns**: MUST use the `rust-backend` skill when writing Rust code
- **Validation**: `docs/validation.md` — which `cargo check` flags to use
- **Enterprise**: `docs/enterprise.md` — EE file conventions and PR workflow
- **DB schema**: `backend/summarized_schema.txt`
- **API routes entry point**: `windmill-api/src/lib.rs`
- **OpenAPI spec**: `windmill-api/openapi.yaml`
+122 -106
View File
@@ -860,9 +860,9 @@ dependencies = [
[[package]]
name = "aws-lc-rs"
version = "1.16.0"
version = "1.16.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9a7b350e3bb1767102698302bc37256cbd48422809984b98d292c40e2579aa9"
checksum = "94bffc006df10ac2a68c83692d734a465f8ee6c5b384d8545a636f81d858f4bf"
dependencies = [
"aws-lc-sys",
"zeroize",
@@ -870,9 +870,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.37.1"
version = "0.38.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b092fe214090261288111db7a2b2c2118e5a7f30dc2569f1732c4069a6840549"
checksum = "4321e568ed89bb5a7d291a7f37997c2c0df89809d7b6d12062c81ddb54aa782e"
dependencies = [
"cc",
"cmake",
@@ -1334,9 +1334,9 @@ dependencies = [
[[package]]
name = "aws-smithy-xml"
version = "0.60.14"
version = "0.60.15"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b53543b4b86ed43f051644f704a98c7291b3618b67adf057ee77a366fa52fcaa"
checksum = "0ce02add1aa3677d022f8adf81dcbe3046a95f17a1b1e8979c145cd21d3d22b3"
dependencies = [
"xmlparser",
]
@@ -6173,20 +6173,20 @@ dependencies = [
"cfg-if",
"js-sys",
"libc",
"r-efi",
"r-efi 5.3.0",
"wasip2",
"wasm-bindgen",
]
[[package]]
name = "getrandom"
version = "0.4.1"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "139ef39800118c7683f2fd3c98c1b23c09ae076556b435f8e9064ae108aaeeec"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"r-efi 6.0.0",
"wasip2",
"wasip3",
]
@@ -7421,9 +7421,9 @@ dependencies = [
[[package]]
name = "ipnet"
version = "2.11.0"
version = "2.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "469fb0b9cefa57e3ef31275ee7cacb78f2fdca44e4765491884a2b119d4eb130"
checksum = "d98f6fed1fde3f8c21bc40a1abb88dd75e67924f9cffc3ef95607bad8017f8e2"
[[package]]
name = "ipnetwork"
@@ -8049,13 +8049,14 @@ dependencies = [
[[package]]
name = "libredox"
version = "0.1.12"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d0b95e02c851351f877147b7deea7b1afb1df71b63aa5f8270716e0c5720616"
checksum = "1744e39d1d6a9948f4f388969627434e31128196de472883b39f148769bfe30a"
dependencies = [
"bitflags 2.9.4",
"libc",
"redox_syscall 0.7.2",
"plain",
"redox_syscall 0.7.3",
]
[[package]]
@@ -8599,9 +8600,9 @@ dependencies = [
[[package]]
name = "moka"
version = "0.12.13"
version = "0.12.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ac832c50ced444ef6be0767a008b02c106a909ba79d1d830501e94b96f6b7e"
checksum = "85f8024e1c8e71c778968af91d43700ce1d11b219d127d79fb2934153b82b42b"
dependencies = [
"async-lock",
"crossbeam-channel",
@@ -10085,18 +10086,18 @@ dependencies = [
[[package]]
name = "pin-project"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "677f1add503faace112b9f1373e43e9e054bfdd22ff1a63c1bc485eaec6a6a8a"
checksum = "f1749c7ed4bcaf4c3d0a3efc28538844fb29bcdd7d2b67b2be7e20ba861ff517"
dependencies = [
"pin-project-internal",
]
[[package]]
name = "pin-project-internal"
version = "1.1.10"
version = "1.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e918e4ff8c4549eb882f14b3a4bc8c8bc93de829416eacf579f1207a8fbf861"
checksum = "d9b20ed30f105399776b9c883e68e536ef602a16ae6f596d2c473591d6ad64c6"
dependencies = [
"proc-macro2",
"quote",
@@ -10105,9 +10106,9 @@ dependencies = [
[[package]]
name = "pin-project-lite"
version = "0.2.16"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "pin-utils"
@@ -10159,6 +10160,12 @@ version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7edddbd0b52d732b21ad9a5fab5c704c14cd949e5e9a1ec5929a24fded1b904c"
[[package]]
name = "plain"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4596b6d070b27117e987119b4dac604f3c58cfb0b191112e24771b2faeac1a6"
[[package]]
name = "png"
version = "0.17.16"
@@ -10716,6 +10723,12 @@ version = "5.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "69cdb34c158ceb288df11e18b4bd39de994f6657d83847bdffdbd7f346754b0f"
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "radium"
version = "0.7.0"
@@ -10855,9 +10868,9 @@ dependencies = [
[[package]]
name = "range-alloc"
version = "0.1.4"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3d6831663a5098ea164f89cff59c6284e95f4e3c76ce9848d4529f5ccca9bde"
checksum = "ca45419789ae5a7899559e9512e58ca889e41f04f1f2445e9f4b290ceccd1d08"
[[package]]
name = "raw-cpuid"
@@ -10989,9 +11002,9 @@ dependencies = [
[[package]]
name = "redox_syscall"
version = "0.7.2"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d94dd2f7cd932d4dc02cc8b2b50dfd38bd079a4e5d79198b99743d7fcf9a4b4"
checksum = "6ce70a74e890531977d37e532c34d45e9055d2409ed08ddba14529471ed0be16"
dependencies = [
"bitflags 2.9.4",
]
@@ -12586,9 +12599,9 @@ checksum = "1b6709c7b6754dca1311b3c73e79fcce40dd414c782c66d88e8823030093b02b"
[[package]]
name = "sketches-ddsketch"
version = "0.3.0"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c1e9a774a6c28142ac54bb25d25562e6bcf957493a184f15ad4eebccb23e410a"
checksum = "0c6f73aeb92d671e0cc4dca167e59b2deb6387c375391bc99ee743f326994a2b"
dependencies = [
"serde",
]
@@ -13843,7 +13856,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "82a72c767771b47409d2345987fda8628641887d5466101319899796367354a0"
dependencies = [
"fastrand",
"getrandom 0.4.1",
"getrandom 0.4.2",
"once_cell",
"rustix 1.1.4",
"windows-sys 0.61.2",
@@ -15725,7 +15738,7 @@ dependencies = [
[[package]]
name = "windmill"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-nats",
@@ -15789,7 +15802,7 @@ dependencies = [
[[package]]
name = "windmill-alerting"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15802,7 +15815,7 @@ dependencies = [
[[package]]
name = "windmill-api"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"argon2",
@@ -15940,7 +15953,7 @@ dependencies = [
[[package]]
name = "windmill-api-agent-workers"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15963,7 +15976,7 @@ dependencies = [
[[package]]
name = "windmill-api-assets"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -15976,7 +15989,7 @@ dependencies = [
[[package]]
name = "windmill-api-auth"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16002,7 +16015,7 @@ dependencies = [
[[package]]
name = "windmill-api-client"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"reqwest 0.12.28",
"serde",
@@ -16012,7 +16025,7 @@ dependencies = [
[[package]]
name = "windmill-api-configs"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16029,7 +16042,7 @@ dependencies = [
[[package]]
name = "windmill-api-debug"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"base64 0.22.1",
@@ -16052,7 +16065,7 @@ dependencies = [
[[package]]
name = "windmill-api-embeddings"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16075,7 +16088,7 @@ dependencies = [
[[package]]
name = "windmill-api-flow-conversations"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16091,7 +16104,7 @@ dependencies = [
[[package]]
name = "windmill-api-flows"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16111,7 +16124,7 @@ dependencies = [
[[package]]
name = "windmill-api-groups"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16131,7 +16144,7 @@ dependencies = [
[[package]]
name = "windmill-api-inputs"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16145,7 +16158,7 @@ dependencies = [
[[package]]
name = "windmill-api-integration-tests"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-nats",
@@ -16167,11 +16180,12 @@ dependencies = [
"windmill-common",
"windmill-native-triggers",
"windmill-test-utils",
"windmill-worker",
]
[[package]]
name = "windmill-api-jobs"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16196,7 +16210,7 @@ dependencies = [
[[package]]
name = "windmill-api-npm-proxy"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"flate2",
@@ -16207,13 +16221,14 @@ dependencies = [
"tar",
"tower-http",
"tracing",
"url",
"windmill-api-auth",
"windmill-common",
]
[[package]]
name = "windmill-api-openapi"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16234,7 +16249,7 @@ dependencies = [
[[package]]
name = "windmill-api-schedule"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16254,7 +16269,7 @@ dependencies = [
[[package]]
name = "windmill-api-scripts"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16284,7 +16299,7 @@ dependencies = [
[[package]]
name = "windmill-api-settings"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16311,7 +16326,7 @@ dependencies = [
[[package]]
name = "windmill-api-sse"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"lazy_static",
"serde",
@@ -16323,7 +16338,7 @@ dependencies = [
[[package]]
name = "windmill-api-users"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"argon2",
"axum 0.7.9",
@@ -16346,7 +16361,7 @@ dependencies = [
[[package]]
name = "windmill-api-workers"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16360,7 +16375,7 @@ dependencies = [
[[package]]
name = "windmill-api-workspaces"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"axum 0.7.9",
"chrono",
@@ -16390,7 +16405,7 @@ dependencies = [
[[package]]
name = "windmill-audit"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"chrono",
"lazy_static",
@@ -16404,7 +16419,7 @@ dependencies = [
[[package]]
name = "windmill-autoscaling"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -16423,7 +16438,7 @@ dependencies = [
[[package]]
name = "windmill-common"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"aes-gcm",
"anyhow",
@@ -16522,7 +16537,7 @@ dependencies = [
[[package]]
name = "windmill-dep-map"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"chrono",
"itertools 0.14.0",
@@ -16541,7 +16556,7 @@ dependencies = [
[[package]]
name = "windmill-git-sync"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"regex",
"serde",
@@ -16556,7 +16571,7 @@ dependencies = [
[[package]]
name = "windmill-indexer"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"astral-tokio-tar",
@@ -16580,7 +16595,7 @@ dependencies = [
[[package]]
name = "windmill-jseval"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"futures",
@@ -16597,7 +16612,7 @@ dependencies = [
[[package]]
name = "windmill-macros"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"itertools 0.14.0",
"lazy_static",
@@ -16613,7 +16628,7 @@ dependencies = [
[[package]]
name = "windmill-mcp"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16634,7 +16649,7 @@ dependencies = [
[[package]]
name = "windmill-native-triggers"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -16665,7 +16680,7 @@ dependencies = [
[[package]]
name = "windmill-oauth"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-oauth2",
@@ -16689,7 +16704,7 @@ dependencies = [
[[package]]
name = "windmill-object-store"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-stream",
@@ -16723,7 +16738,7 @@ dependencies = [
[[package]]
name = "windmill-operator"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"futures",
@@ -16741,7 +16756,7 @@ dependencies = [
[[package]]
name = "windmill-parser"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"convert_case 0.6.0",
"serde",
@@ -16750,7 +16765,7 @@ dependencies = [
[[package]]
name = "windmill-parser-bash"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16762,7 +16777,7 @@ dependencies = [
[[package]]
name = "windmill-parser-csharp"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16774,7 +16789,7 @@ dependencies = [
[[package]]
name = "windmill-parser-go"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"gosyn",
@@ -16786,7 +16801,7 @@ dependencies = [
[[package]]
name = "windmill-parser-graphql"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16798,7 +16813,7 @@ dependencies = [
[[package]]
name = "windmill-parser-java"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"serde_json",
@@ -16810,7 +16825,7 @@ dependencies = [
[[package]]
name = "windmill-parser-nu"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"nu-parser",
@@ -16821,7 +16836,7 @@ dependencies = [
[[package]]
name = "windmill-parser-php"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16832,7 +16847,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"itertools 0.14.0",
@@ -16845,7 +16860,7 @@ dependencies = [
[[package]]
name = "windmill-parser-py-imports"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16869,7 +16884,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ruby"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16883,7 +16898,7 @@ dependencies = [
[[package]]
name = "windmill-parser-rust"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"convert_case 0.6.0",
@@ -16900,7 +16915,7 @@ dependencies = [
[[package]]
name = "windmill-parser-sql"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16915,7 +16930,7 @@ dependencies = [
[[package]]
name = "windmill-parser-ts"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"lazy_static",
@@ -16934,7 +16949,7 @@ dependencies = [
[[package]]
name = "windmill-parser-yaml"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"serde",
@@ -16945,7 +16960,7 @@ dependencies = [
[[package]]
name = "windmill-queue"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -16982,7 +16997,7 @@ dependencies = [
[[package]]
name = "windmill-runtime-nativets"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"const_format",
@@ -17020,8 +17035,9 @@ dependencies = [
[[package]]
name = "windmill-sql-datatype-parser-wasm"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"getrandom 0.3.4",
"wasm-bindgen",
"wasm-bindgen-test",
"windmill-parser",
@@ -17030,7 +17046,7 @@ dependencies = [
[[package]]
name = "windmill-store"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-recursion",
@@ -17059,7 +17075,7 @@ dependencies = [
[[package]]
name = "windmill-test-utils"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"axum 0.7.9",
@@ -17082,7 +17098,7 @@ dependencies = [
[[package]]
name = "windmill-trigger"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17115,7 +17131,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-email"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17135,7 +17151,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-gcp"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17169,7 +17185,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-http"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17204,7 +17220,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-kafka"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17227,7 +17243,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-mqtt"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17251,7 +17267,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-nats"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-nats",
@@ -17275,7 +17291,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-postgres"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17310,7 +17326,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-sqs"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17338,7 +17354,7 @@ dependencies = [
[[package]]
name = "windmill-trigger-websocket"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-trait",
@@ -17361,7 +17377,7 @@ dependencies = [
[[package]]
name = "windmill-types"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"bitflags 2.9.4",
@@ -17379,7 +17395,7 @@ dependencies = [
[[package]]
name = "windmill-worker"
version = "1.645.0"
version = "1.649.0"
dependencies = [
"anyhow",
"async-once-cell",
@@ -18349,18 +18365,18 @@ dependencies = [
[[package]]
name = "zerocopy"
version = "0.8.39"
version = "0.8.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db6d35d663eadb6c932438e763b262fe1a70987f9ae936e60158176d710cae4a"
checksum = "a789c6e490b576db9f7e6b6d661bcc9799f7c0ac8352f56ea20193b2681532e5"
dependencies = [
"zerocopy-derive",
]
[[package]]
name = "zerocopy-derive"
version = "0.8.39"
version = "0.8.40"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4122cd3169e94605190e77839c9a40d40ed048d305bfdc146e7df40ab0f3e517"
checksum = "f65c489a7071a749c849713807783f70672b28094011623e200cb86dcb835953"
dependencies = [
"proc-macro2",
"quote",
@@ -18455,9 +18471,9 @@ dependencies = [
[[package]]
name = "zlib-rs"
version = "0.6.2"
version = "0.6.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c745c48e1007337ed136dc99df34128b9faa6ed542d80a1c673cf55a6d7236c8"
checksum = "3be3d40e40a133f9c916ee3f9f4fa2d9d63435b5fbe1bfc6d9dae0aa0ada1513"
[[package]]
name = "zstd"
+5 -5
View File
@@ -1,6 +1,6 @@
[package]
name = "windmill"
version = "1.645.0"
version = "1.649.0"
authors.workspace = true
edition.workspace = true
@@ -76,7 +76,7 @@ members = [
exclude = ["./windmill-duckdb-ffi-internal"]
[workspace.package]
version = "1.645.0"
version = "1.649.0"
authors = ["Ruben Fiszel <ruben@windmill.dev>"]
edition = "2021"
@@ -159,12 +159,12 @@ all_languages = ["python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "
# For windows we have another set of languages enabled
all_languages_windows = ["python", "deno_core", "rust", "mysql", "oracledb", "duckdb", "mssql-winauth", "bigquery", "csharp", "nu", "php", "java"]
# Edition meta-features: shared groups
inline_preview = ["windmill-api/inline_preview"]
run_inline = ["windmill-api/run_inline"]
oss_core = [
"embedding", "parquet", "openidconnect", "license",
"http_trigger", "zip", "oauth2", "postgres_trigger",
"mqtt_trigger", "websocket", "smtp", "native_trigger",
"static_frontend", "mcp", "bedrock", "inline_preview",
"static_frontend", "mcp", "bedrock", "run_inline",
"quickjs"
]
ce_core = ["oss_core", "private", "operator"]
@@ -351,7 +351,7 @@ tower-cookies = "^0.10"
serde = "=1.0.219"
serde_json = { version = "^1", features = ["preserve_order", "raw_value"] }
serde_yml = "0.0.12"
uuid = { version = "^1", features = ["serde", "v4"] }
uuid = { version = "^1", features = ["serde", "v4", "js"] }
thiserror = "^2"
anyhow = "^1"
chrono = { version = "^0.4", features = ["serde"] }
+1 -1
View File
@@ -1 +1 @@
a797dd4d619cdab737e133ce593f2f8582ba21de
9b3339730eb4bb0b564c7c56ac546f33fb3d8905
@@ -0,0 +1 @@
DROP INDEX IF EXISTS ix_v2_job_completed_failure_workspace;
@@ -0,0 +1,7 @@
-- Partial index for fast failure/canceled filtering on the runs page.
-- When failures are sparse (<1%) this avoids scanning millions of successful jobs.
-- The query orders by completed_at DESC (switched from created_at when success=false),
-- so this index provides both filtering and ordering in a single scan.
CREATE INDEX IF NOT EXISTS ix_v2_job_completed_failure_workspace
ON v2_job_completed (workspace_id, completed_at DESC)
WHERE status IN ('failure', 'canceled');
@@ -1,4 +1,4 @@
use std::collections::BTreeMap;
use std::collections::{BTreeMap, HashSet};
use sqlparser::{
ast::{
@@ -45,6 +45,10 @@ struct AssetCollector {
var_identifiers: BTreeMap<String, (AssetKind, String)>,
// e.g USE dl;
currently_used_asset: Option<(AssetKind, String)>,
// CTE names in scope (stack for nested queries)
cte_name_stack: Vec<HashSet<String>>,
// Locally created tables (not attached to an asset)
local_table_names: HashSet<String>,
}
impl AssetCollector {
@@ -54,9 +58,30 @@ impl AssetCollector {
current_access_type_stack: Vec::with_capacity(8),
var_identifiers: BTreeMap::new(),
currently_used_asset: None,
cte_name_stack: Vec::new(),
local_table_names: HashSet::new(),
}
}
/// If the name resolves to an attached asset, record it. Otherwise, register it as a local
/// table/view so that subsequent references are not mistakenly attributed to the active asset.
fn track_table_definition(&mut self, name: &ObjectName) {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(W)) {
self.assets.push(asset);
} else if let Some(simple_name) = get_trivial_obj_name(name) {
self.local_table_names.insert(simple_name.to_lowercase());
}
}
fn is_locally_defined(&self, name: &str) -> bool {
let name_lower = name.to_lowercase();
self.local_table_names.contains(&name_lower)
|| self
.cte_name_stack
.iter()
.any(|set| set.contains(&name_lower))
}
// Detect when we do 'a.b' and 'a' is associated with an asset in var_identifiers
// Or when we access 'b' and we did USE a;
fn get_associated_asset_from_obj_name(
@@ -72,6 +97,14 @@ impl AssetCollector {
return None;
}
if name.0.len() == 1 {
if let Some(ident) = name.0.first().and_then(|id| id.as_ident()) {
if self.is_locally_defined(&ident.value) {
return None;
}
}
}
if name.0.len() == 1 || name.0.len() == 2 {
if name
.0
@@ -452,6 +485,7 @@ impl Visitor for AssetCollector {
) -> std::ops::ControlFlow<Self::Break> {
match statement {
sqlparser::ast::Statement::Query(q) => {
self.cte_name_stack.push(collect_cte_names(q));
if let Some(select) = q.body.as_select() {
// First, handle table references (adds table-level assets)
for t in &select.from {
@@ -612,17 +646,11 @@ impl Visitor for AssetCollector {
}
sqlparser::ast::Statement::CreateTable(create_table) => {
if let Some(asset) =
self.get_associated_asset_from_obj_name(&create_table.name, Some(W))
{
self.assets.push(asset);
}
self.track_table_definition(&create_table.name);
}
sqlparser::ast::Statement::CreateView { name, .. } => {
if let Some(asset) = self.get_associated_asset_from_obj_name(name, Some(W)) {
self.assets.push(asset);
}
self.track_table_definition(name);
}
sqlparser::ast::Statement::Copy { target: CopyTarget::File { filename }, .. } => {
@@ -672,16 +700,20 @@ impl Visitor for AssetCollector {
fn post_visit_statement(
&mut self,
_statement: &sqlparser::ast::Statement,
statement: &sqlparser::ast::Statement,
) -> std::ops::ControlFlow<Self::Break> {
if matches!(statement, sqlparser::ast::Statement::Query(_)) {
self.cte_name_stack.pop();
}
std::ops::ControlFlow::Continue(())
}
fn pre_visit_query(
&mut self,
_query: &sqlparser::ast::Query,
query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.push(R);
self.cte_name_stack.push(collect_cte_names(query));
std::ops::ControlFlow::Continue(())
}
@@ -690,12 +722,22 @@ impl Visitor for AssetCollector {
_query: &sqlparser::ast::Query,
) -> std::ops::ControlFlow<Self::Break> {
self.current_access_type_stack.pop();
self.cte_name_stack.pop();
std::ops::ControlFlow::Continue(())
}
// We do not use pre_visit_relation because we cannot know if an ObjectName is a table or a function
}
fn collect_cte_names(query: &sqlparser::ast::Query) -> HashSet<String> {
query.with.as_ref().map_or_else(HashSet::new, |with| {
with.cte_tables
.iter()
.map(|cte| cte.alias.name.value.to_lowercase())
.collect()
})
}
fn is_read_fn(fname: &str) -> bool {
fname.eq_ignore_ascii_case("read_parquet")
|| fname.eq_ignore_ascii_case("read_csv")
@@ -1509,6 +1551,235 @@ mod tests {
assert!(result[0].columns.is_none());
}
#[test]
fn test_sql_asset_parser_cte_not_treated_as_asset() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1 AS x)
SELECT * FROM tmp;
SELECT * FROM real_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_scope_does_not_leak() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1) SELECT * FROM tmp;
SELECT * FROM tmp;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/tmp".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_multiple_ctes() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH cte1 AS (SELECT 1), cte2 AS (SELECT 2)
SELECT * FROM cte1 JOIN cte2 ON true;
SELECT * FROM real_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_create_table_overrides_asset() {
let input = r#"
CREATE TABLE local_tbl (id INT);
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
SELECT * FROM local_tbl;
SELECT * FROM asset_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/asset_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_create_table_with_use_is_still_asset() {
let input = r#"
ATTACH 'ducklake' AS dl; USE dl;
CREATE TABLE friends (
name text,
age int
);
INSERT INTO friends VALUES ($name, $age);
SELECT * FROM friends;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "main/friends".to_string(),
access_type: Some(RW),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_create_view_overrides_asset() {
let input = r#"
CREATE VIEW my_view AS SELECT 1;
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
SELECT * FROM my_view;
SELECT * FROM asset_table;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/asset_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_create_view_with_use_is_still_asset() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
CREATE VIEW my_view AS SELECT 1;
SELECT * FROM my_view;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/my_view".to_string(),
access_type: Some(RW),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_mixed_with_asset_tables() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH tmp AS (SELECT 1 AS x)
SELECT * FROM tmp JOIN real_table ON true;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_local_table_insert_and_select() {
let input = r#"
CREATE TABLE staging (id INT, val TEXT);
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
INSERT INTO staging VALUES (1, 'a');
SELECT * FROM staging;
INSERT INTO real_table VALUES (2, 'b');
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/real_table".to_string(),
access_type: Some(W),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_qualified_ref_bypasses_local() {
// Even if 'tbl' is local, 'dl.tbl' is an explicit asset reference
let input = r#"
CREATE TABLE tbl (id INT);
ATTACH 'ducklake://my_dl' AS dl;
SELECT * FROM dl.tbl;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl/tbl".to_string(),
access_type: Some(R),
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_cte_case_insensitive() {
let input = r#"
ATTACH 'ducklake://my_dl' AS dl;
USE dl;
WITH MyTable AS (SELECT 1)
SELECT * FROM mytable;
"#;
let s = parse_assets(input).map(|s| s.assets);
assert_eq!(
s.map_err(|e| e.to_string()),
Ok(vec![ParseAssetsResult {
kind: AssetKind::Ducklake,
path: "my_dl".to_string(),
access_type: None,
columns: None
},])
);
}
#[test]
fn test_sql_asset_parser_s3_read_csv_columns() {
let input = r#"
+260 -19
View File
@@ -58,16 +58,23 @@ pub fn parse_oracledb_sig(code: &str) -> anyhow::Result<MainArgSignature> {
}
pub fn parse_pgsql_sig(code: &str) -> anyhow::Result<MainArgSignature> {
let (sig, _) = parse_pgsql_sig_with_typed_schema(code)?;
Ok(sig)
}
pub fn parse_pgsql_sig_with_typed_schema(code: &str) -> anyhow::Result<(MainArgSignature, bool)> {
let parsed = parse_pg_file(&code)?;
if let Some(x) = parsed {
let args = x;
Ok(MainArgSignature {
star_args: false,
star_kwargs: false,
args,
no_main_func: None,
has_preprocessor: None,
})
if let Some((args, typed_schema)) = parsed {
Ok((
MainArgSignature {
star_args: false,
star_kwargs: false,
args,
no_main_func: None,
has_preprocessor: None,
},
typed_schema,
))
} else {
Err(anyhow!("Error parsing sql".to_string()))
}
@@ -216,7 +223,7 @@ lazy_static::lazy_static! {
static ref RE_ARG_MYSQL: Regex = Regex::new(r#"(?m)^-- \? (\w+) \((\w+)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
pub static ref RE_ARG_MYSQL_NAMED: Regex = Regex::new(r#"(?m)^-- :([a-z_][a-z0-9_]*) \((\w+(?:\([\w, ]+\))?)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
static ref RE_ARG_PGSQL: Regex = Regex::new(r#"(?m)^-- \$(\d+) (\w+)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
static ref RE_ARG_PGSQL: Regex = Regex::new(r#"(?m)^-- \$(\d+) (\w+)(?: \(([A-Za-z0-9_\[\]]+)\))?(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
// -- @name (type) = default
static ref RE_ARG_BIGQUERY: Regex = Regex::new(r#"(?m)^-- @(\w+) \((\w+(?:\[\])?)\)(?: ?\= ?(.+))? *(?:\r|\n|$)"#).unwrap();
@@ -478,21 +485,62 @@ pub fn parse_pg_statement_arg_indices(code: &str) -> HashSet<i32> {
arg_indices
}
fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
fn parse_pg_file(code: &str) -> anyhow::Result<Option<(Vec<Arg>, bool)>> {
let mut args = vec![];
// Track which args have explicit types in declaration comments
let mut explicitly_typed_args: HashSet<i32> = HashSet::new();
// First pass: collect args from declaration comments (-- $1 argName (type))
for cap in RE_ARG_PGSQL.captures_iter(code) {
let idx = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?;
let name = cap.get(2).map(|x| x.as_str().to_string()).unwrap();
let explicit_type = cap.get(3).map(|x| x.as_str().to_string().to_lowercase());
let default = cap.get(4).map(|x| x.as_str().to_string());
let has_default = default.is_some();
if let Some(typ) = explicit_type {
// If explicitly typed, use that type and don't infer from usage
explicitly_typed_args.insert(idx);
let parsed_typ = parse_pg_typ(typ.as_str());
let parsed_default = default.and_then(|x| parsed_default(&parsed_typ, x));
args.push(Arg {
name,
typ: parsed_typ,
default: parsed_default,
otyp: Some(typ),
has_default,
oidx: Some(idx),
});
}
}
// Second pass: infer types from usage for non-explicitly-typed args
let mut hm: HashMap<i32, String> = HashMap::new();
for cap in RE_CODE_PGSQL.captures_iter(code) {
let idx = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?;
// Skip if this arg was explicitly typed in declaration
if explicitly_typed_args.contains(&idx) {
continue;
}
let typ = cap
.get(2)
.map(|cap| transform_types_with_spaces(&cap, &code))
.unwrap_or("text");
hm.insert(
cap.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.ok_or_else(|| anyhow!("Impossible to parse arg digit"))?,
typ.to_string(),
);
hm.insert(idx, typ.to_string());
}
// Add inferred args
for (i, v) in hm.iter() {
let typ = v.to_lowercase();
args.push(Arg {
@@ -504,19 +552,28 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
oidx: Some(*i),
});
}
// Sort by index
args.sort_by(|a, b| a.oidx.unwrap().cmp(&b.oidx.unwrap()));
// Third pass: update names and defaults for inferred args
for cap in RE_ARG_PGSQL.captures_iter(code) {
let i = cap
.get(1)
.and_then(|x| x.as_str().parse::<i32>().ok())
.map(|x| x);
// Skip explicitly typed args (already handled)
if i.is_some_and(|idx| explicitly_typed_args.contains(&idx)) {
continue;
}
if let Some(arg_pos) = args
.iter()
.position(|x| i.is_some_and(|i| x.oidx.unwrap() == i))
{
let name = cap.get(2).map(|x| x.as_str().to_string()).unwrap();
let default = cap.get(3).map(|x| x.as_str().to_string());
let default = cap.get(4).map(|x| x.as_str().to_string());
let has_default = default.is_some();
let oarg = args[arg_pos].clone();
let parsed_default = default.and_then(|x| parsed_default(&oarg.typ, x));
@@ -532,8 +589,10 @@ fn parse_pg_file(code: &str) -> anyhow::Result<Option<Vec<Arg>>> {
}
}
let typed_schema = !explicitly_typed_args.is_empty();
args.append(&mut parse_sql_sanitized_interpolation(code));
Ok(Some(args))
Ok(Some((args, typed_schema)))
}
// The regex doesn't parse types with space such as "character varying"
@@ -1306,4 +1365,186 @@ SELECT * FROM table_name WHERE thing = :name4;
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_at_declaration() -> anyhow::Result<()> {
let code = r#"
-- $1 user_id (bigint)
-- $2 email
SELECT * FROM users WHERE id = $1 AND email = $2::text;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("bigint".to_string()),
name: "user_id".to_string(),
typ: Typ::Int,
default: None,
has_default: false,
oidx: Some(1),
},
Arg {
otyp: Some("text".to_string()),
name: "email".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(2),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_with_default() -> anyhow::Result<()> {
let code = r#"
-- $1 limit (integer) = 10
-- $2 offset (bigint) = 0
SELECT * FROM users LIMIT $1 OFFSET $2;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("integer".to_string()),
name: "limit".to_string(),
typ: Typ::Int,
default: Some(json!(10)),
has_default: true,
oidx: Some(1),
},
Arg {
otyp: Some("bigint".to_string()),
name: "offset".to_string(),
typ: Typ::Int,
default: Some(json!(0)),
has_default: true,
oidx: Some(2),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_mixed_explicit_and_inferred() -> anyhow::Result<()> {
let code = r#"
-- $1 user_id (bigint)
-- $2 status
-- $3 created_at (timestamptz)
SELECT * FROM users
WHERE id = $1
AND status = $2::text
AND created_at > $3;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![
Arg {
otyp: Some("bigint".to_string()),
name: "user_id".to_string(),
typ: Typ::Int,
default: None,
has_default: false,
oidx: Some(1),
},
Arg {
otyp: Some("text".to_string()),
name: "status".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(2),
},
Arg {
otyp: Some("timestamptz".to_string()),
name: "created_at".to_string(),
typ: Typ::Datetime,
default: None,
has_default: false,
oidx: Some(3),
},
],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_array() -> anyhow::Result<()> {
let code = r#"
-- $1 ids (bigint[])
SELECT * FROM users WHERE id = ANY($1);
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![Arg {
otyp: Some("bigint[]".to_string()),
name: "ids".to_string(),
typ: Typ::List(Box::new(Typ::Int)),
default: None,
has_default: false,
oidx: Some(1),
},],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
#[test]
fn test_parse_pgsql_explicit_type_does_not_infer_from_usage() -> anyhow::Result<()> {
// Even though $1 is used as ::integer in the query,
// the explicit type (text) should take precedence
let code = r#"
-- $1 value (text)
SELECT $1::integer;
"#;
assert_eq!(
parse_pgsql_sig(code)?,
MainArgSignature {
star_args: false,
star_kwargs: false,
args: vec![Arg {
otyp: Some("text".to_string()),
name: "value".to_string(),
typ: Typ::Str(None),
default: None,
has_default: false,
oidx: Some(1),
},],
no_main_func: None,
has_preprocessor: None
}
);
Ok(())
}
}
+6 -2
View File
@@ -126,6 +126,7 @@ pub fn json_to_typ(js: &Value, precise_arrays: bool) -> Typ {
pub fn to_snake_case(s: &str) -> String {
s.with_boundaries(&Boundary::defaults())
.without_boundaries(&Boundary::letter_digit())
.without_boundaries(&[Boundary::DigitLower])
.to_case(Case::Snake)
}
@@ -138,8 +139,8 @@ mod test {
assert_eq!("s3", to_snake_case("S3"));
assert_eq!("s3", to_snake_case("s3"));
assert_eq!("s3_object", to_snake_case("S3Object"));
assert_eq!("s3_object", to_snake_case("S3object"));
assert_eq!("s3_object", to_snake_case("s3object"));
assert_eq!("s3object", to_snake_case("S3object"));
assert_eq!("s3object", to_snake_case("s3object"));
assert_eq!("abc", to_snake_case("ABC"));
assert_eq!("aa_bc", to_snake_case("AaBC"));
assert_eq!("a_b_c", to_snake_case("A_B_C"));
@@ -181,6 +182,9 @@ mod test {
fn test_mixed_case_with_numbers() {
assert_eq!(to_snake_case("testCase1"), "test_case1");
assert_eq!(to_snake_case("Test123Case"), "test123_case");
// digit followed by lowercase should NOT insert underscore (issue #7934)
assert_eq!(to_snake_case("Connect2allApi"), "connect2all_api");
assert_eq!(to_snake_case("Foo2barApi"), "foo2bar_api");
}
#[test]
@@ -16,4 +16,7 @@ wasm-bindgen-test.workspace = true
[dependencies]
windmill-parser.workspace = true
windmill-parser-sql.workspace = true
wasm-bindgen.workspace = true
wasm-bindgen.workspace = true
# getrandom 0.3 is pulled in transitively by rand 0.9 (via windmill-types).
# It requires the "wasm_js" feature to work on wasm32-unknown-unknown.
getrandom3 = { package = "getrandom", version = "0.3", features = ["wasm_js"] }
+1 -1
View File
@@ -13,7 +13,7 @@ default = []
enterprise = ["dep:windmill-autoscaling"]
private = []
python = []
inline_preview = ["dep:windmill-worker", "dep:itertools"]
run_inline = ["dep:windmill-worker", "dep:itertools"]
[dependencies]
windmill-api-auth.workspace = true
+3 -3
View File
@@ -283,14 +283,14 @@ async fn native_kubernetes_autoscaling_healthcheck() -> Result<(), error::Error>
}
async fn list_available_python_versions() -> error::JsonResult<Vec<String>> {
#[cfg(not(all(feature = "python", feature = "inline_preview")))]
#[cfg(not(all(feature = "python", feature = "run_inline")))]
return Err(error::Error::BadRequest(
"Python listing available only with 'python' feature enabled".to_string(),
));
#[cfg(all(feature = "python", feature = "inline_preview"))]
#[cfg(all(feature = "python", feature = "run_inline"))]
use itertools::Itertools;
#[cfg(all(feature = "python", feature = "inline_preview"))]
#[cfg(all(feature = "python", feature = "run_inline"))]
return Ok(Json(
windmill_worker::PyV::list_available_python_versions()
.await
+16 -6
View File
@@ -146,15 +146,24 @@ async fn get_input_history(
"AND parent_job IS NULL"
};
let sql = &format!(
"select id, v2_job_completed.completed_at, created_by, 'null'::jsonb as args, status = 'success' as success from v2_job JOIN v2_job_completed USING (id) \
where v2_job.workspace_id = $3 and {} = $1 and kind = any($2) {args_query} AND v2_job_completed.status != 'skipped' {include_non_root} \
order by v2_job_completed.completed_at desc limit $4 offset $5",
r.runnable_type.column_name(),
// Two-step approach: first fetch 2*(per_page+offset) rows using created_at ordering
// (which leverages the ix_job_root_job_index_by_path_2 index on v2_job), then sort
// the small result set by completed_at. This works because created_at and completed_at
// are highly correlated.
let inner_limit = 2 * (per_page + offset);
let sql = &format!(
"SELECT id, completed_at, created_by, args, success FROM (\
SELECT id, v2_job_completed.completed_at, created_by, 'null'::jsonb as args, \
status = 'success' as success \
FROM v2_job JOIN v2_job_completed USING (id) \
WHERE v2_job.workspace_id = $3 AND {} = $1 AND kind = any($2) \
{args_query} AND v2_job_completed.status != 'skipped' {include_non_root} \
ORDER BY v2_job.created_at DESC LIMIT $4\
) t ORDER BY completed_at DESC LIMIT $5 OFFSET $6",
r.runnable_type.column_name(),
);
// tracing::info!("sql: {}", sql);
let query = sqlx::query_as::<_, CompletedJobMini>(sql);
let query = match r.runnable_type {
@@ -175,6 +184,7 @@ async fn get_input_history(
let rows = query
.bind(job_kinds)
.bind(&w_id)
.bind(inner_limit as i32)
.bind(per_page as i32)
.bind(offset as i32)
.fetch_all(&mut *tx)
@@ -14,6 +14,7 @@ private = ["windmill-test-utils/private", "dep:aws-config", "dep:aws-credential-
enterprise = ["windmill-test-utils/enterprise", "dep:base64"]
deno_core = ["windmill-test-utils/deno_core"]
mcp = []
run_inline = ["dep:windmill-worker", "windmill-test-utils/run_inline", "windmill-test-utils/duckdb"]
[dependencies]
windmill-test-utils.workspace = true
@@ -21,6 +22,7 @@ windmill-api-client.workspace = true
windmill-common = { workspace = true, default-features = false }
windmill-native-triggers = { workspace = true, features = ["native_trigger"] }
windmill-api-auth.workspace = true
windmill-worker = { workspace = true, optional = true }
sqlx.workspace = true
serde_json.workspace = true
serde.workspace = true
@@ -0,0 +1,258 @@
use serde_json::json;
use sqlx::{Pool, Postgres};
use windmill_test_utils::*;
#[cfg(feature = "run_inline")]
async fn init_inline_utils(port: u16) -> anyhow::Result<()> {
use std::sync::Once;
static INIT: Once = Once::new();
INIT.call_once(|| {
let (killpill_tx, killpill_rx) = windmill_common::KillpillSender::new(1);
let base_internal_url = format!("http://localhost:{}", port);
windmill_worker::init_worker_internal_server_inline_utils(killpill_rx, base_internal_url)
.expect("Failed to initialize inline utils");
// Keep killpill_tx alive for the test duration
std::mem::forget(killpill_tx);
});
Ok(())
}
fn run_inline_url(port: u16, endpoint: &str) -> String {
format!("http://localhost:{port}/api/w/test-workspace/jobs/run_inline/{endpoint}")
}
fn client() -> reqwest::Client {
reqwest::Client::new()
}
fn authed(builder: reqwest::RequestBuilder) -> reqwest::RequestBuilder {
builder.header("Authorization", "Bearer SECRET_TOKEN")
}
fn new_script(
path: &str,
summary: &str,
content: &str,
language: &str,
schema_properties: serde_json::Value,
) -> serde_json::Value {
json!({
"path": path,
"summary": summary,
"description": "",
"content": content,
"language": language,
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": schema_properties,
"required": []
}
})
}
#[cfg(feature = "run_inline")]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_run_inline_by_path(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace");
// Initialize inline utils for script execution
init_inline_utils(port).await?;
// Create a DuckDB script (one of the languages that supports inline execution)
// DuckDB requires parameter declarations in comments: -- $param_name (type)
let script_path = "u/test-user/inline_test";
let script_content = "-- $x (integer)
-- $y (integer)
SELECT $x + $y as result";
let resp = authed(client().post(format!("{base}/scripts/create")))
.json(&new_script(
script_path,
"Inline test script",
script_content,
"duckdb",
json!({
"x": {"type": "integer"},
"y": {"type": "integer"}
}),
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 201, "create script: {}", resp.text().await?);
// Test run_inline by path with args
let resp = authed(client().post(run_inline_url(port, &format!("p/{script_path}"))))
.json(&json!({
"args": {
"x": 5,
"y": 15
}
}))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200, "run_inline by path with args failed");
let result = resp.json::<serde_json::Value>().await?;
// DuckDB query should return array with one row containing result field
// The result structure is [{"result": 20}]
assert!(result.is_array(), "expected array result, got: {}", result);
let rows = result.as_array().unwrap();
assert_eq!(rows.len(), 1, "expected 1 row, got: {}", rows.len());
assert_eq!(
rows[0]["result"],
json!(20),
"expected result 20, got: {}",
rows[0]
);
Ok(())
}
#[cfg(feature = "run_inline")]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_run_inline_by_hash(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
let base = format!("http://localhost:{port}/api/w/test-workspace");
// Initialize inline utils for script execution
init_inline_utils(port).await?;
// Create a DuckDB script and get its hash
let script_path = "u/test-user/inline_hash_test";
let script_content = "-- $a (integer)
-- $b (integer)
SELECT $a * $b as product";
let resp = authed(client().post(format!("{base}/scripts/create")))
.json(&new_script(
script_path,
"Inline hash test script",
script_content,
"duckdb",
json!({
"a": {"type": "integer"},
"b": {"type": "integer"}
}),
))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 201, "create script: {}", resp.text().await?);
// Get the script to retrieve its hash
let resp = authed(client().get(format!("{base}/scripts/get/p/{script_path}")))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200);
let script_data = resp.json::<serde_json::Value>().await?;
let hash = script_data["hash"]
.as_str()
.expect("hash should be present");
// Test run_inline by hash with args
let resp = authed(client().post(run_inline_url(port, &format!("h/{hash}"))))
.json(&json!({
"args": {
"a": 7,
"b": 3
}
}))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200, "run_inline by hash with args failed");
let result = resp.json::<serde_json::Value>().await?;
// Should return array with one row: [{"product": 21}]
assert!(result.is_array(), "expected array result");
let rows = result.as_array().unwrap();
assert_eq!(rows.len(), 1, "expected 1 row");
assert_eq!(
rows[0]["product"],
json!(21),
"expected product 21, got: {}",
rows[0]
);
Ok(())
}
#[cfg(feature = "run_inline")]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_run_inline_preview(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
// Initialize inline utils for script execution
init_inline_utils(port).await?;
// Test run_inline preview with direct DuckDB content
let resp = authed(client().post(run_inline_url(port, "preview")))
.json(&json!({
"content": "-- $msg (text)\nSELECT 'Hello, ' || $msg || '!' as greeting",
"language": "duckdb",
"args": {
"msg": "World"
}
}))
.send()
.await
.unwrap();
assert_eq!(resp.status(), 200, "run_inline preview failed");
let result = resp.json::<serde_json::Value>().await?;
// Should return array with one row: [{"greeting": "Hello, World!"}]
assert!(result.is_array(), "expected array result");
let rows = result.as_array().unwrap();
assert_eq!(rows.len(), 1, "expected 1 row");
assert_eq!(
rows[0]["greeting"],
json!("Hello, World!"),
"expected 'Hello, World!', got: {}",
rows[0]
);
Ok(())
}
#[cfg(feature = "run_inline")]
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_run_inline_nonexistent_script(db: Pool<Postgres>) -> anyhow::Result<()> {
initialize_tracing().await;
let server = ApiServer::start(db.clone()).await?;
let port = server.addr.port();
// Initialize inline utils
init_inline_utils(port).await?;
// Test run_inline by path with non-existent script - should return an error
let resp = authed(client().post(run_inline_url(port, "p/u/test-user/nonexistent_script")))
.json(&json!({
"args": null
}))
.send()
.await
.unwrap();
// Should return an error (script not found)
assert!(
resp.status().is_client_error() || resp.status().is_server_error(),
"expected error status for nonexistent script, got: {}",
resp.status()
);
Ok(())
}
+4 -1
View File
@@ -556,7 +556,10 @@ pub fn list_completed_jobs_query(
let mut sqlb = SqlBuilder::select_from("v2_job_completed")
.fields(fields)
.order_by(
if lq.completed_before.is_some() || lq.completed_after.is_some() {
if lq.completed_before.is_some()
|| lq.completed_after.is_some()
|| lq.success == Some(false)
{
"v2_job_completed.completed_at"
} else {
"v2_job.created_at"
@@ -20,3 +20,4 @@ sqlx.workspace = true
tar.workspace = true
tower-http.workspace = true
tracing.workspace = true
url.workspace = true
+25 -8
View File
@@ -131,12 +131,29 @@ pub fn workspaced_service() -> Router {
)
}
fn build_registry_request(url: &str, auth_token: &Option<String>) -> reqwest::RequestBuilder {
fn build_registry_request(
url: &str,
auth_token: &Option<String>,
registry_base_url: &str,
) -> Result<reqwest::RequestBuilder> {
let parsed_url =
url::Url::parse(url).map_err(|e| Error::BadRequest(format!("Invalid URL: {}", e)))?;
let parsed_base = url::Url::parse(registry_base_url)
.map_err(|e| Error::BadRequest(format!("Invalid registry URL: {}", e)))?;
if parsed_url.host_str() != parsed_base.host_str() {
return Err(Error::BadRequest(format!(
"Tarball URL host '{}' does not match registry host '{}'",
parsed_url.host_str().unwrap_or("unknown"),
parsed_base.host_str().unwrap_or("unknown"),
)));
}
let mut req = HTTP_CLIENT.get(url);
if let Some(token) = auth_token {
req = req.bearer_auth(token);
}
req
Ok(req)
}
/// Get package metadata (versions and tags) from the private registry
@@ -153,7 +170,7 @@ async fn get_package_metadata(
tracing::info!("Fetching package metadata from: {}", package_url);
let response = build_registry_request(&package_url, &auth_token)
let response = build_registry_request(&package_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to fetch package metadata: {}", e)))?;
@@ -204,7 +221,7 @@ async fn resolve_package_version(
tracing::info!("Resolving package version from: {}", package_url);
let response = build_registry_request(&package_url, &auth_token)
let response = build_registry_request(&package_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to fetch package metadata: {}", e)))?;
@@ -258,7 +275,7 @@ async fn get_package_filetree(
tracing::info!("Fetching package filetree from: {}", package_url);
let response = build_registry_request(&package_url, &auth_token)
let response = build_registry_request(&package_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to fetch package metadata: {}", e)))?;
@@ -283,7 +300,7 @@ async fn get_package_filetree(
.and_then(|t| t.as_str())
.ok_or_else(|| Error::NotFound(format!("Tarball not found for {}@{}", package, version)))?;
let tarball_response = build_registry_request(tarball_url, &auth_token)
let tarball_response = build_registry_request(tarball_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to download tarball: {}", e)))?;
@@ -329,7 +346,7 @@ async fn get_package_file(
tracing::info!("Fetching package file from: {}", package_url);
let response = build_registry_request(&package_url, &auth_token)
let response = build_registry_request(&package_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to fetch package metadata: {}", e)))?;
@@ -354,7 +371,7 @@ async fn get_package_file(
.and_then(|t| t.as_str())
.ok_or_else(|| Error::NotFound(format!("Tarball not found for {}@{}", package, version)))?;
let tarball_response = build_registry_request(tarball_url, &auth_token)
let tarball_response = build_registry_request(tarball_url, &auth_token, &registry_url)?
.send()
.await
.map_err(|e| Error::InternalErr(format!("Failed to download tarball: {}", e)))?;
@@ -2863,9 +2863,8 @@ async fn clone_workspace_data(
// Clone workspace runnable dependencies and dependency map
clone_workspace_runnable_dependencies(tx, source_workspace_id, target_workspace_id).await?;
// TODO: Enable when git sync is implemented for workspace dependencies.
// // Clone workspace dependencies
// clone_workspace_dependencies(tx, source_workspace_id, target_workspace_id).await?;
// Clone workspace dependencies
clone_workspace_dependencies(tx, source_workspace_id, target_workspace_id).await?;
Ok(())
}
@@ -3362,13 +3361,12 @@ async fn clone_workspace_runnable_dependencies(
Ok(())
}
#[allow(dead_code)]
async fn clone_workspace_dependencies(
tx: &mut Transaction<'_, Postgres>,
source_workspace_id: &str,
target_workspace_id: &str,
) -> Result<()> {
// Clone workspace_runnable_dependencies
// Clone workspace_dependencies
sqlx::query!(
"INSERT INTO workspace_dependencies (workspace_id, language, name, description, content, archived, created_at)
SELECT $1, language, name, description, content, archived, created_at
@@ -3502,17 +3500,6 @@ async fn create_workspace_fork(
// Clone all data from the parent workspace using Rust implementation
clone_workspace_data(&mut tx, &parent_workspace_id, &forked_id).await?;
sqlx::query!(
"INSERT INTO workspace_invite (workspace_id, email, is_admin, operator)
SELECT $1, email, is_admin, operator
FROM usr
WHERE workspace_id = $2",
&forked_id,
&parent_workspace_id
)
.execute(&mut *tx)
.await?;
audit_log(
&mut *tx,
&authed,
+1 -1
View File
@@ -13,7 +13,7 @@ default = []
private = ["windmill-audit/private", "windmill-common/private", "windmill-api-auth/private", "windmill-store/private", "windmill-api-users/private", "windmill-api-workspaces/private", "windmill-api-groups/private", "windmill-api-configs/private", "windmill-api-settings/private", "windmill-api-agent-workers?/private", "windmill-trigger-kafka?/private", "windmill-trigger-postgres?/private", "windmill-trigger-mqtt?/private", "windmill-trigger-websocket?/private", "windmill-trigger-nats?/private", "windmill-trigger-sqs?/private", "windmill-trigger-gcp?/private", "windmill-trigger-email?/private"]
enterprise = ["windmill-queue/enterprise", "windmill-audit/enterprise", "windmill-git-sync/enterprise", "windmill-common/enterprise", "windmill-worker?/enterprise", "windmill-api-auth/enterprise", "windmill-store/enterprise", "windmill-api-jobs/enterprise", "windmill-api-scripts/enterprise", "windmill-api-flows/enterprise", "windmill-api-users/enterprise", "windmill-api-workspaces/enterprise", "windmill-api-groups/enterprise", "windmill-api-configs/enterprise", "windmill-api-settings/enterprise", "windmill-api-agent-workers?/enterprise", "windmill-trigger/enterprise", "windmill-trigger-kafka?/enterprise", "windmill-trigger-postgres?/enterprise", "windmill-trigger-mqtt?/enterprise", "windmill-trigger-websocket?/enterprise", "windmill-trigger-email?/enterprise", "windmill-trigger-nats?/enterprise", "windmill-trigger-sqs?/enterprise", "windmill-trigger-gcp?/enterprise", "windmill-trigger-http?/enterprise", "windmill-native-triggers?/enterprise", "dep:windmill-autoscaling", "windmill-autoscaling/enterprise"]
stripe = []
inline_preview = ["dep:windmill-worker", "windmill-api-configs/inline_preview"]
run_inline = ["dep:windmill-worker", "windmill-api-configs/run_inline"]
agent_worker_server = ["dep:windmill-worker", "dep:windmill-api-agent-workers"]
enterprise_saml = ["dep:samael", "dep:libxml"]
benchmark = []
+148 -4
View File
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
version: 1.645.0
version: 1.649.0
title: Windmill API
contact:
@@ -9173,6 +9173,54 @@ paths:
application/json:
schema: {}
/w/{workspace}/jobs/run_inline/p/{path}:
post:
summary: run script by path without starting a new job
operationId: runScriptByPathInline
tags:
- job
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/ScriptPath"
requestBody:
description: script args
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/InlineScriptArgs"
responses:
"200":
description: script result
content:
application/json:
schema: {}
/w/{workspace}/jobs/run_inline/h/{hash}:
post:
summary: run script by hash without starting a new job
operationId: runScriptByHashInline
tags:
- job
parameters:
- $ref: "#/components/parameters/WorkspaceId"
- $ref: "#/components/parameters/ScriptHash"
requestBody:
description: script args
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/InlineScriptArgs"
responses:
"200":
description: script result
content:
application/json:
schema: {}
/w/{workspace}/jobs/run_wait_result/preview:
post:
summary: run script preview and wait for result
@@ -16925,9 +16973,9 @@ paths:
description: count of log lines that matched the query per hostname
type: object
/srch/index/delete/{idx_name}:
/indexer/delete/{idx_name}:
delete:
summary: Restart container and delete the index to recreate it.
summary: Clear an index and restart the indexer.
operationId: clearIndex
tags:
- indexSearch
@@ -16942,12 +16990,102 @@ paths:
- ServiceLogIndex
responses:
"200":
description: idx to be deleted and container restarting
description: idx to be deleted and indexer restarting
content:
text/plain:
schema:
type: string
/indexer/storage:
get:
summary: Get index storage sizes (disk and S3).
operationId: getIndexStorageSizes
tags:
- indexSearch
responses:
"200":
description: storage sizes for each index
content:
application/json:
schema:
type: object
properties:
job_index:
type: object
properties:
disk_size_bytes:
type: integer
nullable: true
s3_size_bytes:
type: integer
nullable: true
service_log_index:
type: object
properties:
disk_size_bytes:
type: integer
nullable: true
s3_size_bytes:
type: integer
nullable: true
/indexer/status:
get:
summary: Get indexer status including liveness and storage sizes.
operationId: getIndexerStatus
tags:
- indexSearch
responses:
"200":
description: indexer status for each index
content:
application/json:
schema:
type: object
properties:
job_indexer:
type: object
properties:
is_alive:
type: boolean
last_locked_at:
type: string
format: date-time
nullable: true
owner:
type: string
nullable: true
storage:
type: object
properties:
disk_size_bytes:
type: integer
nullable: true
s3_size_bytes:
type: integer
nullable: true
log_indexer:
type: object
properties:
is_alive:
type: boolean
last_locked_at:
type: string
format: date-time
nullable: true
owner:
type: string
nullable: true
storage:
type: object
properties:
disk_size_bytes:
type: integer
nullable: true
s3_size_bytes:
type: integer
nullable: true
/w/{workspace}/assets/list:
get:
summary: List all assets in the workspace with cursor pagination
@@ -19811,6 +19949,12 @@ components:
$ref: "#/components/schemas/ScriptLang"
required: [content, args, language]
InlineScriptArgs:
type: object
properties:
args:
$ref: "#/components/schemas/ScriptArgs"
WorkflowTask:
type: object
properties:
+3
View File
@@ -81,6 +81,9 @@ lazy_static::lazy_static! {
(20260225100000, include_str!(
"../../migrations/20260225100000_asset_covering_index.up.sql"
).replace("CREATE INDEX", "CREATE INDEX CONCURRENTLY").replace("DROP INDEX", "DROP INDEX CONCURRENTLY")),
(20260228000000, include_str!(
"../../migrations/20260228000000_v2_job_completed_failure_index.up.sql"
).replace("CREATE INDEX", "CREATE INDEX CONCURRENTLY")),
].into_iter().collect();
}
+5
View File
@@ -14,3 +14,8 @@ pub fn workspaced_service() -> Router {
pub fn global_service() -> Router {
Router::new()
}
#[cfg(not(feature = "private"))]
pub fn management_service() -> Router {
Router::new()
}
+132 -11
View File
@@ -27,20 +27,22 @@ use url::Url;
#[cfg(all(feature = "enterprise", feature = "smtp"))]
use windmill_common::auth::is_super_admin_email;
use windmill_common::auth::TOKEN_PREFIX_LEN;
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
use windmill_common::client::AuthedClient;
use windmill_common::db::UserDbWithAuthed;
use windmill_common::error::JsonResult;
use windmill_common::flow_status::{JobResult, RestartedFrom};
#[cfg(feature = "inline_preview")]
use windmill_common::jobs::RunInlinePreviewScriptFnParams;
use windmill_common::jobs::{
format_completed_job_result, format_result, DynamicInput, ENTRYPOINT_OVERRIDE,
};
#[cfg(feature = "run_inline")]
use windmill_common::jobs::{
InlineScriptTarget, RunInlinePreviewScriptFnParams, RunInlineScriptFnParams,
};
use windmill_common::runnable_settings::{
ConcurrencySettings, ConcurrencySettingsWithCustom, DebouncingSettings,
};
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
use windmill_common::runtime_assets::{register_runtime_asset, InsertRuntimeAssetParams};
use windmill_common::scripts::ScriptRunnableSettingsInline;
use windmill_common::triggers::TriggerMetadata;
@@ -53,15 +55,15 @@ use windmill_common::DYNAMIC_INPUT_CACHE;
#[cfg(all(feature = "enterprise", feature = "smtp"))]
use windmill_common::{email_oss::send_email_html, server::load_smtp_config};
use windmill_object_store::upload_artifact_to_store;
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
use windmill_parser::asset_parser::AssetKind;
use windmill_types::s3::BundleFormat;
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
use windmill_worker::get_worker_internal_server_inline_utils;
use windmill_common::variables::get_workspace_key;
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
use crate::db::OptJobAuthed;
use crate::triggers::trigger_helpers::{FlowId, ScriptId};
use crate::{
@@ -242,6 +244,11 @@ pub fn workspaced_service() -> Router {
)
.route("/run/preview", post(run_preview_script))
.route("/run_inline/preview", post(run_inline_preview_script))
.route(
"/run_inline/p/*script_path",
post(run_inline_script_by_path),
)
.route("/run_inline/h/:hash", post(run_inline_script_by_hash))
.route(
"/run_wait_result/preview",
post(run_wait_result_preview_script),
@@ -2055,6 +2062,8 @@ async fn list_jobs(
let sql = if lq.success.is_none()
&& lq.label.is_none()
&& lq.result.is_none()
&& !lq.is_skipped.unwrap_or(false)
&& lq.created_before.is_none()
&& lq.started_before.is_none()
&& lq.created_or_started_before.is_none()
@@ -2853,7 +2862,7 @@ struct Preview {
flow_path: Option<String>,
}
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
#[derive(Debug, Deserialize)]
struct PreviewInline {
content: String,
@@ -2861,6 +2870,12 @@ struct PreviewInline {
language: ScriptLang,
}
#[cfg(feature = "run_inline")]
#[derive(Debug, Deserialize)]
struct InlineScriptArgs {
args: Option<HashMap<String, Box<JsonRawValue>>>,
}
#[derive(Deserialize)]
pub struct WorkflowTask {
pub args: Option<HashMap<String, Box<JsonRawValue>>>,
@@ -4573,7 +4588,7 @@ async fn run_preview_script(
Ok((StatusCode::CREATED, uuid.to_string()))
}
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
async fn run_inline_preview_script(
OptJobAuthed { authed, job_id }: OptJobAuthed,
Tokened { token }: Tokened,
@@ -4610,14 +4625,120 @@ async fn run_inline_preview_script(
Ok(Json(to_raw_value(&result)).into_response())
}
#[cfg(not(feature = "inline_preview"))]
#[cfg(not(feature = "run_inline"))]
async fn run_inline_preview_script() -> error::Result<Response> {
Err(error::Error::InternalErr(
"inline preview requires the worker feature".to_string(),
))
}
#[cfg(feature = "inline_preview")]
#[cfg(feature = "run_inline")]
async fn run_inline_script_by_path(
OptJobAuthed { authed, .. }: OptJobAuthed,
Tokened { token }: Tokened,
Extension(db): Extension<DB>,
Extension(user_db): Extension<UserDB>,
Path((w_id, script_path)): Path<(String, StripPath)>,
Json(body): Json<InlineScriptArgs>,
) -> error::Result<Response> {
let script_path_str = script_path.to_path();
check_scopes(&authed, || format!("jobs:run:scripts:{script_path_str}"))?;
run_inline_script_inner(
authed,
token,
db,
w_id,
InlineScriptTarget::Path(script_path.to_path().to_string()),
body.args,
Some(user_db),
)
.await
}
#[cfg(not(feature = "run_inline"))]
async fn run_inline_script_by_path() -> error::Result<Response> {
Err(error::Error::InternalErr(
"inline script by path requires the worker feature".to_string(),
))
}
#[cfg(feature = "run_inline")]
async fn run_inline_script_by_hash(
OptJobAuthed { authed, .. }: OptJobAuthed,
Tokened { token }: Tokened,
Extension(db): Extension<DB>,
Extension(user_db): Extension<UserDB>,
Path((w_id, script_hash)): Path<(String, ScriptHash)>,
Json(body): Json<InlineScriptArgs>,
) -> error::Result<Response> {
// Resolve the script path from the hash and check scopes properly
let hash = script_hash.0;
let userdb_authed = UserDbWithAuthed { db: user_db.clone(), authed: &authed.to_authed_ref() };
let ScriptHashInfo { path, .. } =
get_script_info_for_hash(Some(userdb_authed), &db, &w_id, hash)
.await?
.prefetch_cached(&db)
.await?;
check_scopes(&authed, || format!("jobs:run:scripts:{path}"))?;
run_inline_script_inner(
authed,
token,
db,
w_id,
InlineScriptTarget::Hash(hash),
body.args,
Some(user_db),
)
.await
}
#[cfg(not(feature = "run_inline"))]
async fn run_inline_script_by_hash() -> error::Result<Response> {
Err(error::Error::InternalErr(
"inline script by hash requires the worker feature".to_string(),
))
}
#[cfg(feature = "run_inline")]
async fn run_inline_script_inner(
authed: ApiAuthed,
token: String,
db: DB,
w_id: String,
target: InlineScriptTarget,
args: Option<HashMap<String, Box<JsonRawValue>>>,
user_db: Option<UserDB>,
) -> error::Result<Response> {
let utils = get_worker_internal_server_inline_utils()?;
let authed_owned: windmill_common::db::Authed = authed.clone().into();
let result = utils.run_inline_script.as_ref()(RunInlineScriptFnParams {
target,
args,
workspace_id: w_id.clone(),
base_internal_url: utils.base_internal_url.clone(),
killpill_rx: utils.killpill_rx.resubscribe(),
created_by: authed.display_username().to_string(),
permissioned_as: username_to_permissioned_as(&authed.username),
permissioned_as_email: authed.email.clone(),
job_dir: "".to_string(),
worker_name: "".to_string(),
worker_dir: "".to_string(),
client: AuthedClient {
base_internal_url: utils.base_internal_url.clone(),
force_client: None,
token,
workspace: w_id,
},
conn: windmill_common::worker::Connection::Sql(db),
user_db: user_db.map(|udb| (udb, authed_owned)),
})
.await?;
Ok(Json(to_raw_value(&result)).into_response())
}
#[cfg(feature = "run_inline")]
fn register_potential_assets_on_inline_execution(
job_id: Uuid,
w_id: &str,
+10 -7
View File
@@ -58,10 +58,7 @@ use windmill_common::db::UserDB;
use windmill_common::worker::CLOUD_HOSTED;
#[allow(unused_imports)]
pub(crate) use windmill_common::BASE_URL;
use windmill_common::{
utils::GIT_VERSION,
INSTANCE_NAME,
};
use windmill_common::{utils::GIT_VERSION, INSTANCE_NAME};
use crate::scim_oss::has_scim_token;
use windmill_common::error::AppError;
@@ -550,6 +547,7 @@ pub async fn run_server(
.nest("/embeddings", embeddings::global_service())
.nest("/ai", ai::global_service())
.nest("/inkeep", inkeep_oss::global_service())
.nest("/indexer", indexer_oss::management_service())
.nest("/mcp/w/:workspace_id/list_tools", mcp_list_tools_service)
.nest("/health/detailed", health::detailed_service())
.route_layer(from_extractor::<ApiAuthed>())
@@ -612,8 +610,10 @@ pub async fn run_server(
if let Some(agent_workers_job_completed_tx) =
agent_workers_job_completed_tx.clone()
{
windmill_api_agent_workers::global_service(agent_workers_job_completed_tx)
.layer(Extension(agent_cache.clone()))
windmill_api_agent_workers::global_service(
agent_workers_job_completed_tx,
)
.layer(Extension(agent_cache.clone()))
} else {
Router::new()
}
@@ -785,7 +785,10 @@ pub async fn run_server(
},
)
// JWKS endpoint for HashiCorp Vault JWT authentication (must be outside /api prefix)
.route("/.well-known/jwks.json", get(windmill_api_settings::get_jwks))
.route(
"/.well-known/jwks.json",
get(windmill_api_settings::get_jwks),
)
.fallback(static_assets::static_handler)
.layer(middleware_stack);
+15
View File
@@ -17,6 +17,21 @@ pub struct Authed {
pub token_prefix: Option<String>,
}
impl Authed {
pub fn to_authed_ref(&self) -> AuthedRef<'_> {
AuthedRef {
email: &self.email,
username: &self.username,
is_admin: &self.is_admin,
is_operator: &self.is_operator,
groups: &self.groups,
folders: &self.folders,
scopes: &self.scopes,
token_prefix: &self.token_prefix,
}
}
}
#[derive(Clone, Debug, Hash)]
pub struct AuthedRef<'a> {
pub email: &'a str,
+162 -9
View File
@@ -285,7 +285,7 @@ pub struct GlobalSettings {
#[serde(skip_serializing_if = "Option::is_none")]
pub indexer_settings: Option<IndexerSettings>,
#[serde(skip_serializing_if = "Option::is_none")]
pub oauths: Option<BTreeMap<String, OAuthClient>>,
pub oauths: Option<BTreeMap<String, OAuthClientEntry>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub otel: Option<OtelSettings>,
#[serde(skip_serializing_if = "Option::is_none")]
@@ -406,6 +406,49 @@ pub struct IndexerSettings {
// OAuth
// ---------------------------------------------------------------------------
/// Wrapper that tries to deserialize as a typed [`OAuthClient`] and falls
/// back to a raw JSON value when the stored config contains unexpected types
/// (e.g. `"true"` instead of `true` for a boolean field). This prevents the
/// entire `/api/settings/instance_config` endpoint from failing because of
/// one malformed provider entry.
#[derive(Serialize, Clone, Debug)]
#[cfg_attr(feature = "instance_config_schema", derive(schemars::JsonSchema))]
#[serde(untagged)]
pub enum OAuthClientEntry {
Typed(OAuthClient),
/// Fallback: the raw JSON value plus the error that prevented typed parsing.
Raw {
value: serde_json::Value,
/// Human-readable deserialization error (not serialized).
#[serde(skip)]
error: String,
},
}
impl<'de> serde::Deserialize<'de> for OAuthClientEntry {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let raw = serde_json::Value::deserialize(deserializer)?;
match serde_json::from_value::<OAuthClient>(raw.clone()) {
Ok(client) => Ok(OAuthClientEntry::Typed(client)),
Err(e) => Ok(OAuthClientEntry::Raw { value: raw, error: e.to_string() }),
}
}
}
impl OAuthClientEntry {
/// Returns a mutable reference to the inner [`OAuthClient`] if this entry
/// was successfully parsed, or `None` for raw/fallback entries.
pub fn as_typed_mut(&mut self) -> Option<&mut OAuthClient> {
match self {
OAuthClientEntry::Typed(c) => Some(c),
OAuthClientEntry::Raw { .. } => None,
}
}
}
/// OAuth client configuration for a single provider.
#[derive(Deserialize, Serialize, Clone, Debug)]
#[cfg_attr(feature = "instance_config_schema", derive(schemars::JsonSchema))]
@@ -413,13 +456,23 @@ pub struct OAuthClient {
pub id: String,
pub secret: StringOrSecretRef,
#[serde(skip_serializing_if = "Option::is_none")]
pub display_name: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub allowed_domains: Option<Vec<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub connect_config: Option<OAuthConfig>,
#[serde(skip_serializing_if = "Option::is_none")]
pub login_config: Option<OAuthConfig>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tenant: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub share_with_workspaces: Option<bool>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub grant_types: Vec<String>,
/// Catch-all for provider-specific fields (e.g. Keycloak `org`, Okta `domain`/`custom`).
#[serde(flatten)]
pub extra: BTreeMap<String, serde_json::Value>,
}
/// OAuth provider endpoint configuration.
@@ -438,6 +491,8 @@ pub struct OAuthConfig {
pub extra_params_callback: Option<BTreeMap<String, String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub req_body_auth: Option<bool>,
#[serde(default, skip_serializing_if = "Vec::is_empty")]
pub grant_types: Vec<String>,
}
// ---------------------------------------------------------------------------
@@ -1158,8 +1213,18 @@ pub fn resolve_env_refs(settings: &mut GlobalSettings) -> Result<(), String> {
}
if let Some(oauths) = &mut settings.oauths {
for oauth in oauths.values_mut() {
resolve_env_field(&mut oauth.secret)?;
for (name, entry) in oauths.iter_mut() {
match entry {
OAuthClientEntry::Typed(oauth) => {
resolve_env_field(&mut oauth.secret)?;
}
OAuthClientEntry::Raw { error, .. } => {
tracing::error!(
"OAuth entry '{name}' could not be deserialized as OAuthClient \
and was kept as raw JSON it likely has an unexpected shape: {error}"
);
}
}
}
}
@@ -1926,6 +1991,88 @@ mod tests {
assert_eq!(parsed.id, client.id);
}
#[test]
fn oauth_entry_valid_config_deserializes_as_typed() {
let json = r#"{
"id": "slack_id",
"secret": "slack_secret",
"connect_config": {
"auth_url": "https://slack.com/oauth/v2/authorize",
"token_url": "https://slack.com/api/oauth.v2.access",
"scopes": ["channels:read", "chat:write"],
"req_body_auth": true
}
}"#;
let entry: OAuthClientEntry = serde_json::from_str(json).unwrap();
match &entry {
OAuthClientEntry::Typed(c) => {
assert_eq!(c.id, "slack_id");
let cc = c.connect_config.as_ref().unwrap();
assert_eq!(cc.req_body_auth, Some(true));
}
OAuthClientEntry::Raw { .. } => panic!("expected Typed variant"),
}
// round-trip preserves the value
let serialized = serde_json::to_string(&entry).unwrap();
assert!(serialized.contains("slack_id"));
}
#[test]
fn oauth_entry_string_bool_falls_back_to_raw_with_error() {
// This is the real-world scenario: req_body_auth stored as "true" (string)
let json = r#"{
"id": "custom_provider",
"secret": "secret",
"connect_config": {
"auth_url": "https://example.com/auth",
"token_url": "https://example.com/token",
"req_body_auth": "true"
}
}"#;
let entry: OAuthClientEntry = serde_json::from_str(json).unwrap();
match &entry {
OAuthClientEntry::Raw { error, .. } => {
assert!(
error.contains("invalid type"),
"error should mention type mismatch, got: {error}"
);
}
OAuthClientEntry::Typed(_) => panic!("expected Raw fallback for string bool"),
}
// Serialization still round-trips the raw JSON faithfully
let serialized = serde_json::to_string(&entry).unwrap();
assert!(serialized.contains("custom_provider"));
assert!(serialized.contains(r#""req_body_auth":"true""#));
}
#[test]
fn oauth_entry_map_with_mixed_valid_and_invalid() {
// Simulates what InstanceConfig.oauths looks like when one provider is
// well-formed and another has a string-typed boolean.
let json = r#"{
"google": {
"id": "google_id",
"secret": "google_secret",
"connect_config": {
"auth_url": "https://accounts.google.com/o/oauth2/v2/auth",
"token_url": "https://oauth2.googleapis.com/token"
}
},
"broken": {
"id": "broken_id",
"secret": "broken_secret",
"connect_config": {
"auth_url": "https://example.com/auth",
"token_url": "https://example.com/token",
"req_body_auth": "false"
}
}
}"#;
let map: BTreeMap<String, OAuthClientEntry> = serde_json::from_str(json).unwrap();
assert!(matches!(map["google"], OAuthClientEntry::Typed(_)));
assert!(matches!(map["broken"], OAuthClientEntry::Raw { .. }));
}
#[test]
fn custom_instance_pg_databases_roundtrips() {
let json = r#"{
@@ -2226,16 +2373,20 @@ mod tests {
let mut m = BTreeMap::new();
m.insert(
"google".to_string(),
OAuthClient {
OAuthClientEntry::Typed(OAuthClient {
id: "id".to_string(),
secret: StringOrSecretRef::EnvRef(EnvRefWrapper {
env_ref: "__WM_TEST_OAUTH_SECRET".to_string(),
}),
display_name: None,
allowed_domains: None,
connect_config: None,
login_config: None,
tenant: None,
share_with_workspaces: None,
},
grant_types: vec![],
extra: BTreeMap::new(),
}),
);
m
}),
@@ -2253,10 +2404,12 @@ mod tests {
.and_then(|v| v.as_literal()),
Some("smtp-secret")
);
assert_eq!(
gs.oauths.as_ref().unwrap()["google"].secret.as_literal(),
Some("oauth-secret")
);
match &gs.oauths.as_ref().unwrap()["google"] {
OAuthClientEntry::Typed(c) => {
assert_eq!(c.secret.as_literal(), Some("oauth-secret"));
}
OAuthClientEntry::Raw { .. } => panic!("expected Typed variant"),
}
unsafe {
std::env::remove_var("__WM_TEST_SMTP_PWD");
+29
View File
@@ -326,6 +326,28 @@ pub struct RunInlinePreviewScriptFnParams {
pub killpill_rx: tokio::sync::broadcast::Receiver<()>,
}
pub enum InlineScriptTarget {
Path(String),
Hash(i64),
}
pub struct RunInlineScriptFnParams {
pub workspace_id: String,
pub target: InlineScriptTarget,
pub args: Option<HashMap<String, Box<RawValue>>>,
pub created_by: String,
pub permissioned_as: String,
pub permissioned_as_email: String,
pub base_internal_url: String,
pub worker_name: String,
pub conn: crate::worker::Connection,
pub client: AuthedClient,
pub job_dir: String,
pub worker_dir: String,
pub killpill_rx: tokio::sync::broadcast::Receiver<()>,
pub user_db: Option<(crate::db::UserDB, crate::db::Authed)>,
}
#[derive(Clone)]
pub struct WorkerInternalServerInlineUtils {
pub killpill_rx: Arc<tokio::sync::broadcast::Receiver<()>>,
@@ -337,6 +359,13 @@ pub struct WorkerInternalServerInlineUtils {
+ Send
+ Sync,
>,
pub run_inline_script: Arc<
dyn Fn(
RunInlineScriptFnParams,
) -> Pin<Box<dyn Future<Output = error::Result<Box<RawValue>>> + Send>>
+ Send
+ Sync,
>,
}
// To run a script inline, bypassing the db and job queue, windmill-api uses these functions.
// They should only be called by the internal server of a worker.
@@ -510,7 +510,7 @@ impl WorkspaceDependenciesPrefetched {
// external.get(0).map(|wd| dbg!(wd.content.clone())).or(Some(
// "
// module mymod
// go 1.25
// go 1.26
// require ()
// "
// .to_owned(),
@@ -872,16 +872,29 @@ impl WorkspaceDependenciesAnnotatedRefs<String> {
validity_re_o: Option<&Regex>,
runnable_path: &str,
) -> Option<Self> {
let (extra_deps, manual_deps) = (format!("extra_{keyword}:"), format!("{keyword}:"));
let extra_deps_underscore = format!("extra_{keyword}:");
let extra_deps_hyphen = format!("extra-{keyword}:");
let manual_deps = format!("{keyword}:");
let Some((pos, mat)) = code.lines().find_position(|l| {
l.starts_with(&comment) && (l.contains(&extra_deps) || l.contains(&manual_deps))
}) else {
let is_extra = |l: &str| {
l.strip_prefix(comment)
.map(str::trim_start)
.is_some_and(|s| {
s.starts_with(&extra_deps_underscore) || s.starts_with(&extra_deps_hyphen)
})
};
let is_manual = |l: &str| {
l.strip_prefix(comment)
.map(str::trim_start)
.is_some_and(|s| s.starts_with(&manual_deps))
};
let Some((pos, mat)) = code.lines().find_position(|l| is_extra(l) || is_manual(l)) else {
return None;
};
let mut lines_it = code.lines().skip(pos);
let mode = if mat.contains(&extra_deps) {
let mode = if is_extra(mat) {
Mode::extra
} else {
Mode::manual
@@ -904,7 +917,9 @@ impl WorkspaceDependenciesAnnotatedRefs<String> {
.map(|s| {
match mode {
Mode::manual => s.replace(&manual_deps, ""),
Mode::extra => s.replace(&extra_deps, ""),
Mode::extra => s
.replace(&extra_deps_underscore, "")
.replace(&extra_deps_hyphen, ""),
}
.replace(comment, "")
})
@@ -993,6 +1008,29 @@ def main():
# extra_requirements: utils
#numpy>=1.24.0
def main():
pass
"#;
let result = WorkspaceDependenciesAnnotatedRefs::<String>::parse(
"#",
"requirements",
code,
None,
"",
)
.unwrap();
assert!(matches!(result.mode, Mode::extra));
assert_eq!(result.external, vec!["utils".to_owned()]);
assert_eq!(result.inline.as_ref().unwrap(), "numpy>=1.24.0");
}
#[test]
fn test_parse_annotation_python_extra_requirements_hyphen() {
let code = r#"
# extra-requirements: utils
#numpy>=1.24.0
def main():
pass
"#;
+13 -9
View File
@@ -564,19 +564,23 @@ async fn transform_json_unchecked(
transform_json_unchecked(&resource, w_id, db).await?
}
serde_json::Value::String(s) if s.starts_with("$var:") => {
let variable = sqlx::query_scalar!(
"SELECT value FROM variable WHERE workspace_id = $1 AND path = $2",
&w_id,
&s[5..]
let (value, is_secret): (String, bool) = sqlx::query_as(
"SELECT value, is_secret FROM variable WHERE workspace_id = $1 AND path = $2",
)
.bind(&w_id)
.bind(&s[5..])
.fetch_one(db)
.await
.map_err(to_anyhow)?;
let mc = build_crypt(&db, &w_id).await?;
let variable = decrypt(&mc, variable).map_err(|e| {
Error::internal_err(format!("Error decrypting variable {}: {}", &s, e))
})?;
serde_json::Value::String(variable)
let value = if is_secret {
let mc = build_crypt(&db, &w_id).await?;
decrypt(&mc, value).map_err(|e| {
Error::internal_err(format!("Error decrypting variable {}: {}", &s, e))
})?
} else {
value
};
serde_json::Value::String(value)
}
s @ serde_json::Value::String(_) => s.clone(),
x => x.clone(),
@@ -34,8 +34,8 @@ use windmill_api_auth::ApiAuthed;
#[cfg(feature = "native_trigger")]
use crate::{
decrypt_oauth_data, delete_token_by_prefix, delete_workspace_integration, resolve_endpoint,
store_workspace_integration, ServiceName,
decrypt_oauth_data, delete_token_by_prefix, delete_workspace_integration,
nextcloud::OcsResponse, resolve_endpoint, store_workspace_integration, ServiceName,
};
#[cfg(feature = "native_trigger")]
@@ -233,6 +233,21 @@ async fn try_delete_nextcloud_webhook(base_url: &str, access_token: &str, extern
.await;
}
#[cfg(feature = "native_trigger")]
async fn fetch_nextcloud_user_id(base_url: &str, access_token: &str) -> anyhow::Result<String> {
let url = format!("{}/ocs/v2.php/cloud/user", base_url);
let resp = HTTP_CLIENT
.get(&url)
.bearer_auth(access_token)
.header("OCS-APIRequest", "true")
.header("Accept", "application/json")
.send()
.await?
.error_for_status()?;
let ocs: OcsResponse<NextcloudUserData> = resp.json().await?;
Ok(ocs.ocs.data.id)
}
/// Delete all native triggers for a workspace+service, including remote webhook cleanup.
/// This is best-effort: errors during remote cleanup or token deletion are logged but ignored.
#[cfg(feature = "native_trigger")]
@@ -445,6 +460,12 @@ struct OAuthCallbackBody {
resource_path: Option<String>,
}
#[cfg(feature = "native_trigger")]
#[derive(Debug, Deserialize)]
struct NextcloudUserData {
id: String,
}
#[cfg(feature = "native_trigger")]
async fn oauth_callback(
authed: ApiAuthed,
@@ -542,7 +563,27 @@ async fn oauth_callback(
.map_err(|e| Error::InternalErr(format!("Failed to create variable: {}", e)))?;
// 3. Create resource pointing to the variable
let resource_value = json!({ "token": format!("$var:{}", resource_path) });
let resource_value = if service_name == ServiceName::Nextcloud {
let token_value = format!("$var:{}", resource_path);
let base_url = &oauth_config.base_url;
let user_id = fetch_nextcloud_user_id(base_url, &token_response.access_token).await;
match user_id {
Ok(user_id) => json!({
"token": token_value,
"baseUrl": base_url,
"userId": user_id,
}),
Err(e) => {
tracing::warn!("Failed to fetch Nextcloud user info: {e}");
json!({
"token": token_value,
"baseUrl": base_url,
})
}
}
} else {
json!({ "token": format!("$var:{}", resource_path) })
};
sqlx::query!(
"INSERT INTO resource (workspace_id, path, value, resource_type, description, created_by)
+53
View File
@@ -2285,6 +2285,59 @@ pub struct MiniPulledJob {
pub runnable_settings_handle: Option<i64>,
}
impl MiniPulledJob {
pub fn new_inline(
workspace_id: String,
args: Option<HashMap<String, Box<RawValue>>>,
created_by: String,
permissioned_as: String,
permissioned_as_email: String,
runnable_path: Option<String>,
kind: JobKind,
runnable_id: Option<ScriptHash>,
tag: String,
script_lang: Option<ScriptLang>,
) -> Self {
Self {
workspace_id,
id: Uuid::new_v4(),
args: args.map(Json),
parent_job: None,
created_by,
scheduled_for: chrono::Utc::now(),
started_at: None,
runnable_path,
kind,
runnable_id,
canceled_reason: None,
canceled_by: None,
permissioned_as,
permissioned_as_email,
flow_status: None,
tag,
script_lang,
same_worker: true,
pre_run_error: None,
flow_innermost_root_job: None,
root_job: None,
timeout: None,
flow_step_id: None,
cache_ttl: None,
cache_ignore_s3_path: None,
priority: None,
preprocessed: None,
script_entrypoint_override: None,
trigger: None,
trigger_kind: None,
visible_to_owner: false,
permissioned_as_end_user_email: None,
runnable_settings_handle: None,
concurrent_limit: None,
concurrency_time_window_s: None,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MiniCompletedJob {
pub id: Uuid,
@@ -3076,4 +3076,617 @@ mod debounce {
Ok(())
}
// =========================================================================
// Helpers for focused accumulation tests
// =========================================================================
/// Helper: insert a flow job with flow_status and v2_job_status, simulating a flow with a preprocessor.
async fn insert_flow_job_with_preprocessor(
db: &Pool<Postgres>,
job_id: Uuid,
workspace_id: &str,
runnable_path: &str,
preprocessed: bool,
flow_status_step: i32,
) {
sqlx::query!(
"INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, preprocessed)
VALUES ($1, 'flow', 'flow', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
job_id,
workspace_id,
runnable_path,
preprocessed,
)
.execute(db)
.await
.expect("insert v2_job");
sqlx::query!(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag, running)
VALUES ($1, $2, now(), 'flow', false)",
job_id,
workspace_id,
)
.execute(db)
.await
.expect("insert v2_job_queue");
sqlx::query!("INSERT INTO v2_job_runtime (id) VALUES ($1)", job_id)
.execute(db)
.await
.expect("insert v2_job_runtime");
let flow_status = serde_json::json!({
"step": flow_status_step,
"modules": [
{"id": "a", "type": "WaitingForPriorSteps"}
],
"failure_module": {
"type": "WaitingForPriorSteps",
"id": "failure"
},
"preprocessor_module": {
"type": "Success",
"id": "preprocessor",
"job": Uuid::new_v4().to_string(),
"flow_jobs": null,
"flow_jobs_success": null,
"branch_chosen": null,
"approvers": [],
"failed_retries": [],
"skipped": false
},
"cleanup_module": {"flow_jobs_to_clean": []},
"retry": {"fail_count": 0, "failed_jobs": []}
});
sqlx::query!(
"INSERT INTO v2_job_status (id, flow_status) VALUES ($1, $2)",
job_id,
flow_status,
)
.execute(db)
.await
.expect("insert v2_job_status");
}
/// Helper: insert a script job with args into v2_job + v2_job_queue + v2_job_runtime.
async fn insert_script_job_with_args(
db: &Pool<Postgres>,
job_id: Uuid,
workspace_id: &str,
runnable_path: &str,
args: &serde_json::Value,
) {
sqlx::query!(
"INSERT INTO v2_job (id, kind, tag, created_by, permissioned_as, permissioned_as_email, workspace_id, runnable_path, args)
VALUES ($1, 'script', 'deno', 'test-user', 'u/test-user', 'test@windmill.dev', $2, $3, $4)",
job_id,
workspace_id,
runnable_path,
args,
)
.execute(db)
.await
.expect("insert v2_job with args");
sqlx::query!(
"INSERT INTO v2_job_queue (id, workspace_id, scheduled_for, tag)
VALUES ($1, $2, now(), 'deno')",
job_id,
workspace_id,
)
.execute(db)
.await
.expect("insert v2_job_queue");
sqlx::query!("INSERT INTO v2_job_runtime (id) VALUES ($1)", job_id)
.execute(db)
.await
.expect("insert v2_job_runtime");
}
/// Helper: build a PulledJobResult from job data, for calling maybe_apply_debouncing.
fn make_pulled_job_result(
job_id: Uuid,
workspace_id: &str,
runnable_path: &str,
args: &serde_json::Value,
kind: JobKind,
tag: &str,
rs_handle: Option<i64>,
) -> windmill_queue::PulledJobResult {
use windmill_queue::{MiniPulledJob, PulledJob, PulledJobResult};
let args_hm: HashMap<String, Box<RawValue>> = serde_json::from_value(args.clone()).unwrap();
let mini = MiniPulledJob {
workspace_id: workspace_id.to_string(),
id: job_id,
args: Some(sqlx::types::Json(args_hm)),
parent_job: None,
created_by: "test-user".to_string(),
scheduled_for: Utc::now(),
started_at: None,
runnable_path: Some(runnable_path.to_string()),
kind,
runnable_id: None,
canceled_reason: None,
canceled_by: None,
permissioned_as: "u/test-user".to_string(),
permissioned_as_email: "test@windmill.dev".to_string(),
flow_status: None,
tag: tag.to_string(),
script_lang: None,
same_worker: false,
pre_run_error: None,
concurrent_limit: None,
concurrency_time_window_s: None,
flow_innermost_root_job: None,
root_job: None,
timeout: None,
flow_step_id: None,
cache_ttl: None,
cache_ignore_s3_path: None,
priority: None,
preprocessed: None,
script_entrypoint_override: None,
trigger: None,
trigger_kind: None,
visible_to_owner: false,
permissioned_as_end_user_email: None,
runnable_settings_handle: rs_handle,
};
let pulled = PulledJob {
job: mini,
raw_code: None,
raw_lock: None,
raw_flow: None,
parent_runnable_path: None,
permissioned_as_email: None,
permissioned_as_username: None,
permissioned_as_is_admin: None,
permissioned_as_is_operator: None,
permissioned_as_groups: None,
permissioned_as_folders: None,
};
PulledJobResult {
job: Some(pulled),
suspended: false,
missing_concurrency_key: false,
error_while_preprocessing: None,
}
}
/// Helper: insert debouncing settings into the DB and return the runnable_settings_handle.
async fn setup_debouncing_settings(
db: &Pool<Postgres>,
settings: &DebouncingSettings,
) -> Option<i64> {
use windmill_common::runnable_settings::{
insert_rs, ConcurrencySettings, RunnableSettings, RunnableSettingsTrait,
};
let debouncing_hash = settings.insert_cached(db).await.expect("insert debouncing");
let concurrency_hash = ConcurrencySettings::default()
.insert_cached(db)
.await
.expect("insert concurrency");
insert_rs(
RunnableSettings {
debouncing_settings: debouncing_hash,
concurrency_settings: concurrency_hash,
},
db,
)
.await
.expect("insert rs")
}
/// Helper: assert that accumulated items match expected values.
fn assert_accumulated_items(
result: &windmill_queue::PulledJobResult,
expected: &[i64],
arg_name: &str,
) {
let job = result
.job
.as_ref()
.expect("survivor job should not be nulled out");
let args = job.job.args.as_ref().expect("args should be present");
let items_raw = args.get(arg_name).expect("accumulated arg should exist");
let items: Vec<serde_json::Value> =
serde_json::from_str(items_raw.get()).expect("items should be valid JSON array");
let mut item_nums: Vec<i64> = items
.iter()
.map(|v| v.as_i64().expect("item should be a number"))
.collect();
item_nums.sort();
assert_eq!(
item_nums, expected,
"accumulated items should contain all values from all debounced jobs"
);
}
// =========================================================================
// Argument accumulation tests for scripts, flows, flows with preprocessor
// =========================================================================
/// Test: Script debounce accumulation via push-time maybe_debounce + maybe_apply_debouncing.
/// Pushes 3 script jobs with different "items" values, verifies they accumulate.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_script_debounce_accumulation(db: Pool<Postgres>) -> anyhow::Result<()> {
let settings = DebouncingSettings {
debounce_delay_s: Some(5),
debounce_key: Some("script_accum_key".to_string()),
debounce_args_to_accumulate: Some(vec!["items".to_string()]),
..Default::default()
};
let rs_handle = setup_debouncing_settings(&db, &settings).await;
let jobs: Vec<(Uuid, serde_json::Value)> = vec![
(
Uuid::new_v4(),
serde_json::json!({"items": [1, 2], "other": "x"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [3], "other": "x"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [4, 5, 6], "other": "x"}),
),
];
// Insert script jobs and set runnable_settings_handle
for (id, args) in &jobs {
insert_script_job_with_args(&db, *id, "test-workspace", "f/test/script", args).await;
sqlx::query!(
"UPDATE v2_job_queue SET runnable_settings_handle = $1 WHERE id = $2",
rs_handle,
id,
)
.execute(&db)
.await?;
}
// Push-time debounce: each job debounces the previous one
for (id, args) in &jobs {
let args_hm: HashMap<String, Box<RawValue>> =
serde_json::from_value(args.clone()).unwrap();
let push_args = PushArgs::from(&args_hm);
let mut scheduled_for = None;
let mut tx = db.begin().await?;
windmill_queue::jobs_ee::maybe_debounce(
&settings,
&mut scheduled_for,
&Some("f/test/script".to_string()),
"test-workspace",
JobKind::Script,
*id,
&push_args,
&mut tx,
)
.await?;
tx.commit().await?;
}
// Last job should survive, first two should be completed (skipped)
let survivor_id = jobs[2].0;
assert!(
is_queued(&db, &survivor_id).await,
"last job should survive in queue"
);
assert!(
is_completed(&db, &jobs[0].0).await,
"job 0 should be debounced"
);
assert!(
is_completed(&db, &jobs[1].0).await,
"job 1 should be debounced"
);
// Call maybe_apply_debouncing on the survivor
let mut result = make_pulled_job_result(
survivor_id,
"test-workspace",
"f/test/script",
&jobs[2].1,
JobKind::Script,
"deno",
rs_handle,
);
result.maybe_apply_debouncing(&db).await?;
// Verify accumulation
assert_accumulated_items(&result, &[1, 2, 3, 4, 5, 6], "items");
// "other" arg should be unchanged
let job = result.job.as_ref().unwrap();
let other_raw = job.job.args.as_ref().unwrap().get("other").unwrap();
let other: String = serde_json::from_str(other_raw.get())?;
assert_eq!(other, "x", "non-accumulated arg should be unchanged");
Ok(())
}
/// Test: Flow (without preprocessor) debounce accumulation via push-time maybe_debounce.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_flow_debounce_accumulation_no_preprocessor(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
let settings = DebouncingSettings {
debounce_delay_s: Some(5),
debounce_key: Some("flow_accum_key".to_string()),
debounce_args_to_accumulate: Some(vec!["items".to_string()]),
..Default::default()
};
let rs_handle = setup_debouncing_settings(&db, &settings).await;
let jobs: Vec<(Uuid, serde_json::Value)> = vec![
(
Uuid::new_v4(),
serde_json::json!({"items": [10, 20], "tag": "a"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [30], "tag": "a"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [40, 50], "tag": "a"}),
),
];
for (id, args) in &jobs {
insert_flow_job_with_args(&db, *id, "test-workspace", "f/test/flow_no_pp", args).await;
sqlx::query!(
"UPDATE v2_job_queue SET runnable_settings_handle = $1 WHERE id = $2",
rs_handle,
id,
)
.execute(&db)
.await?;
}
// Push-time debounce
for (id, args) in &jobs {
let args_hm: HashMap<String, Box<RawValue>> =
serde_json::from_value(args.clone()).unwrap();
let push_args = PushArgs::from(&args_hm);
let mut scheduled_for = None;
let mut tx = db.begin().await?;
windmill_queue::jobs_ee::maybe_debounce(
&settings,
&mut scheduled_for,
&Some("f/test/flow_no_pp".to_string()),
"test-workspace",
JobKind::Flow,
*id,
&push_args,
&mut tx,
)
.await?;
tx.commit().await?;
}
let survivor_id = jobs[2].0;
assert!(
is_queued(&db, &survivor_id).await,
"last job should survive"
);
assert!(
is_completed(&db, &jobs[0].0).await,
"job 0 should be debounced"
);
assert!(
is_completed(&db, &jobs[1].0).await,
"job 1 should be debounced"
);
let mut result = make_pulled_job_result(
survivor_id,
"test-workspace",
"f/test/flow_no_pp",
&jobs[2].1,
JobKind::Flow,
"flow",
rs_handle,
);
result.maybe_apply_debouncing(&db).await?;
assert_accumulated_items(&result, &[10, 20, 30, 40, 50], "items");
Ok(())
}
/// Test: Flow WITH preprocessor debounce accumulation via maybe_debounce_post_preprocessing.
/// This is the bug case: after preprocessing completes, the worker must store the flow's
/// debouncing settings in runnable_settings_handle so that maybe_apply_debouncing can find
/// them when the surviving job is pulled.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_flow_debounce_accumulation_with_preprocessor(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
let settings = DebouncingSettings {
debounce_delay_s: Some(5),
debounce_key: None,
debounce_args_to_accumulate: Some(vec!["items".to_string()]),
..Default::default()
};
let rs_handle = setup_debouncing_settings(&db, &settings).await;
let jobs: Vec<(Uuid, serde_json::Value)> = vec![
(
Uuid::new_v4(),
serde_json::json!({"items": [100, 200], "extra": "v"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [300], "extra": "v"}),
),
(
Uuid::new_v4(),
serde_json::json!({"items": [400, 500, 600], "extra": "v"}),
),
];
// Insert flow jobs with preprocessor state (step=0, preprocessor=Success)
for (id, args) in &jobs {
insert_flow_job_with_preprocessor(
&db,
*id,
"test-workspace",
"f/test/flow_pp",
true,
0,
)
.await;
sqlx::query!("UPDATE v2_job SET args = $2 WHERE id = $1", id, args)
.execute(&db)
.await?;
}
// Post-preprocessing debounce
for (id, args) in &jobs {
let args_hm: HashMap<String, Box<RawValue>> =
serde_json::from_value(args.clone()).unwrap();
let push_args = PushArgs::from(&args_hm);
windmill_queue::jobs_ee::maybe_debounce_post_preprocessing(
&settings,
&Some("f/test/flow_pp".to_string()),
"test-workspace",
*id,
&push_args,
&db,
)
.await?;
}
let survivor_id = jobs[2].0;
assert!(
is_queued(&db, &survivor_id).await,
"last job should survive"
);
// Simulate the fix: worker stores runnable_settings_handle on the surviving job
sqlx::query!(
"UPDATE v2_job_queue SET runnable_settings_handle = $1 WHERE id = $2",
rs_handle,
survivor_id,
)
.execute(&db)
.await?;
let mut result = make_pulled_job_result(
survivor_id,
"test-workspace",
"f/test/flow_pp",
&jobs[2].1,
JobKind::Flow,
"flow",
rs_handle,
);
result.maybe_apply_debouncing(&db).await?;
assert_accumulated_items(&result, &[100, 200, 300, 400, 500, 600], "items");
// "extra" arg should be unchanged
let job = result.job.as_ref().unwrap();
let extra_raw = job.job.args.as_ref().unwrap().get("extra").unwrap();
let extra: String = serde_json::from_str(extra_raw.get())?;
assert_eq!(extra, "v", "non-accumulated arg should be unchanged");
Ok(())
}
/// Test: Flow WITH preprocessor but WITHOUT the runnable_settings_handle fix.
/// Proves the bug: when runnable_settings_handle is NULL, accumulation does nothing.
#[sqlx::test(migrations = "../migrations", fixtures("base"))]
async fn test_flow_debounce_accumulation_with_preprocessor_no_fix(
db: Pool<Postgres>,
) -> anyhow::Result<()> {
let settings = DebouncingSettings {
debounce_delay_s: Some(5),
debounce_key: None,
debounce_args_to_accumulate: Some(vec!["items".to_string()]),
..Default::default()
};
let jobs: Vec<(Uuid, serde_json::Value)> = vec![
(Uuid::new_v4(), serde_json::json!({"items": [1, 2]})),
(Uuid::new_v4(), serde_json::json!({"items": [3]})),
(Uuid::new_v4(), serde_json::json!({"items": [4, 5]})),
];
for (id, args) in &jobs {
insert_flow_job_with_preprocessor(
&db,
*id,
"test-workspace",
"f/test/flow_pp_nofix",
true,
0,
)
.await;
sqlx::query!("UPDATE v2_job SET args = $2 WHERE id = $1", id, args)
.execute(&db)
.await?;
}
for (id, args) in &jobs {
let args_hm: HashMap<String, Box<RawValue>> =
serde_json::from_value(args.clone()).unwrap();
let push_args = PushArgs::from(&args_hm);
windmill_queue::jobs_ee::maybe_debounce_post_preprocessing(
&settings,
&Some("f/test/flow_pp_nofix".to_string()),
"test-workspace",
*id,
&push_args,
&db,
)
.await?;
}
let survivor_id = jobs[2].0;
assert!(
is_queued(&db, &survivor_id).await,
"last job should survive"
);
// DO NOT set runnable_settings_handle — simulating the bug (no fix applied)
let mut result = make_pulled_job_result(
survivor_id,
"test-workspace",
"f/test/flow_pp_nofix",
&jobs[2].1,
JobKind::Flow,
"flow",
None, // No runnable_settings_handle — this is the bug
);
result.maybe_apply_debouncing(&db).await?;
// Without the fix, only the survivor's own items are present (no accumulation)
let job = result.job.as_ref().expect("job should still exist");
let args = job.job.args.as_ref().expect("args should be present");
let items_raw = args.get("items").expect("items should exist");
let items: Vec<serde_json::Value> = serde_json::from_str(items_raw.get())?;
let item_nums: Vec<i64> = items.iter().map(|v| v.as_i64().unwrap()).collect();
assert_eq!(
item_nums,
vec![4, 5],
"Without the fix: only the survivor's own items should be present (no accumulation)"
);
Ok(())
}
}
+2
View File
@@ -15,6 +15,8 @@ enterprise = []
python = ["windmill-common/python"]
deno_core = ["dep:windmill-runtime-nativets"]
agent_worker_server = ["dep:windmill-api-agent-workers"]
run_inline = ["windmill-api/run_inline"]
duckdb = ["windmill-worker/duckdb"]
[dependencies]
windmill-api = { workspace = true, default-features = false }
+59 -9
View File
@@ -24,16 +24,16 @@ use tokio_postgres::{
use uuid::Uuid;
use windmill_common::error::to_anyhow;
use windmill_common::error::{self, Error};
use windmill_object_store::convert_json_line_stream;
use windmill_common::worker::{
to_raw_value, Connection, SqlResultCollectionStrategy, CLOUD_HOSTED,
};
use windmill_common::workspaces::get_datatable_resource_from_db_unchecked;
use windmill_common::{PgDatabase, PrepareQueryColumnInfo, PrepareQueryResult};
use windmill_object_store::convert_json_line_stream;
use windmill_parser::{Arg, Typ};
use windmill_parser_sql::{
parse_db_resource, parse_pg_statement_arg_indices, parse_pgsql_sig, parse_s3_mode,
parse_sql_blocks,
parse_db_resource, parse_pg_statement_arg_indices, parse_pgsql_sig_with_typed_schema,
parse_s3_mode, parse_sql_blocks,
};
use windmill_queue::{CanceledBy, MiniPulledJob};
@@ -58,6 +58,42 @@ lazy_static! {
pub static ref LAST_QUERY: AtomicU64 = AtomicU64::new(0);
}
fn otyp_to_pg_type(otyp: &str) -> error::Result<Type> {
let base = otyp.trim_end_matches("[]");
let is_array = otyp.ends_with("[]");
let (scalar, array) = match base {
"bool" | "boolean" => (Type::BOOL, Type::BOOL_ARRAY),
"char" | "character" => (Type::CHAR, Type::CHAR_ARRAY),
"smallint" | "smallserial" | "int2" | "serial2" => (Type::INT2, Type::INT2_ARRAY),
"int" | "integer" | "int4" | "serial" => (Type::INT4, Type::INT4_ARRAY),
"bigint" | "bigserial" | "int8" | "serial8" => (Type::INT8, Type::INT8_ARRAY),
"real" | "float4" => (Type::FLOAT4, Type::FLOAT4_ARRAY),
"double" | "float8" => (Type::FLOAT8, Type::FLOAT8_ARRAY),
"numeric" | "decimal" => (Type::NUMERIC, Type::NUMERIC_ARRAY),
"text" => (Type::TEXT, Type::TEXT_ARRAY),
"varchar" | "character varying" => (Type::VARCHAR, Type::VARCHAR_ARRAY),
"uuid" => (Type::UUID, Type::UUID_ARRAY),
"date" => (Type::DATE, Type::DATE_ARRAY),
"time" => (Type::TIME, Type::TIME_ARRAY),
"timetz" => (Type::TIMETZ, Type::TIMETZ_ARRAY),
"timestamp" => (Type::TIMESTAMP, Type::TIMESTAMP_ARRAY),
"timestamptz" => (Type::TIMESTAMPTZ, Type::TIMESTAMPTZ_ARRAY),
"json" => (Type::JSON, Type::JSON_ARRAY),
"jsonb" => (Type::JSONB, Type::JSONB_ARRAY),
"bytea" => (Type::BYTEA, Type::BYTEA_ARRAY),
"oid" => (Type::OID, Type::OID_ARRAY),
_ => {
return Err(Error::ExecutionErr(format!(
"Unsupported PostgreSQL type for typed schema: {}",
otyp
)))
}
};
Ok(if is_array { array } else { scalar })
}
fn do_postgresql_inner<'a>(
mut query: String,
param_idx_to_arg_and_value: &HashMap<i32, (&Arg, Option<&Value>)>,
@@ -67,8 +103,10 @@ fn do_postgresql_inner<'a>(
skip_collect: bool,
first_row_only: bool,
s3: Option<S3ModeWorkerData>,
typed_schema: bool,
) -> error::Result<BoxFuture<'a, error::Result<Vec<Box<RawValue>>>>> {
let mut query_params = vec![];
let mut param_types = vec![];
let arg_indices = parse_pg_statement_arg_indices(&query);
@@ -86,13 +124,14 @@ fn do_postgresql_inner<'a>(
let typ = &arg.typ;
let param = convert_val(value, arg_t, typ)?;
query_params.push(param);
if typed_schema {
param_types.push(otyp_to_pg_type(arg_t)?);
}
i += 1;
}
}
let result_f = async move {
// Now we can execute a simple statement that just returns its parameter.
let mut res: Vec<Box<serde_json::value::RawValue>> = vec![];
let query_params = query_params
@@ -100,14 +139,23 @@ fn do_postgresql_inner<'a>(
.map(|p| &**p as &(dyn ToSql + Sync))
.collect_vec();
let statement = if typed_schema {
client
.prepare_typed(&query, &param_types)
.await
.map_err(to_anyhow)?
} else {
client.prepare(&query).await.map_err(to_anyhow)?
};
if skip_collect {
client
.execute_raw(&query, query_params)
.execute_raw(&statement, query_params)
.await
.map_err(to_anyhow)?;
} else if let Some(ref s3) = s3 {
let rows_stream = client
.query_raw(&query, query_params)
.query_raw(&statement, query_params)
.map_err(to_anyhow)
.await?
.map_err(to_anyhow)
@@ -121,7 +169,7 @@ fn do_postgresql_inner<'a>(
return Ok(vec![to_raw_value(&s3.to_return_s3_obj())]);
} else {
let rows = client
.query_raw(&query, query_params)
.query_raw(&statement, query_params)
.await
.map_err(to_anyhow)?;
@@ -272,7 +320,8 @@ pub async fn do_postgresql(
(Some((client, handle)), None)
};
let sig = parse_pgsql_sig(&query).map_err(|x| Error::ExecutionErr(x.to_string()))?;
let (sig, typed_schema) = parse_pgsql_sig_with_typed_schema(&query)
.map_err(|x| Error::ExecutionErr(x.to_string()))?;
let reserved_variables =
get_reserved_variables(job, &client.token, conn, parent_runnable_path).await?;
@@ -346,6 +395,7 @@ pub async fn do_postgresql(
&& i < queries.len() - 1,
collection_strategy.collect_first_row_only(),
s3.clone(),
typed_schema,
)?
.await?;
results.push(result);
+96 -37
View File
@@ -14,6 +14,10 @@ use futures::TryFutureExt;
use tokio::sync::Mutex;
use tokio::time::timeout;
use windmill_common::client::AuthedClient;
use windmill_common::db::UserDbWithAuthed;
use windmill_common::get_latest_deployed_hash_for_path;
use windmill_common::jobs::InlineScriptTarget;
use windmill_common::jobs::RunInlineScriptFnParams;
use windmill_common::jobs::WorkerInternalServerInlineUtils;
use windmill_common::jobs::WORKER_INTERNAL_SERVER_INLINE_UTILS;
use windmill_common::runtime_assets::init_runtime_asset_loop;
@@ -4705,43 +4709,18 @@ pub fn init_worker_internal_server_inline_utils(
base_internal_url,
killpill_rx: Arc::new(killpill_rx),
run_inline_preview_script: Arc::new(|params| {
let job = MiniPulledJob {
workspace_id: params.workspace_id,
id: Uuid::new_v4(),
args: params.args.map(Json),
parent_job: None,
created_by: params.created_by,
scheduled_for: chrono::Utc::now(),
started_at: None,
runnable_path: None,
kind: JobKind::Preview,
runnable_id: None,
canceled_reason: None,
canceled_by: None,
permissioned_as: params.permissioned_as,
permissioned_as_email: params.permissioned_as_email,
flow_status: None,
tag: "inline_preview".to_string(),
script_lang: Some(params.lang),
same_worker: true,
pre_run_error: None,
flow_innermost_root_job: None,
root_job: None,
timeout: None,
flow_step_id: None,
cache_ttl: None,
cache_ignore_s3_path: None,
priority: None,
preprocessed: None,
script_entrypoint_override: None,
trigger: None,
trigger_kind: None,
visible_to_owner: false,
permissioned_as_end_user_email: None,
runnable_settings_handle: None,
concurrent_limit: None,
concurrency_time_window_s: None,
};
let job = MiniPulledJob::new_inline(
params.workspace_id,
params.args,
params.created_by,
params.permissioned_as,
params.permissioned_as_email,
None,
JobKind::Preview,
None,
"inline_preview".to_string(),
Some(params.lang),
);
Box::pin(async move {
let mut mem_peak: i32 = -1;
let mut canceled_by: Option<CanceledBy> = None;
@@ -4778,6 +4757,86 @@ pub fn init_worker_internal_server_inline_utils(
.await
})
}),
run_inline_script: Arc::new(|params: RunInlineScriptFnParams| {
Box::pin(async move {
let (script_hash, runnable_path) = match params.target {
InlineScriptTarget::Path(ref path) => {
let db = params
.conn
.as_sql()
.ok_or_else(|| {
error::Error::InternalErr(
"run_inline_script by path requires a SQL connection"
.to_string(),
)
})?
.clone();
let authed_ref = params.user_db.as_ref().map(|(_, a)| a.to_authed_ref());
let user_db_authed =
params.user_db.as_ref().zip(authed_ref.as_ref()).map(
|((udb, _), ar)| UserDbWithAuthed { db: udb.clone(), authed: ar },
);
let script_hash_info = get_latest_deployed_hash_for_path(
user_db_authed,
db,
&params.workspace_id,
path,
)
.await?;
(ScriptHash(script_hash_info.hash), Some(path.clone()))
}
InlineScriptTarget::Hash(hash) => (ScriptHash(hash), None),
};
let content_info =
get_script_content_by_hash(&script_hash, &params.workspace_id, &params.conn)
.await?;
let job = MiniPulledJob::new_inline(
params.workspace_id,
params.args,
params.created_by,
params.permissioned_as,
params.permissioned_as_email,
runnable_path,
JobKind::Script,
Some(script_hash),
"inline_run".to_string(),
content_info.language,
);
let mut mem_peak: i32 = -1;
let mut canceled_by: Option<CanceledBy> = None;
let mut column_order: Option<Vec<String>> = None;
let mut new_args: Option<HashMap<String, Box<RawValue>>> = None;
let mut occupancy_metrics = OccupancyMetrics::new(Instant::now());
let mut has_stream: bool = false;
let mut killpill_rx = params.killpill_rx;
run_language_executor(
&job,
&params.conn,
&params.client,
None,
&params.job_dir,
&params.worker_dir,
&mut mem_peak,
&mut canceled_by,
&params.base_internal_url,
&params.worker_name,
&mut column_order,
&mut new_args,
&mut occupancy_metrics,
&mut killpill_rx,
None,
&mut has_stream,
content_info.language,
&content_info.content,
&content_info.envs,
&content_info.codebase,
&content_info.lockfile,
true,
)
.await
})
}),
};
WORKER_INTERNAL_SERVER_INLINE_UTILS
.set(utils)
+59 -13
View File
@@ -47,7 +47,9 @@ use windmill_common::jobs::{
check_tag_available_for_workspace_internal, script_path_to_payload, JobKind, JobPayload,
OnBehalfOf, RawCode, ENTRYPOINT_OVERRIDE,
};
use windmill_common::runnable_settings::{ConcurrencySettingsWithCustom, DebouncingSettings};
use windmill_common::runnable_settings::{
ConcurrencySettingsWithCustom, DebouncingSettings, RunnableSettingsTrait,
};
use windmill_common::scripts::{ScriptHash, ScriptRunnableSettingsInline};
use windmill_common::users::username_to_permissioned_as;
use windmill_common::utils::WarnAfterExt;
@@ -547,6 +549,25 @@ pub async fn update_flow_status_after_job_completion_internal(
Ok::<_, Error>(args.clone())
};
// Pre-compute whether the completed job was an identity (skipped) job.
// When a step has skip_if set and the condition was true, the step runs
// as an identity job. We must skip the stop_after_if evaluation in that
// case because the result is just a pass-through of the previous step's
// result, not the output of the actual step logic.
let is_identity_job = if current_module.is_some_and(|m| m.skip_if.is_some()) {
sqlx::query_scalar!(
"SELECT kind = 'identity' FROM v2_job WHERE id = $1",
job_id_for_status
)
.fetch_optional(db)
.await
.map_err(|e| Error::internal_err(format!("error during identity job check: {e:#}")))?
.flatten()
.unwrap_or(false)
} else {
false
};
let (mut stop_early, mut stop_early_err_msg, mut skip_if_stop_early, continue_on_error) =
if stop_early_override.is_some()
&& !is_flow_stop_early_override
@@ -562,6 +583,7 @@ pub async fn update_flow_status_after_job_completion_internal(
let stop_early = success
&& !is_branch_all // we don't support stop_early per branch
&& !parallel_loop // we don't support anymore stop_early per iteration when parallel for loop (removed from frontend)
&& !is_identity_job // don't evaluate stop_after_if for skipped (identity) steps
&& if let Some(expr) = current_module
.stop_after_if
.as_ref()
@@ -1101,17 +1123,7 @@ pub async fn update_flow_status_after_job_completion_internal(
{
let is_skipped = (stop_early && skip_if_stop_early)
|| if current_module.as_ref().is_some_and(|m| m.skip_if.is_some()) {
sqlx::query_scalar!(
"SELECT kind = 'identity' FROM v2_job WHERE id = $1",
job_id_for_status
)
.fetch_optional(db)
.await
.map_err(|e| {
Error::internal_err(format!("error during skip check: {e:#}"))
})?
.flatten()
.unwrap_or(false)
is_identity_job
} else {
false
};
@@ -1424,6 +1436,38 @@ pub async fn update_flow_status_after_job_completion_internal(
}
};
// When debouncing is applied, store the flow's debouncing settings in
// the runnable_settings_handle so that maybe_apply_debouncing can find
// them after re-pull and perform argument accumulation.
let new_runnable_settings_handle: Option<i64> = if scheduled_for.is_some() {
let debouncing_hash = flow_value
.debouncing_settings
.insert_cached(db)
.await
.unwrap_or_else(|e| {
tracing::error!(
"Failed to insert debouncing settings for post-preprocessing: {e:#}"
);
None
});
windmill_common::runnable_settings::insert_rs(
windmill_common::runnable_settings::RunnableSettings {
debouncing_settings: debouncing_hash,
concurrency_settings: None,
},
db,
)
.await
.unwrap_or_else(|e| {
tracing::error!(
"Failed to insert runnable settings for post-preprocessing: {e:#}"
);
None
})
} else {
None
};
sqlx::query!(
"WITH job_result AS (
SELECT result
@@ -1434,7 +1478,8 @@ pub async fn update_flow_status_after_job_completion_internal(
UPDATE v2_job_queue
SET running = false,
tag = COALESCE($3, tag),
scheduled_for = COALESCE($6, scheduled_for)
scheduled_for = COALESCE($6, scheduled_for),
runnable_settings_handle = COALESCE($7, runnable_settings_handle)
WHERE id = $2
)
UPDATE v2_job
@@ -1463,6 +1508,7 @@ pub async fn update_flow_status_after_job_completion_internal(
concurrent_limit,
concurrency_time_window_s,
scheduled_for,
new_runnable_settings_handle,
)
.execute(db)
.await
+14 -2
View File
@@ -1,7 +1,19 @@
#!/bin/bash
# Navigate to the target directory
cd ../../windmill-ee-private || cd ~/windmill-ee-private || { echo "Directory not found"; exit 1; }
# Detect the current branch name
branch=$(git rev-parse --abbrev-ref HEAD 2>/dev/null)
# Try, in order: matching EE worktree, sibling repo, home directory fallback
ee_worktree="$HOME/windmill-ee-private__worktrees/$branch"
if [ -n "$branch" ] && [ -d "$ee_worktree" ]; then
cd "$ee_worktree"
elif cd ../../windmill-ee-private 2>/dev/null; then
:
elif cd ~/windmill-ee-private 2>/dev/null; then
:
else
echo "Directory not found"; exit 1
fi
# Get the current commit hash
commit_hash=$(git rev-parse HEAD)
+1 -1
View File
@@ -2,7 +2,7 @@ import { sleep } from "https://deno.land/x/sleep@v1.2.1/mod.ts";
import * as windmill from "https://deno.land/x/windmill@v1.174.0/mod.ts";
import * as api from "https://deno.land/x/windmill@v1.174.0/windmill-api/index.ts";
export const VERSION = "v1.645.0";
export const VERSION = "v1.649.0";
export async function login(email: string, password: string): Promise<string> {
return await windmill.UserService.login({
+30 -9
View File
@@ -38,6 +38,7 @@
"wmill": "src/main.ts"
},
"devDependencies": {
"@types/bun": "^1.3.9",
"@types/diff": "^5.2.3",
"@types/node": "^22.0.0",
"@types/tar-stream": "^3.1.4",
@@ -696,6 +697,16 @@
"acorn": "^8.9.0"
}
},
"node_modules/@types/bun": {
"version": "1.3.9",
"resolved": "https://registry.npmjs.org/@types/bun/-/bun-1.3.9.tgz",
"integrity": "sha512-KQ571yULOdWJiMH+RIWIOZ7B2RXQGpL1YQrBtLIV3FqDcCu6FsbFUBwhdKUlCKUpS3PJDsHlJ1QKlpxoVR+xtw==",
"dev": true,
"license": "MIT",
"dependencies": {
"bun-types": "1.3.9"
}
},
"node_modules/@types/diff": {
"version": "5.2.3",
"resolved": "https://registry.npmjs.org/@types/diff/-/diff-5.2.3.tgz",
@@ -852,6 +863,16 @@
"node": "20 || >=22"
}
},
"node_modules/bun-types": {
"version": "1.3.9",
"resolved": "https://registry.npmjs.org/bun-types/-/bun-types-1.3.9.tgz",
"integrity": "sha512-+UBWWOakIP4Tswh0Bt0QD0alpTY8cb5hvgiYeWCMet9YukHbzuruIEeXC2D7nMJPB12kbh8C7XJykSexEqGKJg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/bundle-name": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/bundle-name/-/bundle-name-4.1.0.tgz",
@@ -1384,9 +1405,9 @@
"integrity": "sha512-AJLFiUy6af+LpUe7CddDo4+JOmw3c0K/1iOWh8NdTwXcLDj90lL6089mdsVo1apyloLgrTbcuFDzZMXVGBgtCg=="
},
"node_modules/windmill-parser-wasm-php": {
"version": "1.574.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-php/-/windmill-parser-wasm-php-1.574.1.tgz",
"integrity": "sha512-COyid6B1RYs+bpzUCInsA4HY/WZkpDLfkQ90+AqU/TVTpzYSbAC2JCbIwy0cRElBvlhI4bQ+9Wg6hSQKMpEkpA=="
"version": "1.647.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-php/-/windmill-parser-wasm-php-1.647.1.tgz",
"integrity": "sha512-u2qaMkupSdhJibxvkLh3r/y36IARvnYNTLXWvOKxcQ0G/BPUB4+yF5o/yf47vv9zUV5WZv4mrdsKDt/pZDYeDg=="
},
"node_modules/windmill-parser-wasm-py": {
"version": "1.628.3",
@@ -1404,14 +1425,14 @@
"integrity": "sha512-rMBQA8s21wmL2kA5ztRs/ZgVA3ckxe9/NLjxl3iQPL0CX6DlvfaUH0O+AnhpXXDMyBs1Y1SZIhcnbnvsHZ3R8g=="
},
"node_modules/windmill-parser-wasm-rust": {
"version": "1.558.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-rust/-/windmill-parser-wasm-rust-1.558.1.tgz",
"integrity": "sha512-21S7lm1KF8zO1187rbq14hzPHII2RdM2+D44MoAh1F6VoaScj+Puq0z5B1O/hwn/95R/a9jBlL2D8jbkXtlD1A=="
"version": "1.647.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-rust/-/windmill-parser-wasm-rust-1.647.1.tgz",
"integrity": "sha512-9yGLYZX2Hn9TdTqGY/5Fp50ftzgUsrfBkSK9vJkKJd5Amyg+yXLBGzd8pz6Org+4uxMenz/16wpsgijvo6uhhQ=="
},
"node_modules/windmill-parser-wasm-ts": {
"version": "1.623.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-ts/-/windmill-parser-wasm-ts-1.623.1.tgz",
"integrity": "sha512-FBwi/zXxjhZcCvi04oFdNivazru1ynIqSbafHSArfaaBWesBO3nye9UO/WXUlWZm5a7BExbU+3R/eVJrGaornw=="
"version": "1.647.1",
"resolved": "https://registry.npmjs.org/windmill-parser-wasm-ts/-/windmill-parser-wasm-ts-1.647.1.tgz",
"integrity": "sha512-64iSAUMU5W/WtePqE1vtDvglDqtkiZVndyieYBVDX0nl7UuovS+wPgH/P3TEoKbR+FwAPacki0CX3DsEzZ/Yxw=="
},
"node_modules/windmill-parser-wasm-yaml": {
"version": "1.593.0",
+15 -5
View File
@@ -1496,11 +1496,21 @@ async function compareDynFSElement(
}
}
changes.sort((a, b) =>
getOrderFromPath(a.path) == getOrderFromPath(b.path)
? a.path.localeCompare(b.path)
: getOrderFromPath(a.path) - getOrderFromPath(b.path),
);
changes.sort((a, b) => {
const orderA = getOrderFromPath(a.path);
const orderB = getOrderFromPath(b.path);
if (orderA !== orderB) {
return orderA - orderB;
}
// Within the same entity type, process deletes before adds/edits
// to avoid conflicts (e.g. unique path constraints on triggers)
const deletePriority = (name: string) => (name === "deleted" ? 0 : 1);
const dp = deletePriority(a.name) - deletePriority(b.name);
if (dp !== 0) {
return dp;
}
return a.path.localeCompare(b.path);
});
return changes;
}
+1 -1
View File
@@ -67,7 +67,7 @@ export {
workspaceAdd,
};
export const VERSION = "1.645.0";
export const VERSION = "1.649.0";
// Re-exported from constants.ts to maintain backwards compatibility
export { WM_FORK_PREFIX } from "./core/constants.ts";
+20 -5
View File
@@ -339,16 +339,29 @@ export function extractWorkspaceDepsAnnotation(
if (!config) return null;
const { comment, keyword, validityRe } = config;
const extraMarker = `extra_${keyword}:`;
const extraMarkerUnderscore = `extra_${keyword}:`;
const extraMarkerHyphen = `extra-${keyword}:`;
const manualMarker = `${keyword}:`;
const stripComment = (l: string): string | null => {
if (!l.startsWith(comment)) return null;
return l.substring(comment.length).trimStart();
};
const isExtra = (l: string): boolean => {
const s = stripComment(l);
return s !== null && (s.startsWith(extraMarkerUnderscore) || s.startsWith(extraMarkerHyphen));
};
const isManual = (l: string): boolean => {
const s = stripComment(l);
return s !== null && s.startsWith(manualMarker);
};
const lines = scriptContent.split("\n");
// Find first annotation line (mirrors Rust find_position)
let pos = -1;
for (let i = 0; i < lines.length; i++) {
const l = lines[i];
if (l.startsWith(comment) && (l.includes(extraMarker) || l.includes(manualMarker))) {
if (isExtra(lines[i]) || isManual(lines[i])) {
pos = i;
break;
}
@@ -356,10 +369,12 @@ export function extractWorkspaceDepsAnnotation(
if (pos === -1) return null;
const annotationLine = lines[pos];
const mode: AnnotationMode = annotationLine.includes(extraMarker) ? "extra" : "manual";
const mode: AnnotationMode = isExtra(annotationLine) ? "extra" : "manual";
// Parse external references from the annotation line
const marker = mode === "extra" ? extraMarker : manualMarker;
const marker = mode === "extra"
? (annotationLine.includes(extraMarkerUnderscore) ? extraMarkerUnderscore : extraMarkerHyphen)
: manualMarker;
const unparsed = annotationLine.replaceAll(marker, "").replaceAll(comment, "");
const external = unparsed
.split(",")
+32 -5
View File
@@ -46,15 +46,28 @@ function extractWorkspaceDepsAnnotation(
if (!config) return null;
const { comment, keyword, validityRe } = config;
const extraMarker = `extra_${keyword}:`;
const extraMarkerUnderscore = `extra_${keyword}:`;
const extraMarkerHyphen = `extra-${keyword}:`;
const manualMarker = `${keyword}:`;
const stripComment = (l: string): string | null => {
if (!l.startsWith(comment)) return null;
return l.substring(comment.length).trimStart();
};
const isExtra = (l: string): boolean => {
const s = stripComment(l);
return s !== null && (s.startsWith(extraMarkerUnderscore) || s.startsWith(extraMarkerHyphen));
};
const isManual = (l: string): boolean => {
const s = stripComment(l);
return s !== null && s.startsWith(manualMarker);
};
const lines = scriptContent.split("\n");
let pos = -1;
for (let i = 0; i < lines.length; i++) {
const l = lines[i];
if (l.startsWith(comment) && (l.includes(extraMarker) || l.includes(manualMarker))) {
if (isExtra(lines[i]) || isManual(lines[i])) {
pos = i;
break;
}
@@ -62,9 +75,11 @@ function extractWorkspaceDepsAnnotation(
if (pos === -1) return null;
const annotationLine = lines[pos];
const mode: AnnotationMode = annotationLine.includes(extraMarker) ? "extra" : "manual";
const mode: AnnotationMode = isExtra(annotationLine) ? "extra" : "manual";
const marker = mode === "extra" ? extraMarker : manualMarker;
const marker = mode === "extra"
? (annotationLine.includes(extraMarkerUnderscore) ? extraMarkerUnderscore : extraMarkerHyphen)
: manualMarker;
const unparsed = annotationLine.replaceAll(marker, "").replaceAll(comment, "");
const external = unparsed
.split(",")
@@ -185,6 +200,18 @@ def main():
expect(r.inline).toEqual("numpy>=1.24.0");
});
test("python: extra-requirements (hyphen) mode", () => {
const code = `# extra-requirements: utils
#numpy>=1.24.0
def main():
pass`;
const r = extractWorkspaceDepsAnnotation(code, "python3")!;
expect(r.mode).toEqual("extra");
expect(r.external).toEqual(["utils"]);
expect(r.inline).toEqual("numpy>=1.24.0");
});
test("python: empty requirements (opt-out)", () => {
const code = `# requirements:
def main():
+9
View File
@@ -64,6 +64,9 @@ async function createScript(
language: "bun",
is_template: false,
kind: "script",
// Provide a non-empty lock to prevent async lock generation by the backend
// worker, which causes flaky tests due to race conditions on Windows CI.
lock: "\n",
schema: {
$schema: "https://json-schema.org/draft/2020-12/schema",
type: "object",
@@ -255,6 +258,12 @@ async function verifyNoDiffOnPull(backend: any, tempDir: string): Promise<void>
const output = parseJsonFromCLIOutput(pullResult.stdout);
const changes = output.changes || [];
if (changes.length > 0) {
console.error(
`Unexpected changes on dry-run pull (expected 0, got ${changes.length}):`,
JSON.stringify(changes, null, 2)
);
}
expect(changes.length).toEqual(0);
}
-3
View File
@@ -1,3 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_csharp(code: string): string;
-121
View File
@@ -1,121 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
const cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_csharp(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_csharp(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
Binary file not shown.
-29
View File
@@ -1,29 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export const parse_csharp: (a: number, b: number) => [number, number];
export const abort: () => void;
export const malloc: (a: number) => number;
export const calloc: (a: number, b: number) => number;
export const realloc: (a: number, b: number) => number;
export const free: (a: number) => void;
export const strncmp: (a: number, b: number, c: number) => number;
export const iswspace: (a: number) => number;
export const iswalnum: (a: number) => number;
export const clock: () => bigint;
export const isprint: (a: number) => number;
export const fprintf: (a: number, b: number, c: number) => number;
export const fputs: (a: number, b: number) => number;
export const fputc: (a: number, b: number) => number;
export const fdopen: (a: number, b: number) => number;
export const fclose: (a: number) => number;
export const fwrite: (a: number, b: number, c: number, d: number) => number;
export const vsnprintf: (a: number, b: number, c: number, d: number) => number;
export const clock_gettime: (a: number, b: number) => void;
export const snprintf: () => void;
export const __assert_fail: (a: number, b: number, c: number, d: number) => void;
export const __wbindgen_export_0: WebAssembly.Table;
export const __wbindgen_malloc: (a: number, b: number) => number;
export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
export const __wbindgen_free: (a: number, b: number, c: number) => void;
export const __wbindgen_start: () => void;
-3
View File
@@ -1,3 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_go(code: string): string;
-121
View File
@@ -1,121 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
const cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_go(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_go(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
Binary file not shown.
-9
View File
@@ -1,9 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export const parse_go: (a: number, b: number) => [number, number];
export const __wbindgen_export_0: WebAssembly.Table;
export const __wbindgen_malloc: (a: number, b: number) => number;
export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
export const __wbindgen_free: (a: number, b: number, c: number) => void;
export const __wbindgen_start: () => void;
-3
View File
@@ -1,3 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_java(code: string): string;
-121
View File
@@ -1,121 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
const cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_java(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_java(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
Binary file not shown.
-29
View File
@@ -1,29 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export const parse_java: (a: number, b: number) => [number, number];
export const abort: () => void;
export const malloc: (a: number) => number;
export const calloc: (a: number, b: number) => number;
export const realloc: (a: number, b: number) => number;
export const free: (a: number) => void;
export const strncmp: (a: number, b: number, c: number) => number;
export const iswspace: (a: number) => number;
export const iswalnum: (a: number) => number;
export const clock: () => bigint;
export const isprint: (a: number) => number;
export const fprintf: (a: number, b: number, c: number) => number;
export const fputs: (a: number, b: number) => number;
export const fputc: (a: number, b: number) => number;
export const fdopen: (a: number, b: number) => number;
export const fclose: (a: number) => number;
export const fwrite: (a: number, b: number, c: number, d: number) => number;
export const vsnprintf: (a: number, b: number, c: number, d: number) => number;
export const clock_gettime: (a: number, b: number) => void;
export const snprintf: () => void;
export const __assert_fail: (a: number, b: number, c: number, d: number) => void;
export const __wbindgen_export_0: WebAssembly.Table;
export const __wbindgen_malloc: (a: number, b: number) => number;
export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
export const __wbindgen_free: (a: number, b: number, c: number) => void;
export const __wbindgen_start: () => void;
-3
View File
@@ -1,3 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_nu(code: string): string;
-125
View File
@@ -1,125 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
let cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function decodeText(ptr, len) {
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return decodeText(ptr, len);
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_nu(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_nu(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
-125
View File
@@ -1,125 +0,0 @@
let wasm;
export function __wbg_set_wasm(val) {
wasm = val;
}
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const lTextEncoder = typeof TextEncoder === 'undefined' ? (0, module.require)('util').TextEncoder : TextEncoder;
const cachedTextEncoder = new lTextEncoder('utf-8');
const encodeString = (typeof cachedTextEncoder.encodeInto === 'function'
? function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
}
: function (arg, view) {
const buf = cachedTextEncoder.encode(arg);
view.set(buf);
return {
read: arg.length,
written: buf.length
};
});
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
const lTextDecoder = typeof TextDecoder === 'undefined' ? (0, module.require)('util').TextDecoder : TextDecoder;
let cachedTextDecoder = new lTextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
const MAX_SAFARI_DECODE_BYTES = 2146435072;
let numBytesDecoded = 0;
function decodeText(ptr, len) {
numBytesDecoded += len;
if (numBytesDecoded >= MAX_SAFARI_DECODE_BYTES) {
cachedTextDecoder = new lTextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
numBytesDecoded = len;
}
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return decodeText(ptr, len);
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_nu(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_nu(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
export function __wbindgen_init_externref_table() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
};
Binary file not shown.
-9
View File
@@ -1,9 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export const parse_nu: (a: number, b: number) => [number, number];
export const __wbindgen_export_0: WebAssembly.Table;
export const __wbindgen_malloc: (a: number, b: number) => number;
export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
export const __wbindgen_free: (a: number, b: number, c: number) => void;
export const __wbindgen_start: () => void;
-3
View File
@@ -1,3 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_php(code: string, main_override?: string | null): string;
-132
View File
@@ -1,132 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
function isLikeNone(x) {
return x === undefined || x === null;
}
let cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function decodeText(ptr, len) {
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return decodeText(ptr, len);
}
/**
* @param {string} code
* @param {string | null} [main_override]
* @returns {string}
*/
export function parse_php(code, main_override) {
let deferred3_0;
let deferred3_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
var ptr1 = isLikeNone(main_override) ? 0 : passStringToWasm0(main_override, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
var len1 = WASM_VECTOR_LEN;
const ret = wasm.parse_php(ptr0, len0, ptr1, len1);
deferred3_0 = ret[0];
deferred3_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred3_0, deferred3_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
Binary file not shown.
-9
View File
@@ -1,9 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export const memory: WebAssembly.Memory;
export const parse_php: (a: number, b: number, c: number, d: number) => [number, number];
export const __wbindgen_export_0: WebAssembly.Table;
export const __wbindgen_malloc: (a: number, b: number) => number;
export const __wbindgen_realloc: (a: number, b: number, c: number, d: number) => number;
export const __wbindgen_free: (a: number, b: number, c: number) => void;
export const __wbindgen_start: () => void;
-4
View File
@@ -1,4 +0,0 @@
/* tslint:disable */
/* eslint-disable */
export function parse_python(code: string, main_override?: string | null): string;
export function parse_assets_py(code: string): string;
-151
View File
@@ -1,151 +0,0 @@
let WASM_VECTOR_LEN = 0;
let cachedUint8ArrayMemory0 = null;
function getUint8ArrayMemory0() {
if (cachedUint8ArrayMemory0 === null || cachedUint8ArrayMemory0.byteLength === 0) {
cachedUint8ArrayMemory0 = new Uint8Array(wasm.memory.buffer);
}
return cachedUint8ArrayMemory0;
}
const cachedTextEncoder = (typeof TextEncoder !== 'undefined' ? new TextEncoder('utf-8') : { encode: () => { throw Error('TextEncoder not available') } } );
const encodeString = function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
};
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length, 1) >>> 0;
getUint8ArrayMemory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len, 1) >>> 0;
const mem = getUint8ArrayMemory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3, 1) >>> 0;
const view = getUint8ArrayMemory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
ptr = realloc(ptr, len, offset, 1) >>> 0;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
function isLikeNone(x) {
return x === undefined || x === null;
}
let cachedTextDecoder = (typeof TextDecoder !== 'undefined' ? new TextDecoder('utf-8', { ignoreBOM: true, fatal: true }) : { decode: () => { throw Error('TextDecoder not available') } } );
if (typeof TextDecoder !== 'undefined') { cachedTextDecoder.decode(); };
function decodeText(ptr, len) {
return cachedTextDecoder.decode(getUint8ArrayMemory0().subarray(ptr, ptr + len));
}
function getStringFromWasm0(ptr, len) {
ptr = ptr >>> 0;
return decodeText(ptr, len);
}
/**
* @param {string} code
* @param {string | null} [main_override]
* @returns {string}
*/
export function parse_python(code, main_override) {
let deferred3_0;
let deferred3_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
var ptr1 = isLikeNone(main_override) ? 0 : passStringToWasm0(main_override, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
var len1 = WASM_VECTOR_LEN;
const ret = wasm.parse_python(ptr0, len0, ptr1, len1);
deferred3_0 = ret[0];
deferred3_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred3_0, deferred3_1, 1);
}
}
/**
* @param {string} code
* @returns {string}
*/
export function parse_assets_py(code) {
let deferred2_0;
let deferred2_1;
try {
const ptr0 = passStringToWasm0(code, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
const len0 = WASM_VECTOR_LEN;
const ret = wasm.parse_assets_py(ptr0, len0);
deferred2_0 = ret[0];
deferred2_1 = ret[1];
return getStringFromWasm0(ret[0], ret[1]);
} finally {
wasm.__wbindgen_free(deferred2_0, deferred2_1, 1);
}
}
const imports = {
__wbindgen_placeholder__: {
__wbindgen_init_externref_table: function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
},
},
};
const wasm_url = new URL('windmill_parser_wasm_bg.wasm', import.meta.url);
let wasmCode = '';
switch (wasm_url.protocol) {
case 'file:':
wasmCode = (await import('node:fs')).readFileSync(wasm_url);
break
case 'https:':
case 'http:':
wasmCode = await (await fetch(wasm_url)).arrayBuffer();
break
default:
throw new Error(`Unsupported protocol: ${wasm_url.protocol}`);
}
const wasmInstance = (await WebAssembly.instantiate(wasmCode, imports)).instance;
const wasm = wasmInstance.exports;
export const __wasm = wasm;
wasm.__wbindgen_start();
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More