mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-18 16:02:10 +00:00
2d18a680991babe317ca315bbce40e6ce733afda
24 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
2d18a68099 |
feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
 hugocasa
|
bffa61e33f |
fix: dedicated worker dispatch, cross-workspace deps, UI improvements (#8689)
* feat: restore bun as default runtime for dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add context comment for bun dedicated worker nodejs migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for flows + add E2E tests - Add workspace_id prefix to dedicated worker map lookup keys - Update ee-repo-ref for dedicated worker path handling fix - Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers - Add 6 E2E tests for dedicated workers: - test_dedicated_flow_rawscript (regression for "Script not found" bug) - test_dedicated_flow_workspace_script - test_dedicated_flow_multiple_steps - test_dedicated_standalone_script - test_dedicated_runner_group - test_dedicated_flow_runners - Add dedicated_flows.sql fixture with scripts, flows, and worker config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always run dependency job for dedicated worker scripts When a script with dedicated_worker=true is deployed with a pre-computed lock (e.g. via wmill sync push), no dependency job was created, so the dedicated worker never detected the update and kept running the old version. Now dedicated worker scripts always generate a dependency job regardless of whether a lock is provided. The dependency job runs on the dedicated worker and triggers a restart so it picks up the new script version. Fixes #8638 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races Dedicated worker tests need non-default worker tags in the global WORKER_CONFIG. When run in parallel (CI uses --test-threads=10), multiple tests clobber each other's config. Use #[serial] to ensure dedicated worker tests run sequentially. Also load worker config from DB via load_worker_config() instead of manually setting WORKER_CONFIG fields, ensuring consistency with the monitor's reload path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests Fix script_path shadowing in bun_executor nodejs branch where the wrapper file path was passed to handle_dedicated_process instead of the logical path, causing "Script not found" for all //nodejs dedicated workers. Add E2E tests for dedicated flows in all supported languages: - test_dedicated_flow_deno - test_dedicated_flow_python - test_dedicated_flow_bunnative (V8 PrewarmedIsolate path) - test_dedicated_flow_bun_nodejs (//nodejs annotation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify dedicated worker dispatch + add serialization and E2E tests - Unified lookup: always use {workspace}:{runnable_path} for dedicated worker dispatch, replacing the flow_step_id iteration approach - Added serialization_semaphore parameter to executor start_worker fns - Added E2E tests: cross-workspace isolation, conflicting flow step IDs, preprocessor on dedicated worker - Added workspace field to RunJob for cross-workspace test support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: cross-workspace workspace dependencies on workers page Add two new instance-level endpoints to the configs router: - GET /configs/list_all_workspace_dependencies - GET /configs/list_all_dedicated_with_deps Both require devops role and return data across all workspaces, enabling the workers page to show a consistent view of which workspace dependencies exist regardless of which workspace the user is browsing. Update DedicatedWorkersSelector to use the new cross-workspace endpoints with fallback to per-workspace calls for non-devops users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include dedicated worker lookup simplification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update ee-repo-ref.txt with new reference * sqlx * fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings - Remove serialization_semaphore from executor start_worker signatures - Remove serialization test and fixtures - Fix DedicatedWorkersSelector to preserve tags from other workspaces when toggling in the picker - Track workspace deps per-workspace for conflict detection - Show warning when dep exists in another workspace but not the script's - Group runner groups per-workspace to prevent cross-workspace merging - Add workspace to dep badge link URL Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify exec protocol — execd: for single-script, exec: for runner groups Add execd:/execd_preprocess: commands to bun/deno/python wrappers for single-script dedicated workers (no path needed). Runner groups keep exec:/exec_preprocess: with path for multi-script disambiguation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add unit tests for execd:/exec: wrapper protocol Verify generate_multi_script_wrapper produces both execd: (single-script) and exec: (runner group) protocol handlers, including preprocessor variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update commit reference in ee-repo-ref.txt * fix: remove beta badge from squash loop, keep tooltip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update protocol tests to use execd: for single-script wrappers Deno and bun single-script protocol tests now send execd:{args} instead of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script (runner group) tests continue to use exec:{path}:{args}. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused TEST_SCRIPT_PATH in deno protocol tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — down migration, push_as workspace, UI improvements - Use regexp_replace in down migration for positional accuracy - Fix push_as() to use self.workspace_id instead of hardcoded value - Remove per-workspace API fallbacks, use cross-workspace endpoints only - Skip devops-only API calls when user is not devops (disabled prop) - Fix duplicate key error for cross-workspace runner groups - Add workspace to RunnerGroup for unique keying - Reuse tagRow snippet for standalone items with expand/collapse - Fix picker alignment: remove empty column for non-expandable items Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: comprehensive dedicated worker test coverage, fix Python execd_preprocess - Add Python execd_preprocess: handler (was missing for single-script dedicated workers) - Add 10 E2E tests: flow+standalone conflict, mixed lang fallback, unsupported lang flow runners, python runner group, bun/python/deno/bunnative preprocessors, runner group preprocessors, branchone flow - Add 4 Python unit tests for execd:/execd_preprocess: protocol - Update EE ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: review feedback — migration escaping, deno try/catch, loadRunnables guard - Down migration: use E'...' so \n matches actual newlines - Up migration: anchor regex with ^ to avoid mid-content matches - Deno execd_preprocess: move JSON.parse inside try/catch - DedicatedWorkersSelector: skip devops-only API calls when disabled Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add dedicated worker relative import tests for bun and python Verifies that build_loader's CURRENT_PATH correctly resolves workspace- relative imports when running on a dedicated worker subprocess. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for nested flow structures (branches/loops) - Add extract_flow_root() to strip nesting segments from runnable_path - Dispatch uses flow_root/flow_step_id for nested paths, runnable_path for flat paths — deterministic, O(1) - Fix assert_ran_on_dedicated_worker to BFS all descendants - Fix python mode labels (python vs python3 for runner groups) - Add tests: simple forloop, multi-step forloop, whileloop, branchall, nested branch-in-loop, mixed lang fallback, unsupported lang runners Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: fix ee-repo-ref SHA Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hide picker and skip API calls for read-only users, hide empty runner badge - Hide "Add more scripts/flows" section when disabled (read-only) - Skip per-runnable API calls (getScriptByPath, getFlowByPath) for disabled users — just show path info - Hide "0 runners" badge on flows with no eligible steps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 9422b189762ae27edfc346541ae668a4ad728325 This commit updates the EE repository reference after PR #503 was merged in windmill-ee-private. Previous ee-repo-ref: 4c6ba214bfc23fff05d1dc3200ac59e650af3f4f New ee-repo-ref: 9422b189762ae27edfc346541ae668a4ad728325 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
c4c9ef5fd7 |
feat: add optional labels to scripts, flows, apps, schedules, triggers (#8609)
* feat: add optional labels to scripts, flows, apps, raw apps, schedules, and triggers
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: update sqlx cache, make labels optional in openapi, regenerate system prompts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add minimal labels input UI to script, flow, and schedule editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce gap between summary and labels input
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to script/flow detail pages and summary/path popover
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move labels inside SummaryPathDisplay trigger for clickable area, reduce gap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: display labels inline to the right of summary, not below
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase gap between summary and labels
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to resources/variables, make labels nullable, add home page label filter badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels to workspace export/import, resources, variables + test coverage
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make migration idempotent, regenerate sqlx cache after merge
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in script create and flow create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add labels input UI to resource and variable editors
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove negative margin from LabelsInput to prevent overlap
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add top and left margin to LabelsInput for better spacing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce left margin on LabelsInput
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: widen label input to w-32
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use inline-flex so LabelsInput doesn't stretch full width
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: remove flex-wrap so label input stays on same line as badges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label filter presets to resources, variables, and schedules search
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use max-w-32 on label input to prevent stretching
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pull labels closer to summary with negative top margin
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase negative margin to pull labels even closer to summary
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass labels in schedule create/update API calls
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use COALESCE to preserve existing labels when not provided in schedule/flow update
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add labels to CreateResource, EditResource, CreateVariable, EditVariable in OpenAPI spec
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on resource and variable list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label badges on schedule and all trigger list pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add folder and label presets to schedules search filter
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: apply user_folders_only filter on all workspaces including admins
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add label presets to resources and variables search filters
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: derive folder presets from loaded items, not all workspace folders
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add label query parameter to resource and variable list endpoints in OpenAPI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: display label filter badges inline with folder filters on home page
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat: display label filter badges inline with folder filters on home page"
This reverts commit
|
||
|
hugocasa
|
61a867f086 |
Revert "feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)" (#8687)
This reverts commit
|
||
|
hugocasa
|
619ebb65ce |
feat: restore bun for dedicated workers, fix dispatch & serialization, cross-workspace deps (#8645)
* feat: restore bun as default runtime for dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: add context comment for bun dedicated worker nodejs migration Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker dispatch for flows + add E2E tests - Add workspace_id prefix to dedicated worker map lookup keys - Update ee-repo-ref for dedicated worker path handling fix - Add spawn_test_worker_dedicated/in_test_worker_dedicated test helpers - Add 6 E2E tests for dedicated workers: - test_dedicated_flow_rawscript (regression for "Script not found" bug) - test_dedicated_flow_workspace_script - test_dedicated_flow_multiple_steps - test_dedicated_standalone_script - test_dedicated_runner_group - test_dedicated_flow_runners - Add dedicated_flows.sql fixture with scripts, flows, and worker config Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: always run dependency job for dedicated worker scripts When a script with dedicated_worker=true is deployed with a pre-computed lock (e.g. via wmill sync push), no dependency job was created, so the dedicated worker never detected the update and kept running the old version. Now dedicated worker scripts always generate a dependency job regardless of whether a lock is provided. The dependency job runs on the dedicated worker and triggers a restart so it picks up the new script version. Fixes #8638 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use serial_test for dedicated worker tests to avoid WORKER_CONFIG races Dedicated worker tests need non-default worker tags in the global WORKER_CONFIG. When run in parallel (CI uses --test-threads=10), multiple tests clobber each other's config. Use #[serial] to ensure dedicated worker tests run sequentially. Also load worker config from DB via load_worker_config() instead of manually setting WORKER_CONFIG fields, ensuring consistency with the monitor's reload path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: nodejs dedicated worker script_path shadowing + add multi-language E2E tests Fix script_path shadowing in bun_executor nodejs branch where the wrapper file path was passed to handle_dedicated_process instead of the logical path, causing "Script not found" for all //nodejs dedicated workers. Add E2E tests for dedicated flows in all supported languages: - test_dedicated_flow_deno - test_dedicated_flow_python - test_dedicated_flow_bunnative (V8 PrewarmedIsolate path) - test_dedicated_flow_bun_nodejs (//nodejs annotation) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify dedicated worker dispatch + add serialization and E2E tests - Unified lookup: always use {workspace}:{runnable_path} for dedicated worker dispatch, replacing the flow_step_id iteration approach - Added serialization_semaphore parameter to executor start_worker fns - Added E2E tests: cross-workspace isolation, conflicting flow step IDs, preprocessor on dedicated worker - Added workspace field to RunJob for cross-workspace test support Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: cross-workspace workspace dependencies on workers page Add two new instance-level endpoints to the configs router: - GET /configs/list_all_workspace_dependencies - GET /configs/list_all_dedicated_with_deps Both require devops role and return data across all workspaces, enabling the workers page to show a consistent view of which workspace dependencies exist regardless of which workspace the user is browsing. Update DedicatedWorkersSelector to use the new cross-workspace endpoints with fallback to per-workspace calls for non-devops users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to include dedicated worker lookup simplification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: use branch name for ee-repo-ref (CI can't fetch by SHA from non-default branch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update ee-repo-ref.txt with new reference * sqlx * fix: revert serialization semaphore, multi-workspace picker, dep conflict warnings - Remove serialization_semaphore from executor start_worker signatures - Remove serialization test and fixtures - Fix DedicatedWorkersSelector to preserve tags from other workspaces when toggling in the picker - Track workspace deps per-workspace for conflict detection - Show warning when dep exists in another workspace but not the script's - Group runner groups per-workspace to prevent cross-workspace merging - Add workspace to dep badge link URL Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: simplify exec protocol — execd: for single-script, exec: for runner groups Add execd:/execd_preprocess: commands to bun/deno/python wrappers for single-script dedicated workers (no path needed). Runner groups keep exec:/exec_preprocess: with path for multi-script disambiguation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add unit tests for execd:/exec: wrapper protocol Verify generate_multi_script_wrapper produces both execd: (single-script) and exec: (runner group) protocol handlers, including preprocessor variants. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update commit reference in ee-repo-ref.txt * fix: remove beta badge from squash loop, keep tooltip Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update protocol tests to use execd: for single-script wrappers Deno and bun single-script protocol tests now send execd:{args} instead of exec:{path}:{args}, matching the updated wrapper protocol. Multi-script (runner group) tests continue to use exec:{path}:{args}. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused TEST_SCRIPT_PATH in deno protocol tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a46aa641f9 |
feat: add R language support (#8263)
* feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com> |
||
|
Ruben Fiszel
|
f40cdaf434 |
fix(cli): app push crash, lint path, push --message, run validation, history timestamps (#8585)
* fix(cli): app push crash, lint entry point, push --message, run arg validation, history timestamps Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): update sqlx cache and fix second history query missing created_at Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(cli): regenerate system prompts after new CLI options Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
0389d9601c |
chore: upgrade axum 0.7 to 0.8 (#8539)
* chore: upgrade axum 0.7 to 0.8 and related dependencies Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add route reachability tests for ~80 previously untested endpoints Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: switch feature-gated trigger handlers from axum::async_trait to async_trait crate Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update new trash routes to axum 0.8 path syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to latest EE commit Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: upgrade route tests to assert 2xx responses with proper data setup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: restore npm_proxy and ai_routes tests using local echo servers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate workspace fork test behind enterprise feature flag Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add ~40 more endpoint tests (jobs authed, health, favorites, ACLs, reachability) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review findings from axum 0.8 upgrade - Use cookie value_trimmed() instead of value() for cookie 0.18 compat - Update comments still referencing old :workspace_id syntax Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 61ae055ea31481f1899953e9d5f65566b8c707b1 This commit updates the EE repository reference after PR #486 was merged in windmill-ee-private. Previous ee-repo-ref: 0059d175a6fdddf52998b183bf91059b224704ac New ee-repo-ref: 61ae055ea31481f1899953e9d5f65566b8c707b1 Automated by sync-ee-ref workflow. * test: add test for new get_imports endpoint Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove unused import in raw_apps test Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
71549c3db0 |
fix: resolve parent_hash race condition in sync push with auto_parent (#8545)
* fix: resolve parent_hash race condition in sync push with auto_parent During concurrent sync push operations (parallel CLI groups or separate CI pipelines), multiple requests could read the same remote script hash and both try to create a new version with the same parent_hash, causing "the lineage must be linear" errors. Adds an opt-in `auto_parent` field to the create_script API. When set, the backend resolves the parent_hash to the current head script at that path within the transaction, atomically. This eliminates the client-side race window where the parent could change between read and write. The CLI now sends `auto_parent: true` when updating existing scripts, so sync push is resilient to concurrent deployments. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing auto_parent field in clone_script NewScript initializer Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add advisory lock to serialize concurrent auto_parent script creates Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * sqlx * fix: add sqlx anchor for CE-only user count query Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
69ce946241 | feat: add trashbin system for soft-deleting items (#8519) | ||
|
hugocasa
|
c28314f424 |
feat: runner groups for shared-process multi-script dedicated workers (#8434)
* feat: add runner groups for shared-process multi-script dedicated workers Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: unify dedicated worker and runner group wrappers into single multi-script wrapper Replace per-language single-script wrappers with the unified load/exec/exec_preprocess/end protocol. Each start_worker() now writes scripts to scripts/<safe_name>/ and uses generate_multi_script_wrapper(). handle_dedicated_process() sends load: on start and exec: per job instead of raw JSON args. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: merge runner groups into dedicated workers with inline arg metadata Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to match EE branch Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate EE-only functions behind cfg(feature = "private") to fix OSS dead_code errors Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: auto-detect runner groups from workspace dependency annotations - New endpoint GET /scripts/list_dedicated_with_deps: returns dedicated scripts with parsed workspace dependency names from content annotations - Frontend: show dep badges in DedicatedWorkersSelector with links to workspace settings, warn when referenced dep doesn't exist, group scripts sharing deps into "Shared runner" sections - Remove manual "Runner groups" tab and RunnerGroupSelector component - Remove runner_groups from WorkerConfigOpt/WorkerConfig (auto-detected) - Fix Node.js single dedicated workers: transpile main.ts -> main.js via Bun.build so the multi-script wrapper's dynamic import() works under Node - Add package.json with type:module in scripts dir to silence Node warning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: unify dedicated worker wrappers with baked-in codegen and routing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add e2e tests for multi-script dedicated worker routing (bun, deno, python) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove dead generate_dedicated_worker_wrapper function Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add dependency installation to runner groups + make dep functions pub(crate) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: prevent bun loader from intercepting absolute paths within cwd When a plugin's onResolve returns an absolute path, Bun re-invokes the resolver with that path. The loader was then routing it through the remote URL resolver, breaking runner group script imports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use _wm_ prefix for runner group scripts to avoid bun loader interception Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: extract DENO_UNSTABLE_ARGS constant to avoid repeating flags Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate system prompts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate private-only exports behind cfg(feature = "private") for OSS build Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: move format strings before handle_dedicated_process to fix lifetime Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx offline cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix sqlx * fix: skip empty lines in deno e2e tests (double newline from console.log + '\n') Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use dict() instead of {{}} in python wrapper to avoid set literal {{{{}}}} in format!() produces {{}} which Python interprets as an empty set, not a dict. Use dict() which is unambiguous. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove deno from runner groups and associated tests Deno resolves dependencies at runtime via URLs/import maps, so there's no shared node_modules/pip install to benefit from runner groups. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: revert deno wrapper to inline old-style with exec: protocol Since deno doesn't support runner groups, the unified multi-script wrapper is unnecessary. Reverted to the old inline wrapper from main but adapted to use the exec:<path>:<args> protocol. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: extract deno wrapper into reusable function and add e2e tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use codebase presence (not nodejs annotation) to determine wrapper import extension On main, codebase scripts import ./main.js (pre-bundled JS). The wrapper_ext was incorrectly based on annotation.nodejs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: improve dedicated workers UI - combine lists, better badges, tooltips - Merge shared runners section with selected tags into one unified list - Move language tag to right side of selector for alignment - Change dep badge color from dark-gray to indigo - Add tooltip on yellow warning badge explaining missing workspace dep Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: group shared runners visually in dedicated workers list - Runner groups shown with a header (Shared runner · language · dep badge) - Scripts in the same group nested under the header - Standalone scripts/flows shown after groups - Used Svelte snippet for reusable tag row rendering Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: improve visual separation between shared runner groups and standalone items Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: give standalone runners same header style as shared runners - Each standalone script/flow gets its own header row with bg-surface-secondary - Header shows "Dedicated runner" / "Flow runner" label, dep link, language badge - Shared runner header: swapped language and dep badge positions - Dep shown as inline link instead of badge in headers for cleaner look Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: inline standalone runner path in header, language badge on right edge, no max height - Standalone items: path shown directly in header row (no sub-row) - Language badge placed after flex-1 spacer (right-aligned) - Removed max-h-64 overflow constraint from the list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: consistent badges across runner list - dep+language on right, depBadge snippet - Shared runner scripts: show (workspace) and language badge on right - Standalone items: dep badges and language badge on right (after flex-1) - Shared runner header: dep badge and language badge on right - Extract depBadge snippet to deduplicate dep badge rendering - Picker selector also uses depBadge snippet Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: show language badge on standalone items, hide from shared runner sub-items - Fetch script language from API when not available from workspace deps - Hide dep+language badges from tagRow when script is inside a runner group (already shown in the group header) - Standalone items now always show language badge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: differentiate badge colors - gray for language, indigo for workspace deps Matches codebase convention: gray for metadata (like script hashes), indigo for linkable features/entities. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use transparent (bordered) badge for language - visible on all backgrounds Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use gray badge for language everywhere Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: revert skills.ts and AI files, add _wm_ exclusion to Windows loader - Revert cli/src/guidance/skills.ts to main (not our change) - Revert AI provider formatting changes (not our change) - Add _wm_ prefix exclusion to loader.bun.windows.js filterResolve Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update ee-repo-ref and regenerate system prompts after merge Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use DISTINCT ON in list_dedicated_with_deps to dedup at DB level Avoids fetching all script versions and deduplicating in Rust. Addresses PR review feedback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use sqlx query! macro for list_dedicated_with_deps and regenerate cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: dedicated worker review fixes and test coverage - Fix Python relative imports in dedicated workers (write loader.py, add import loader to wrapper when needed) - Move Python colon parsing inside try/except to prevent crashes on malformed stdin - Add indexOf guard in Bun/Deno wrappers for malformed protocol messages - Add stderr logging for unrecognized stdin commands in all wrappers - Remove asyncio handling from Python wrapper (consistent with normal path) - Add exec_preprocess protocol tests for Bun, Deno, and Python - Add argument transformation tests (dates, bytes, kwargs, sentinel) - Add relative import detection test for Python wrapper - Add PreprocessedArgs variant to DedicatedWorkerResult test helper Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove symlink from git and gate has_relative_imports behind private feature Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: update ee-repo-ref for dedicated_worker_ee.rs changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add mixed exec+preprocess test to use ProtocolCmd::Exec variant Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove hanging deno missing-preprocessor test The Deno wrapper only generates the exec_preprocess handler when the script has a preprocessor function. Without one, the message is unrecognized and the test hangs reading stdout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 182943e5ad9bf2a905ccdf07d4e346437fb329a9 This commit updates the EE repository reference after PR #466 was merged in windmill-ee-private. Previous ee-repo-ref: 995f701fe3754be6260fc6b679e5de8fc636e68a New ee-repo-ref: 182943e5ad9bf2a905ccdf07d4e346437fb329a9 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
 Pyra
|
9643006f1e |
feat(cli): better stale scripts detection #3 (#8480)
* fix Signed-off-by: pyranota <pyra@duck.com> * reduce tests Signed-off-by: pyranota <pyra@duck.com> * update Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * update Signed-off-by: pyranota <pyra@duck.com> * WIP: stash changes after merge with origin/main * Delete backend/parsers/windmill-parser-wasm/Cargo.lock * reset cargo.toml * feat(cli): integrate dependency tree into generate-metadata command - Add isDirectlyStale field to DependencyNode for staleness tracking - Update addScript to accept itemType, folder, isRawApp, isDirectlyStale - Update propagateStaleness to use isDirectlyStale field instead of parameter - Handlers now determine staleness and pass it to tree.addScript - generate-metadata calls propagateStaleness() and populates staleItems from tree - Pass legacyBehaviour=false and tree to handlers during generation phase 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(cli): store originalPath in tree for correct handler invocation Scripts need the path with extension to be passed to the handler. Added originalPath field to DependencyNode to track this. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix parsers Signed-off-by: pyranota <pyra@duck.com> * rever sqlx removal * update sqlx * feat: make py-imports parser WASM-compatible and add as separate WASM package Gate heavy deps (sqlx, windmill-common, async-recursion, toml, pep440_rs, tracing) behind cfg(not(wasm32)). Make parse_code_for_imports, parse_relative_imports, NImport, and ImportPin public. Remove duplicate import_parser from parser-py (reset to origin/main). Add py-imports-parser feature to windmill-parser-wasm and py-imports target to build.nu. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * safer return * update * fix: CLI metadata fixes - folder filter, staleness detection, WASM py-imports setup - Fix lazy_static cfg gating for WASM compatibility (split into separate blocks) - Fix folder argument filter to match specific file paths (not just directories) - Fix staleness detection to use checkHash with conf (includes module hashes) - Convert relative_imports_skip tests from Deno to bun APIs - Add windmill-parser-wasm-py-imports to CLI and build-npm dependencies - Relax module stale test to not require per-module change detail in output Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: restore temp_script_refs parameter in parse_python_imports Re-adds the temp_script_refs parameter that was lost when resetting py-imports crate to origin/main. This enables resolving relative imports from not-yet-deployed scripts during CLI lock generation. * fixes * extend testsuit * update ee repo ref * fix: diff endpoint bytea cast, upload only mismatched scripts - Add POST /scripts/raw_temp/diff endpoint to batch-compare local content hashes against deployed versions using Postgres sha256() - Use convert_to(content, 'UTF8') instead of content::bytea to avoid failure on scripts containing backslash sequences (e.g. \n) - CLI now diffs all scripts against deployed, uploads only mismatched ones - propagateStaleness no longer deletes non-stale nodes (needed for diff) - Suppress verbose log.info messages during metadata generation - Add E2E tests for locally modified and unpushed helper scripts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * rework * sqlx * fixes * add index * expand tests * fix flows * archive script before executing * disable tests for ci * skip Python-dependent E2E tests on CI Tests requiring the python backend feature are skipped when CI_MINIMAL_FEATURES=true since CI builds with zip-only features. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: make flow fixture lock optional and reset nonDottedPaths after tests Flow fixtures no longer emit an empty lock file by default. The lockContent parameter controls whether a lock: "!inline ..." line appears in flow.yaml. This prevents flows from appearing "up-to-date" when they should be processed by generate-metadata. Also adds afterAll to reset setNonDottedPaths(false) so global state doesn't leak between test files when run together. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add error logging in withTestBackend to diagnose CI failures Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add --bail 1 to CI test runner to show full error on first failure Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: include CLI stdout/stderr in assertion message for workspace deps test Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: set WMDEBUG_FORCE_V0_WORKSPACE_DEPENDENCIES in test backend The workspace deps feature requires workers to report their version, but in test/CI there are no separate workers (standalone mode). The version check fails because workers haven't had time to ping yet. Setting this env var bypasses the version check. Also reverts --bail 1 from CI workflow now that the root cause is fixed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add --bail 1 to Windows CI and assertion messages for Windows failure diagnosis Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: replace TEMP_SCRIPT_REFS_PLACEHOLDER in bun builder tests The loader.bun.js now includes a TEMP_SCRIPT_REFS_PLACEHOLDER that must be replaced before execution. The builder tests were missing this replacement, causing all 6 bun_builder_tests to fail. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use cdirFwd in Windows loader filterLoad regex Raw cdir (with backslashes) interpolated into RegExp causes \r to become carriage return and \w to become word-char, so filterLoad never matches main.ts. This prevents replaceRelativeImports from running, leaving bare relative imports like "./script_b" in the bundled output, which scanImports then misparses as package ".". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: Windows filterLoad regex + graceful fallback for old backends - Fix filterLoad in loader.bun.windows.js to match both native backslash and forward-slash paths from Bun's resolver by escaping cdir for regex - Wrap uploadScripts in try/catch so generate-metadata degrades gracefully when the backend lacks /raw_temp endpoints (locks use deployed versions) - Add TODO for missing TEMP_SCRIPT_REFS support in Windows loader Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add loader/builder debug logging for Windows CI diagnosis Temporary console.log statements to understand: - What path Bun passes to onLoad for main.ts - Whether filterLoad regex matches - Whether replaceRelativeImports fires - What the bundled output contains - What imports scanImports extracts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: trigger CI for cli path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: trigger CI via workflow file change Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Add TEMP_SCRIPT_REFS to Windows loader, use .ts extensions in test imports - Add TEMP_SCRIPT_REFS_PLACEHOLDER support to loader.bun.windows.js (mirrors loader.bun.js) so CLI lock generation can resolve imports from locally-modified scripts on Windows - Use .ts extensions in all test relative imports to work around the Windows filterLoad regex bug (replaceRelativeImports doesn't fire on Windows, so extensionless imports fail) - Remove unused uploadSucceeded variable Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove debug logging from loader_builder.bun.js Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove windmill-parser-wasm-py-imports from frontend package.json This dependency is only needed by the CLI, not the frontend. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * debug: add temp_script_refs logging for Windows CI investigation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: remove --bail 1 from Windows CLI tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: normalize backslashes in folder filter treePath lookup (Windows) On Windows, item.path (originalPath) uses backslashes but tree keys use forward slashes. The isRelevant filter's touchesFolder call passed the unnormalized path to traverseTransitive, which couldn't find the node. This caused cross-folder importers to be excluded from generate-metadata when a folder argument was specified. Also removes debug logging from previous commit. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update cli-tests.yml * fix: normalize backslashes in strict-folder-boundaries warning message (Windows) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: update ee-repo-ref to fe8f0d1d7448464c98474d994e6492c0a45e8e38 This commit updates the EE repository reference after PR #467 was merged in windmill-ee-private. Previous ee-repo-ref: 03e6eaf950776c96b9581848a583af9ad735be60 New ee-repo-ref: fe8f0d1d7448464c98474d994e6492c0a45e8e38 Automated by sync-ee-ref workflow. * revert cli-tests.yml --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
391da1d5af |
add cloud quota usage display and version pruning (#8433)
* feat: add cloud quota usage display and version pruning Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: hard-delete pruned scripts so quota actually decreases Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: update quota error messages to reference workspace settings Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
31d6660d56 |
feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
 wendrul
|
7ac93f6ee3 | feat: option to preserve on_behalf_of and edited_by for admins and users in the new wm_deployers group (#8079) | ||
|
Ruben Fiszel
|
6bf544f507 |
refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend Consolidate all object_store-dependent code from windmill-common into a new windmill-object-store crate. Add a filesystem-backed object store implementation using LocalFileSystem for dev/testing without cloud credentials. Includes 30 comprehensive tests covering render_endpoint, lfs_to_object_store_resource, duckdb_connection_settings, error mapping, and filesystem-backed integration tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * all * fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
hugocasa
|
90fa5b3ced |
native mode nits (#7981)
* native mode UI nits * accept flow jobs on native workers * limit native mode to non-dep jobs + flow tag infobox |
||
|
hugocasa
|
aaa1b92300 |
fix: hash long dedicated worker tags (#7914)
* fix: hash long dedicated worker tags * Update frontend/src/lib/components/dedicated_worker.ts Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
b9e0ccaefd | chore: split api-scripts and api-flows | ||
|
wendrul
|
2851b6b7ca | feat: restriction rulesets for workspaces (#7879) | ||
|
Ruben Fiszel
|
7d37a83d4f | nit backend tests | ||
|
Ruben Fiszel
|
ccc4806b9a |
refactor: extract windmill-types crate from windmill-common (#7851)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
bb57f8cd29 | remove unecessary deps for vanilla cargo run | ||
|
Ruben Fiszel
|
c580572252 |
refactor: extract windmill-api-scripts and windmill-api-users subcrates (#7850)
* refactor: extract windmill-api-scripts and windmill-api-users subcrates Split the monolithic windmill-api crate by extracting scripts.rs, flows.rs, users.rs, and users_oss.rs into dedicated subcrates. This reduces incremental rebuild times when editing these modules. Changes: - Create windmill-api-scripts crate (scripts.rs + flows.rs, ~4.3K lines) - Create windmill-api-users crate (users.rs + users_oss.rs, ~2.4K lines) - Move clear_schedule to windmill-queue (shared by scripts, flows, workspaces) - Move username utilities (VALID_USERNAME, INVALID_USERNAME_CHARS, generate_instance_wide_unique_username) to windmill-common/src/usernames.rs - Move COOKIE_DOMAIN, IS_SECURE, WithStarredInfoQuery, BulkDeleteRequest, WebhookShared to windmill-common for cross-crate access - Original files in windmill-api become thin stubs with pub use re-exports - EE-dependent route handlers remain in windmill-api (create_user, rename_user, set_password, reset_password, etc.) - Feature forwarding for enterprise, private, parquet, no_auth Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: extract windmill-api-workspaces subcrate (Step 3) Move workspaces.rs, workspaces_extra.rs, workspaces_oss.rs, and workspaces_ee.rs into a new windmill-api-workspaces crate (~7K lines). Routes that depend on windmill-api internals (AI copilot, teams, tarball export, critical alerts, stripe) remain in the windmill-api stub. The subcrate handles all other workspace management routes. Also moved send_email_if_possible to windmill-common/email_oss.rs to make it available across subcrates without circular deps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * refactor: extract windmill-api-groups subcrate (groups.rs + folders.rs) Extract groups.rs (1,093 lines) and folders.rs (833 lines) into a new windmill-api-groups subcrate. Both modules had clean dependencies on already-extracted crates (windmill-api-auth, windmill-common, windmill-api-workspaces). Also removes unused re-exports of get_instance_username_or_create_pending and INVALID_USERNAME_CHARS from windmill-api/src/utils.rs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: add granular_acls.rs and folder_history.rs to windmill-api-groups Extract granular_acls.rs (395 lines) and folder_history.rs (68 lines) into the windmill-api-groups subcrate. Both modules only depend on already-extracted crates and belong to the same access-control domain as groups and folders. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: remove unused imports and dead code from subcrate extraction - Remove unused BASE_URL import from lib.rs - Remove workspaces_extra.rs and workspaces_oss.rs re-export stubs (no consumers in windmill-api) - Remove dead send_email_if_possible OSS stub (callers moved to windmill-api-users) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * chore: bust CI cargo cache for subcrate split Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: re-export BASE_URL for EE files that use crate::BASE_URL Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: forward no_auth feature to windmill-api-users, remove dead code - Add "windmill-api-users/no_auth" to windmill-api's no_auth feature so the login bypass in users.rs:1600 activates correctly - Remove dead send_email_if_possible from windmill-api-users/users_oss.rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: re-enable cargo cache for backend tests Cache was disabled to bust stale entries from before subcrate split. Now that a clean build has run, re-enable for faster CI. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: install mold+clang in CI workflows The .cargo/config.toml uses mold linker for x86_64-linux. Build scripts require linking even during cargo check. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: increase cargo test timeout to 30 min Exit code 143 (SIGTERM) means the 20-min timeout was hit during compilation without cache. Bump to 30 min as safety net. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: limit cargo build jobs to 4 to prevent OOM in CI Exit code 143 (SIGTERM) after 8 min = OOM kill during compilation. 8 parallel LLVM codegen jobs exhaust memory on ubicloud-standard-8. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |