mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-21 08:02:26 +00:00
worker-batch-pull-write
224 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
2d18a68099 |
feat: add scheduled job deletion with configurable retention period (#8753)
* feat: add scheduled job deletion with configurable retention period Extends delete_after_use with delete_after_secs to enable configurable retention periods for job args/result/logs. At completion, jobs can be scheduled for future deletion via a new job_delete_schedule table, processed by a monitor task. Supports per-script, per-flow, and per-flow-step configuration. Backward compatible. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: add integration tests, revert query! macros, fix review issues - Add integration tests for resolve_delete_after_secs, schedule_job_deletion, flow-level and module-level delete_after_secs, backward compat - Revert sqlx::query() back to sqlx::query!() macros for compile-time safety - Regenerate sqlx offline cache - Fix FlowModule/NewScript/FlowValue constructions in all test files - Fix autoscaling_ee.rs for updated script_path_to_payload return type Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref.txt for autoscaling_ee fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: gate cleanup_scheduled_job_deletions behind enterprise feature Prevents dead_code warning (which CI treats as error via -D warnings) when compiling without enterprise feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx cache after merge with main Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback on scheduled deletion - Monitor: roll back transaction on any cleanup error so schedule rows survive for retry on next cycle (instead of best-effort then discard) - Migration: add FK with ON DELETE CASCADE to job_delete_schedule.job_id to prevent orphan rows when jobs are deleted through other means - Simplify bool-to-Option conversion with .then_some(true) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: stop setting delete_after_use alongside delete_after_secs No mixed-version deployment scenario exists, so delete_after_secs alone is sufficient. The backend's resolve_delete_after_secs handles (None, Some(secs)) correctly without needing delete_after_use set. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: remove delete_after_use from public API surface Remove delete_after_use from OpenAPI spec, API client, runtime client, and workspace export. Only delete_after_secs is exposed going forward. The field remains in Rust backend types with #[serde(skip_serializing)] for backward-compatible deserialization of existing scripts/flows that were saved with delete_after_use: true. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 This commit updates the EE repository reference after PR #519 was merged in windmill-ee-private. Previous ee-repo-ref: 9eba09a13b778caafc6ae65098b90e53c91984d3 New ee-repo-ref: 1d4b7a31fc115d6aba8640f7cd3fd5a01abe6806 Automated by sync-ee-ref workflow. * fix: regenerate system prompts, remove unused import - Regenerate auto-generated system prompts after openflow schema change - Remove unused serde_json::json import in test file (CI -D warnings) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: insert dummy v2_job row in schedule tests for FK constraint The job_delete_schedule table has a FK to v2_job, so tests need a real v2_job row before inserting into the schedule table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: trigger CI re-run * fix: remove heavy flow integration tests to avoid CI worker contention The flow integration tests spawn workers that compete for CPU with the existing relock_skip tests under --test-threads=10, causing consistent 60s timeouts in CI. Keep only the lightweight unit tests and DB integration tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: restore correct ee-repo-ref for our branch The ref was overwritten to main's EE ref during a rebase. Restore to our branch's EE commit that includes the autoscaling tuple fix. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: retrigger CI on fresh runner * fix: remove FK constraint from job_delete_schedule to unblock CI The FK with ON DELETE CASCADE to v2_job may have caused performance overhead during test DB setup (each sqlx::test creates a fresh DB with all migrations). Remove the FK — orphan schedule rows are harmlessly cleaned by the monitor. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ee-ref --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
6d58d1a74d |
fix: pipeline DISCARD ALL with first query on cached pg connections (#8707)
* perf: pipeline DISCARD ALL with first query on cached pg connections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * perf: use RESET ALL instead of DISCARD ALL for lighter session reset Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add integration test for pg session reset on cached connections Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: release MutexGuard before caching so pg connection cache actually works The old code shadowed the MutexGuard variable without dropping it, so try_lock() in the post-query caching path always failed — connection caching was effectively dead code. Restructure to explicitly drop the guard before connecting. Also adds a CACHE_HITS counter and clear_pg_cache() helper so the integration test can verify the cached-connection path is exercised. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add single-worker session isolation test for SET ROLE + search_path Pushes 3 jobs into the queue before starting the worker so a single worker processes them all sequentially (matching production). Verifies SET ROLE and SET search_path do not leak between jobs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add RESET ROLE to session reset (RESET ALL does not undo SET ROLE) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: use DISCARD ALL for full session reset and retry on stale connections - Switch from pipelined RESET ROLE; RESET ALL to eager DISCARD ALL when validating cached connections. This resets everything: role, GUCs, prepared statements, temp tables, advisory locks, LISTEN registrations. - DISCARD ALL also serves as a health check: if it fails, the stale connection is discarded and a fresh one is created transparently. - Extract new_pg_connection() helper to avoid duplicating the connect + spawn-connection-task logic. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add 100-job single-worker cache stress test Runs 100 varied PG jobs (plain SELECTs, SET ROLE, SET search_path, multi-statement) through one worker. Verifies all succeed, 99 hit the cache, and no session state leaks between jobs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
 hugocasa
|
f0437eba19 |
feat: add endpoint to restart workers in a worker group (#8659)
* feat: add endpoint to restart workers in a worker group Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: regenerate sqlx query cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add missing modules field to RawCode in tests and regenerate sqlx cache Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * update sqlx * fix: use require_devops_role for restart worker group endpoint Matches the permission level of the clean cache endpoint (update_config), allowing both superadmin and devops role users. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address review feedback for restart worker group - Fix OpenAPI description to say "devops role" instead of "superadmin" - Add dispatch('reload') after restart to refresh worker list Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: only dispatch reload on successful restart Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a46aa641f9 |
feat: add R language support (#8263)
* feat: add R language support Add R as a new supported scripting language in Windmill, following the same pattern used for Ruby. Includes: - Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r parser crate with tests, WASM parser binding, R executor with NSJail sandboxing, job dispatch and signature parsing - Frontend: language picker, R icon, syntax highlighting, editor bar insertions (Sys.getenv, get_variable, get_resource), schema inference, init code template, BETA badge - CLI: .r extension mapping, sync support, bootstrap template R scripts use `main <- function(...)` syntax, jsonlite for JSON serialization, and system curl for the Windmill client helper. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add R package resolution and installation Parse library()/require() calls from R scripts to extract dependencies. Resolve versions from CRAN, cache lockfiles in pip_resolution_cache, and install packages to a shared R library cache. The run step sets R_LIBS_USER so installed packages are available to the script. - Parser: parse_r_requirements() extracts package names from AST - Executor: resolve() generates lockfile, install() installs from CRAN - Worker lockfiles: wire up R resolve for dependency jobs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat: add nsjail sandboxing for R resolve and install phases Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: fix R arg inference with JS fallback parser and get_variable/get_resource Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix flake * nsjail * nits * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits - Suppress renv verbose output during resolve and install (controlled by #verbose annotation) - Filter renv from install list (already loaded, causes noisy restart message) - Log compact "resolved N packages" instead of full renv.lock JSON - Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe - Use disable_rl for nsjail install config (R compiles from source) - Reduce default concurrency from 20 to 5 - Add rlang to openflow.openapi.yaml - Fix MainArgSignature (no_main_func -> auto_kind) after main merge Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * final * fix: remove accidental R install from multiplayer Dockerfile Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: remove R from Windows build and DockerfileExtra Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: rename R migration to avoid timestamp collision with trigger_filter_logic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * all * fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add clear error when Rscript binary is missing Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: pyranota <pyra@duck.com> |
||
 Diego Imbert
|
5d1c54d9b3 |
feat: Debounce node (#8324)
* Debounce node works
* sqlx prepare
* sqlx prepare
* fix: address PR review issues for flow node debouncing
- Add sibling check in parent-walking loop to avoid killing branchall siblings
- Remove stale .sqlx cache files from earlier iterations
- Remove single-variant FlowNodeDebounceResult enum, use Result<()>
- Parse flow value once in version guard, recurse into nested modules
- Fix Svelte reactivity when switching selected flow modules
- Fix Tab indentation in FlowModuleComponent
- Use integer types in OpenAPI spec for debounce fields
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ee repo ref
* nit sqlx
* add Debouncing: None
* ee repo ref
* ee repo
* sqlx update
* fix: reject node-level debouncing inside branches (branchall/branchone)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* Revert "fix: reject node-level debouncing inside branches (branchall/branchone)"
This reverts commit
|
||
|
Ruben Fiszel
|
31d6660d56 |
feat: script module mode with CLI sync, preview, and WAC UI improvements (#8380)
* feat: add script module mode with folder model for Bun and Python Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add missing modules field to RawCode in bun_executor Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx * feat: enrich WAC templates with checkpoint and replay semantics Add prominent comments explaining that all computation must happen inside task/step/taskScript or it will be replayed on resume/retry. Clarify that waitForApproval does not hold a worker and that approve/reject URLs are available in the timeline step details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): script module sync idempotency, per-module hash tracking, and preview support - Fix pull→push idempotency: use `??` instead of `||` for module lock field so empty strings are preserved (matches API's `lock: ""`) - Add per-module hash tracking in wmill-lock.yaml following the flow inline script pattern (SCRIPT_TOP_HASH + per-module subpath hashes) - Selective module lock regeneration: only regenerate locks for modules whose content actually changed, not all modules - Use unfiltered rawWorkspaceDependencies for module hashes to match what updateModuleLocks passes to fetchScriptLock - Show changed module names in stale script output for clarity - Add module support to `script preview` command: read modules from __mod/ folder and pass them in the preview API request - Add preview tests for taskScript pattern (flat and folder layout) - Update test assertion for module stale detection output Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): WAC UI improvements — reorder templates, module tab rename, import consolidation - Reorder WAC template buttons: TypeScript before Python in ScriptBuilder, CreateActionsScript, and CreateActionsFlow - Remove dropdown items from +Script button (simplify to direct link) - Move "Import Workflow-as-Code" to +Flow dropdown with dedicated drawer - Add module tab rename: pencil icon on hover opens popover with validation, fixed-width icon container prevents layout shift Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat: remaining module-mode changes from working branch - Backend parser updates for WAC detection - CLI sync/types updates for raw app path and module support - Frontend UI polish (Dev.svelte, ScriptRow, script hash page) - Test fixture updates Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(cli): add test for module modification detection in generate-metadata Verifies that modifying a single module file re-triggers stale detection and only the changed module is listed, not all modules. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): critical fixes from PR review - Fix hardcoded dev path in bun_executor.rs WAC v2 wrapper — use "windmill-client" import instead of absolute filesystem path - Fix missed no_main_func → auto_kind rename in parser TS test - Add modules column to clone_script SQL (windmill-common and windmill-api-workspaces) so cloned scripts retain their modules - Add modules: None to RawCode structs in worker tests - Restore complete sqlx cache (merge main's cache + our new queries) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix clone warning treated as error in CI Change `.clone()` on double reference to `*k` dereference in scripts.rs hash implementation. Update sqlx cache with new query hashes from modified clone_script SQL. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use published parser wasm versions for CI build The local file:// paths for windmill-parser-wasm-py and windmill-parser-wasm-ts don't exist in the Cloudflare Pages build environment. Revert to published npm versions (1.655.0). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): update parser wasm packages to 1.657.2 Use newly published windmill-parser-wasm-ts and windmill-parser-wasm-py v1.657.2 which include auto_kind/WAC detection changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): regenerate package-lock.json for npm ci compatibility Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(frontend): use main's lockfile as base, update only parser wasm packages Regenerating package-lock.json from scratch pulled different dependency versions causing svelte-check type errors. Instead, start from main's lockfile and only update the two changed packages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): add modules column to fetch_script_for_update query The Script<SR> struct has a modules field (FromRow), but fetch_script_for_update didn't SELECT modules, causing a runtime error "no column found for name: modules" when the worker processed dependency jobs. This was the root cause of the relock_skip test timeout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix script module execution for Python and Bun - Fix modules not passed through job queue: inject _MODULES into PushArgs.extra when pushing Code jobs so worker can extract them - Fix Python module imports: use relative imports (from .helper) and add sys.path.insert for module directory in wrapper - Fix Python tests: use relative imports and empty lock to prevent pip from resolving module names as packages - Add local file check in Bun loader for module resolution - Ignore Bun module test (bundle mode loader integration tracked separately) - Add missing modules column to fetch_script_for_update query Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): remove unnecessary empty lock in Python module tests Relative imports (from .helper) are not parsed as pip packages, so the empty lock workaround is not needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(backend): fix module execution for Python and Bun — all tests pass Python modules: - Use relative imports (from .helper import greet) since scripts run as packages - Add sys.path.insert for module directory in wrapper to ensure local modules take precedence over pip packages with same name Bun modules: - Use bundled output (./out/main.js) as wrapper import when modules are present — the bundled output has module content inlined by Bun.build, avoiding runtime loader resolution issues - Add local file check in loader.bun.js onResolve to short-circuit API URL resolution for module files on disk Job queue: - Inject _MODULES into PushArgs.extra when pushing Code jobs so the worker can extract them at execution time Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * refactor: address PR review — simplify, fix correctness, remove dead code Critical fixes: - Replace all CLI `no_main_func` references with `auto_kind` (string) to match the backend migration and API changes - Remove duplicated `compute_python_module_dir` in worker.rs, use the canonical version from python_executor.rs High priority: - Auto-create `__init__.py` in intermediate directories for nested Python modules so imports like `from .utils.math import add` work without users manually creating __init__.py files - Remove redundant `sys_path_insert` — relative imports use Python's package system, not sys.path Medium: - Fix lock file base name extraction: use regex to strip only the final extension (`.replace(/\.[^.]+$/, '')`) instead of `indexOf(".")` which breaks for files like `helper.test.ts` Simplification: - Remove dead `{#if false}` Popover block in ScriptEditor.svelte - Guard loader.bun.js local file check to only run for relative paths (matching the Windows loader pattern) - Add clarifying comment on Bun dual mechanism (build + run phases) - Add maintenance comment on manual Hash impl for NewScript Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: final review fixes — stale cleanup, baseName, auto_kind export - Fix sync.ts baseName extraction using indexOf(".") → regex (same fix as script.ts/metadata.ts, missed this instance) - Add stale module file cleanup in writeModulesToDisk: removes files from __mod/ that are no longer in the modules map before writing, fixing the pull→push cycle that couldn't delete modules - Log warning when _MODULES serialization fails in job push instead of silently dropping modules - Use strict equality (===) for auto_kind comparison - Exclude auto_kind from workspace export — it is auto-detected by the parser at deploy time from script content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(cli): remove auto_kind from push, comparison, and metadata auto_kind is auto-detected by the parser at deploy time, so the CLI should not send it, compare it, or write it to script.yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: remove erroneously added backend/backend/.sqlx directory Duplicate .sqlx cache was committed at the wrong nested path. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address PR review feedback + fix CI dead_code warning Frontend (ScriptEditor.svelte): - Fix switchToMain() missing lastSyncedCode update — prevents stale code sync on external changes while editing a module tab - Fix formatAction saving module code to main script's localStorage draft — now saves main code when on a module tab - Fix non-null assertion on inferModuleLang in renameModule — fall back to original language instead of force unwrap - Remove redundant activeModuleTab truthy check in runTest CLI (script.ts): - Clean up empty directories after removing stale module files in writeModulesToDisk Backend: - Add path traversal guard in write_module_files — reject module paths containing ".." - Fix dead_code warning on auto_kind field in workspace export struct Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(frontend): improve auto_kind UX + address review findings - Rename "Include without main function" toggle to "Include library scripts" in script list (ItemsList.svelte) - Update NoMainFuncBadge: "No main" → "Library" with clearer tooltip - Filter module file extensions by main script language — Python scripts only allow .py modules, TypeScript only .ts, etc. - Split flushModuleState into flushModuleContent (no UI side-effect) and flushModuleState (flush + reset tab), reducing duplication - Dynamic placeholder and hint text in add module popover based on main script language Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5296adeddf |
test: add powershell module detection and execution tests (#8373)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
41e523f827 |
fix: parallel branchall hang on bad stop_after_all_iters_if + results.x.length null (#8276)
Two fixes: 1. When a parallel branchall/forloop has a `stop_after_all_iters_if` expression that fails (e.g. bad JS syntax), the error was propagated with `?`, causing the transaction to roll back the parallel index increment. Since all parallel jobs were already completed, nothing could ever increment the index again and the flow hung forever. Now the error is caught and converted to a stop-early failure so the transaction commits and the flow fails gracefully. 2. Expressions like `results.a.length` in step input transforms resolved to null because the `handle_full_regex` fast path intercepted them and used PostgreSQL's `#>` JSON path operator, which can't resolve JS runtime properties like `.length` on arrays. Now the fast path skips expressions ending with JS-only properties (like `length`), falling through to full QuickJS evaluation where they work correctly. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
4fedfdfd11 |
feat(cli): add consistent get/list/new subcommands for all item types (#8047)
* feat(cli): add consistent get/list/new subcommands for all item types Make the CLI consistent so every item type (script, flow, app, resource, resource-type, variable, schedule, folder, trigger) supports get/list/new subcommands, enabling the CLI to be used as a full API client in bash scripts with jq piping. - Add --json flag to all list commands for machine-readable output - Register explicit "list" subcommand alongside default action - Add "get <path> [--json]" subcommand to fetch single items from API - Rename "bootstrap" to "new" for script/flow, keep "bootstrap" as alias - Add "new" subcommand for resource, resource-type, variable, schedule, folder, and trigger to create local template YAML files - Update cli-commands skill documentation for wmill init - Add integration tests for all new commands Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * feat: install wmill CLI in Docker images and use it for bash variable/resource access - Install windmill-cli via bun in all Dockerfiles that include bun - DockerfileCli: switch from node:slim to oven/bun:slim - CLI: auto-configure from WM_WORKSPACE/WM_TOKEN/BASE_INTERNAL_URL env vars as last-resort fallback when no workspace is configured - Frontend: replace curl-based bash snippets with wmill variable/resource get - Add backend integration tests for wmill CLI in bash scripts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): install windmill-cli in backend test workflow Ensures wmill is available on PATH for bash integration tests that use `wmill variable get` and `wmill resource get`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(cli): replace @std/* Deno dependencies with Node.js equivalents Replace @std/log with a lightweight custom logger (core/log.ts), @std/path with node:path, and @std/yaml with the yaml npm package. Also fix process hang on exit, add --node option to install_dev.sh, and add missing hasRequiredPermissions to NpmProvider. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * refactor(cli): replace @ayonli/jsext and @std/encoding with lightweight alternatives Replace @ayonli/jsext (8.4MB) with tar-stream (32kB) for tar creation, replace @std/encoding with Node.js Buffer.toString("hex"), and fix @windmill-labs/shared-utils to use direct npm instead of JSR mirror. Also resolve merge conflicts in sync.ts and fix pre-existing type errors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(cli): use singleQuote YAML output and pass yamlOptions in gitsync pull The yaml library defaults to double quotes, but the codebase (and tests) expect single-quoted strings. Add singleQuote: true to yamlOptions and pass yamlOptions to gitsync-settings pull writeFile calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * fix(cli): address code review feedback - Install CLI from source in backend tests instead of npm - Fix script bootstrap catch block to re-throw "File already exists" - Add type-safe local variable after trigger kind validation - Use created_by instead of policy.on_behalf_of for app get output - Note --kind is recommended for faster trigger lookup in help text - Document node symlink purpose in Dockerfiles Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): use /usr/bin for wmill wrapper to ensure it's in PATH Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(ci): install wmill to ~/.local/bin to avoid permission issues Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci(backend): switch to Blacksmith runner and add cargo caching - Switch from ubicloud-standard-16 to blacksmith-16vcpu-ubuntu-2404 for faster NVMe-backed builds - Add stickydisk for cargo target directory (persistent NVMe cache across runs) - Add cache for cargo registry and git dependencies - Upgrade DuckDB FFI cache from actions/cache@v3 to useblacksmith/cache@v1 - Enable CARGO_INCREMENTAL=1 to benefit from persistent target cache Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix ci --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
9eb15312f6 |
feat: add .npmrc support for private npm registries (#8039)
* feat: add .npmrc support for private npm registries Add a new `npmrc` instance setting that accepts full .npmrc file content for configuring private npm registries. Works with bun (native .npmrc support since 1.1.18), deno (native .npmrc support in 2.x), and the npm proxy (parses default registry + auth token from .npmrc). Legacy `npm_config_registry` and `bunfig_install_scopes` fields are now hidden when empty, so new users only see the .npmrc field. Also fixes a pre-existing race condition where gen_bunfig was called after start_child_process. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
ea52a8b8ce |
refactor: move integration tests to subcrates to reduce recompilation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
4343b73485 | fix: reduce DB pool contention by eliminating dual-connection patterns (#7861) | ||
|
Ruben Fiszel
|
ed4ea41c9c | test nit | ||
|
Ruben Fiszel
|
c3812e41e1 |
ci: limit test threads to prevent pool exhaustion (#7852)
* ci: limit test threads to 16 to prevent postgres pool exhaustion Running all tests with unlimited parallelism exhausts postgres max_connections (default 100), causing sqlx::test databases to fail setup and producing spurious RowNotFound errors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: CI ruby env vars and mysql test binary coercion - Ruby executor reads RUBY_BUNDLE_PATH and RUBY_GEM_PATH but CI was setting BUNDLE_PATH and GEM_PATH, causing "Executable bundle not found on worker" errors. - MySQL test CAST(CONCAT(...) AS CHAR) returns binary type when param is bound as bytes. Use CONVERT(? USING utf8mb4) to ensure character result type. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
477832dbee |
fix: adapt mysql and ruby test assertions
MySQL CONCAT with binary params returns VARBINARY, causing base64 encoding. Use CAST(... AS CHAR) to force character type output. Ruby executor doesn't support keyword parameters (name:), use positional parameters instead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f22b62d838 |
test: add integration tests for all feasible language runtimes in CI (#7849)
* test: add integration tests for all feasible language runtimes in CI Add integration tests and CI setup for languages that were previously untested. Each test runs a simple "hello world" job through the full worker pipeline to verify end-to-end execution. New language tests added to worker.rs: - Nativets (4 tests): basic string, numeric args, object return, datetime - Bunnative: TypeScript execution via Bun native runtime - CSharp: .NET compilation and execution (feature-gated) - PHP: PHP script execution (feature-gated) - Ruby: Ruby script execution (feature-gated) - MySQL: SQL query via async MySQL client (feature-gated) - PowerShell: pwsh script execution - PostgreSQL: SQL query against test database CI changes (backend-test.yml): - Add MySQL 8.0 service container - Add setup-php (8.3 + composer), setup-ruby (3.3), pwsh install - Enable feature flags: csharp, php, ruby, mysql - Pass language binary paths: PHP_PATH, COMPOSER_PATH, RUBY_PATH, BUNDLE_PATH, GEM_PATH, POWERSHELL_PATH, DOTNET_PATH - Uncomment and modernize CSharp test (was commented out) - Increase test timeout 16m -> 20m for additional runtimes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * sqlx --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
2eafe6df36 |
test: add comprehensive test coverage for extracted backend crates
Add 296 tests across unit and integration test suites to cover the newly extracted crates from the recent refactor commits. Unit tests (270): - windmill-trigger-postgres (96): hex codec, bool parsing, type conversion, relation tracking, replication message parsing, publication data validation - windmill-trigger-http (92): HMAC signature verification for GitHub/Slack/Stripe/TikTok/Twitch/Zoom webhooks, API key auth, Basic Auth, route validation, HTTP method/request type serde - windmill-api-jobs (39): SQL query builder for job listing/counting with filters, pagination, label handling - windmill-trigger (31): TriggerMode serde, query pagination, BaseTriggerData backward compat, HandlerAction, ServerState - windmill-common webhook (7): WebhookMessage serialization tags - worker nativets/postgresql (5): nativets job execution with args/objects/datetime, postgresql query execution Integration tests (26): - backend/tests/triggers.rs: capture config CRUD, capture payload operations, capture API endpoints, HTTP trigger CRUD with mode filtering, all trigger types DB schema validation (websocket, kafka, postgres, nats, sqs), schedule operations Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
b6aa05d39f | fix tests | ||
 claude[bot]
|
bfe35a8421 | fix: check tag availability for flow substeps (#7468) | ||
|
hugocasa
|
a6993823af |
feat: restart flow from step with different flow version (#7409)
* feat: restart flow from step with different flow version * fix tests * fix tests |
||
 Pyra
|
fe56191422 |
feat(internal): runnable settings (#7298)
* rework everything again Signed-off-by: pyranota <pyra@duck.com> * updcate sqlx Signed-off-by: pyranota <pyra@duck.com> * update ref Signed-off-by: pyranota <pyra@duck.com> * fix things Signed-off-by: pyranota <pyra@duck.com> * fix function Signed-off-by: pyranota <pyra@duck.com> * final fixes Signed-off-by: pyranota <pyra@duck.com> * update sqlx Signed-off-by: pyranota <pyra@duck.com> * fix script creation Signed-off-by: pyranota <pyra@duck.com> * address todo Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * fixups Signed-off-by: pyranota <pyra@duck.com> * fix cargo.toml Signed-off-by: pyranota <pyra@duck.com> * update ee repo Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> * ref Signed-off-by: pyranota <pyra@duck.com> * fix Signed-off-by: pyranota <pyra@duck.com> * ee repo Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * ee ref Signed-off-by: pyranota <pyra@duck.com> * remove dbg Signed-off-by: pyranota <pyra@duck.com> * sqlx Signed-off-by: pyranota <pyra@duck.com> * chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395 This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private. Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395 Automated by sync-ee-ref workflow. * ci: force runnable settings Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Diego Imbert
|
9bbab3321e |
feat: Data tables (#7226)
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit |
||
|
Diego Imbert
|
7b1a8b0dce | fix: improve S3 etag caching (#7301) | ||
|
Pyra
|
1dfb0f0078 |
refactor: move concurrency and debouncing settings to their own structs (#7286)
* move concurrency and debouncing settings to it's own structs Signed-off-by: pyranota <pyra@duck.com> * cleanup queue::jobs::pull Signed-off-by: pyranota <pyra@duck.com> * done Signed-off-by: pyranota <pyra@duck.com> * migrated the rest Signed-off-by: pyranota <pyra@duck.com> * implement todo! * impmenet todo! v2 * more cleanup * fix all Signed-off-by: pyranota <pyra@duck.com> * remove unrelated code Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * do not serialize rawcode Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * update ee repo ref Signed-off-by: pyranota <pyra@duck.com> * update ee repo Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * Update ee-repo-ref.txt --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
93c34c6d76 |
feat: flow for loop squashing (#7107)
* feat: flow for loop optimization * fix: make dedicated flows work with flow nodes * fix: transform dedicated inputs + better error handling + differentiate squash from same worker * fix builds * fix build * fix build * make dedicated worker private * update ee ref * frontend nits * nit * add trace to dedicated worker * update ee ref * Update ee-repo-ref.txt --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Diego Imbert
|
fa1bc3c711 |
DuckDB test to ensure FFI doesn't crash on simple query (#7147)
* test_duckdb_ffi * build dev duckdb lib * cache |
||
 dieriba
|
09cdfb4556 |
feat: webhook by flow version (#7062)
* ok * done * update sqlx and repo ref * fix |
||
 Ramtin Mesgari
|
5da494b197 |
refactor: remove legacy database views v2_as_queue and v2_as_completed_job (#6689)
* refactor: remove legacy database views v2_as_queue and v2_as_completed_job Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com> * fix tests * fix jobs.rs * end * fix * improvement * improvement --------- Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
5b1ab610b7 |
improve stack size usage (#6936)
* oom * final * final |
||
|
Ruben Fiszel
|
04890a7a42 | tmp overflow fix | ||
|
Pyra
|
edece035f8 |
feat: job debouncing (#6878)
* feat(EE): job debouncing Signed-off-by: pyranota <pyra@duck.com> * remove 'script' file Signed-off-by: pyranota <pyra@duck.com> * more work Signed-off-by: pyranota <pyra@duck.com> * properly gate it behind enterprise Signed-off-by: pyranota <pyra@duck.com> * update ee repo ref Signed-off-by: pyranota <pyra@duck.com> * change ee repo ref again Signed-off-by: pyranota <pyra@duck.com> * remove unused variable Signed-off-by: pyranota <pyra@duck.com> * feat(EE): implement TODOs and enhance tracing for job debouncing - Add database index on script(workspace_id, debounce_key) for efficient lookups - Update minimum version requirement to 1.564.0 throughout codebase - Add tracing warnings when debouncing is disabled due to worker version mismatch - Fix all documentation links from TODO placeholders to proper URLs - Replace Gauge icon with Timer icon for debouncing UI elements - Update placeholder text and tooltips with clear descriptions Co-authored-by: Pyra <pyranota@users.noreply.github.com> * create -> crate Signed-off-by: pyranota <pyra@duck.com> * remove index Signed-off-by: pyranota <pyra@duck.com> * some updates Signed-off-by: pyranota <pyra@duck.com> * fix once more Signed-off-by: pyranota <pyra@duck.com> * fix it once more Signed-off-by: pyranota <pyra@duck.com> * Remove flow step debouncing, keep top-level flow debouncing - Remove debounce fields from RawScript and FlowScript FlowModuleValue variants - Remove debounce fields from JobPayload::FlowScript and RawCode - Update raw_script_to_payload function signature - Remove debouncing UI from flow step runtime settings - Remove debouncing toggle handler and indicator badge - Preserve top-level flow debouncing in FlowSettings Co-authored-by: Pyra <pyranota@users.noreply.github.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fixup claude's work Signed-off-by: pyranota <pyra@duck.com> * cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments - Removed all dbg! macro calls from production code - Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0 - Added comprehensive documentation comments explaining: - Debouncing feature purpose and mechanics - Database schema for debounce_key and debounce_stale_data tables - Version check logic and guard functions - Improved code clarity and maintainability Co-authored-by: Pyra <pyranota@users.noreply.github.com> * improve fallback Signed-off-by: pyranota <pyra@duck.com> * remove comments from old migration Signed-off-by: pyranota <pyra@duck.com> * fix pull Signed-off-by: pyranota <pyra@duck.com> * fix once more Signed-off-by: pyranota <pyra@duck.com> * Update frontend/src/lib/components/ScriptBuilder.svelte Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> * add sqlx cache Signed-off-by: pyranota <pyra@duck.com> * fix: frontend CI - fix debounce field names and remove leftover flow step debouncing - Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key - Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml - Regenerate frontend types from OpenAPI spec - Remove leftover flow step debouncing code from FlowModuleComponent.svelte - Remove debounce fields from RawScript in openflow.openapi.yaml - Remove unused Timer import from FlowModuleHeader.svelte All frontend checks now passing (0 errors, 0 warnings) Co-authored-by: Pyra <pyranota@users.noreply.github.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * remove unused import Signed-off-by: pyranota <pyra@duck.com> * fix ci again Signed-off-by: pyranota <pyra@duck.com> * udpate ee repo ref Signed-off-by: pyranota <pyra@duck.com> * CI doesn't want to be fixed but I still try Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * ci... Signed-off-by: pyranota <pyra@duck.com> * Update ee-repo-ref.txt * safer migration Signed-off-by: pyranota <pyra@duck.com> * reduce noise in logs Signed-off-by: pyranota <pyra@duck.com> * fix cli for scripts Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Pyra <pyranota@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Pyra
|
defb6c9694 |
feat: dependency job debouncing (#6769)
* v0 Signed-off-by: pyranota <pyra@duck.com> * optimize relocks * make it work with relative relative imports Signed-off-by: pyranota <pyra@duck.com> * use fallback Signed-off-by: pyranota <pyra@duck.com> * remove dbg and todos Signed-off-by: pyranota <pyra@duck.com> * future proof a bit Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * more cleanup Signed-off-by: pyranota <pyra@duck.com> * remove final TODO Signed-off-by: pyranota <pyra@duck.com> * do not use bytemuck Signed-off-by: pyranota <pyra@duck.com> * optimize hashing Signed-off-by: pyranota <pyra@duck.com> * implementation 1 Signed-off-by: pyranota <pyra@duck.com> * almost v0 Signed-off-by: pyranota <pyra@duck.com> * v0 Signed-off-by: pyranota <pyra@duck.com> * add comments and use fallback Signed-off-by: pyranota <pyra@duck.com> * call dissolve for apps Signed-off-by: pyranota <pyra@duck.com> * add comms Signed-off-by: pyranota <pyra@duck.com> * refactor v0 (partially tested + dirty) Signed-off-by: pyranota <pyra@duck.com> * finishing Signed-off-by: pyranota <pyra@duck.com> * remove TODO Signed-off-by: pyranota <pyra@duck.com> * Update SQLx metadata * silence unused argument Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * implement rebuild_map endpoint Signed-off-by: pyranota <pyra@duck.com> * update windmill api client Signed-off-by: pyranota <pyra@duck.com> * almost finish with tests Signed-off-by: pyranota <pyra@duck.com> * add proper testing Signed-off-by: pyranota <pyra@duck.com> * remove unused fixtures Signed-off-by: pyranota <pyra@duck.com> * Update SQLx metadata * partial cleanup Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/scoped_dependency_map.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-common/src/scripts.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * evil doings Signed-off-by: pyranota <pyra@duck.com> * more cleanup * Update SQLx metadata * more cleanup Signed-off-by: pyranota <pyra@duck.com> * fixing CI Signed-off-by: pyranota <pyra@duck.com> * remove python from default features Signed-off-by: pyranota <pyra@duck.com> * feat: dependency job debouncing * checkpoint Signed-off-by: pyranota <pyra@duck.com> * more improvements Signed-off-by: pyranota <pyra@duck.com> * refactor: clean up dependency job debouncing implementation - Add comprehensive comments explaining the debouncing mechanism - Replace debug statements (dbg!) with proper tracing calls - Extract helper functions to reduce code duplication: - extract_to_relock_from_args() for extracting nodes/components - accumulate_debounce_stale_data() for updating stale data - Improve code readability and maintainability Co-authored-by: Pyra <pyranota@users.noreply.github.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * Update SQLx metadata * Update backend/windmill-common/src/jobs.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-common/src/jobs.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-queue/src/jobs.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * better error handling for helper Signed-off-by: pyranota <pyra@duck.com> * update ee-ref Signed-off-by: pyranota <pyra@duck.com> * test: add race condition test for dependency job debouncing - Implement test_2 for scripts to test the race condition edge case - Add comprehensive documentation comments to all test functions - Remove empty test_2 stubs for flows (not needed) - Keep test_2 stub for apps with TODO comment The race condition test simulates the scenario where a job is marked as running but debounce_key hasn't been cleaned up yet, forcing the system to create a new job while reusing the existing debounce_key. This edge case can occur due to the lack of transactions in the pull function for performance reasons (see jobs.rs:4415-4425). Co-authored-by: Pyra <pyranota@users.noreply.github.com> * test: implement race condition test for dependency job debouncing - Add comprehensive test_2 for script module that tests the race condition edge case - Remove empty test_2 stubs from flows and apps modules - Fix unused variable warning in worker_lockfiles.rs - Add detailed comments explaining the race condition scenario and test logic Co-authored-by: Pyra <pyranota@users.noreply.github.com> * implement fallback Signed-off-by: pyranota <pyra@duck.com> * make it mostly work * all tests are almost working Signed-off-by: pyranota <pyra@duck.com> * add comments a bit Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-common/src/scripts.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/worker_lockfiles.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/worker_lockfiles.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * feat: improve debouncing documentation, tracing, and error handling - Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull - Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.) - Add proper error handling with .map_err() and contextual messages - Replace dbg!() with proper tracing::debug!() - Replace todo!() with proper error handling - Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because' - Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o' - Add documentation for debounce cleanup and stale data retrieval - Add trace-level logging for non-error paths to reduce noise Co-authored-by: Pyra <pyranota@users.noreply.github.com> * do some work for future improvements Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * clippy Signed-off-by: pyranota <pyra@duck.com> * update sqlx Signed-off-by: pyranota <pyra@duck.com> * clippy Signed-off-by: pyranota <pyra@duck.com> * update ee ref Signed-off-by: pyranota <pyra@duck.com> * flag tests behind the feature, add timeout Signed-off-by: pyranota <pyra@duck.com> * fix timeout + cleanup Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * row lock debounce_key Signed-off-by: pyranota <pyra@duck.com> * addressing TODOs Signed-off-by: pyranota <pyra@duck.com> * fix test Signed-off-by: pyranota <pyra@duck.com> * ee ref Signed-off-by: pyranota <pyra@duck.com> * ee repo Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com> Co-authored-by: GitHub Action <action@github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Pyra <pyranota@users.noreply.github.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
 Alexander Petric
|
ae8d37fc34 |
feat: add dynamic skip for schedules (#6739)
* feat: add dynamic skip for schedules * npm check * accidental rename * fixing tests * Update SQLx metadata * simplify api / queries --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
2047073c03 | improve tests | ||
|
Pyra
|
ed806bf9d0 |
fix(backend): rework dependency_map handling (#6598)
* v0 Signed-off-by: pyranota <pyra@duck.com> * optimize relocks * make it work with relative relative imports Signed-off-by: pyranota <pyra@duck.com> * use fallback Signed-off-by: pyranota <pyra@duck.com> * remove dbg and todos Signed-off-by: pyranota <pyra@duck.com> * future proof a bit Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * more cleanup Signed-off-by: pyranota <pyra@duck.com> * remove final TODO Signed-off-by: pyranota <pyra@duck.com> * do not use bytemuck Signed-off-by: pyranota <pyra@duck.com> * optimize hashing Signed-off-by: pyranota <pyra@duck.com> * implementation 1 Signed-off-by: pyranota <pyra@duck.com> * almost v0 Signed-off-by: pyranota <pyra@duck.com> * v0 Signed-off-by: pyranota <pyra@duck.com> * add comments and use fallback Signed-off-by: pyranota <pyra@duck.com> * call dissolve for apps Signed-off-by: pyranota <pyra@duck.com> * add comms Signed-off-by: pyranota <pyra@duck.com> * refactor v0 (partially tested + dirty) Signed-off-by: pyranota <pyra@duck.com> * finishing Signed-off-by: pyranota <pyra@duck.com> * remove TODO Signed-off-by: pyranota <pyra@duck.com> * Update SQLx metadata * silence unused argument Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * implement rebuild_map endpoint Signed-off-by: pyranota <pyra@duck.com> * update windmill api client Signed-off-by: pyranota <pyra@duck.com> * almost finish with tests Signed-off-by: pyranota <pyra@duck.com> * add proper testing Signed-off-by: pyranota <pyra@duck.com> * remove unused fixtures Signed-off-by: pyranota <pyra@duck.com> * Update SQLx metadata * partial cleanup Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/scoped_dependency_map.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-common/src/scripts.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * evil doings Signed-off-by: pyranota <pyra@duck.com> * more cleanup * Update SQLx metadata * more cleanup Signed-off-by: pyranota <pyra@duck.com> * fixing CI Signed-off-by: pyranota <pyra@duck.com> * remove python from default features Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com> Co-authored-by: GitHub Action <action@github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
cf70265ee9 |
rust tests cleanup (#6573)
* test improvements * improve tests * improve tests |
||
|
hugocasa
|
b69478cf2b |
feat: better early stop in flows (#6534)
* feat: better early stop * style * nits * nits * fix test |
||
|
Ruben Fiszel
|
d3f6a0da7e | test logs | ||
|
hugocasa
|
fbd942f179 |
fix(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs (#6490)
* feat(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs * nit * nits * set root_job only if additional information meaning if not equal to innermost or parent * nit * feat: add root job env var * nits * fix build * nit * fix build * nit * sqlx |
||
|
pyranota
|
006f32602c |
fix: fix relative imports cache invalidation (#6468)
* fix: new script on deps job for secondary scripts Signed-off-by: pyranota <pyra@duck.com> * make it work (dirty) Signed-off-by: pyranota <pyra@duck.com> * reduce db calls * remove `triggered_by_relative_import` Signed-off-by: pyranota <pyra@duck.com> * add comment to common_dependency_path Signed-off-by: pyranota <pyra@duck.com> * add fallback to old behavior Signed-off-by: pyranota <pyra@duck.com> * remove TODOs Signed-off-by: pyranota <pyra@duck.com> * pass deployed hash to git sync handler function Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> |
||
|
hugocasa
|
958e8af782 |
feat: ai agent steps (#6393)
* feat: ai agent steps base * better backend and graph * feat: anthropic, log viewer * nit * fix(frontend): hide tool nodes from timeline * move ai agent actions from flow status to flow status module * nits and workspace/hub scripts support * tmp ref * fix merge * feat: display agent tools status in the graph * fix reactivity * fix flow status * nit |
||
|
pyranota
|
11dd4118ce |
feat: add ruby support (#5939)
* implement single line pin * make panic-safe * use pin even if multiple modules imported withing single statement * add repins and make imports respect pins * keep all pins * Allow multiple pins * add comments + handle stuff more safely * fix fully qualified imports * remove ignore * sort nested * apply unique to output requirements list * fix typo * remove mut * update sqlx * sort imports * sort imports * fix formatter and format * feat: add ruby support * fix * update sqlx * add ruby to flake.nix Signed-off-by: pyranota <pyra@duck.com> * upgrade par_install_language_dependencies helper rename fields + allow passing custom payload Signed-off-by: pyranota <pyra@duck.com> * change migrations Signed-off-by: pyranota <pyra@duck.com> * make full pipeline work! Signed-off-by: pyranota <pyra@duck.com> * do a bit fixing * stage Signed-off-by: pyranota <pyra@duck.com> * add ruby in dockerfiles Signed-off-by: pyranota <pyra@duck.com> * implement auto-require + 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * create 'windmill/inline' only if does not exist Signed-off-by: pyranota <pyra@duck.com> * add nsjail Signed-off-by: pyranota <pyra@duck.com> * update global settings Signed-off-by: pyranota <pyra@duck.com> * add ruby icon * generate lockfile on deploy (but broken) Signed-off-by: pyranota <pyra@duck.com> * fix(frontend): display deployed script lockfile and other info Signed-off-by: pyranota <pyra@duck.com> * feat: add mini-wmill client Methods: - get_resource - get_variable Additionally add shortcuts on frontend Signed-off-by: pyranota <pyra@duck.com> * add nit warning Signed-off-by: pyranota <pyra@duck.com> * support private repos credentials Signed-off-by: pyranota <pyra@duck.com> * fix compilation error Signed-off-by: pyranota <pyra@duck.com> * found rustc bug + refactor universal installer Signed-off-by: pyranota <pyra@duck.com> * more refactor + fixes Signed-off-by: pyranota <pyra@duck.com> * bug fixing + polishing Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * change 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * delegate env_clear to universal installer Signed-off-by: pyranota <pyra@duck.com> * update init script Signed-off-by: pyranota <pyra@duck.com> * add cli support Signed-off-by: pyranota <pyra@duck.com> * unhardcode frontend Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * fix init script Signed-off-by: pyranota <pyra@duck.com> * make it work on ms windows Signed-off-by: pyranota <pyra@duck.com> * do not clean env in the end Signed-off-by: pyranota <pyra@duck.com> * fix windows (again) Signed-off-by: pyranota <pyra@duck.com> * include PROXY_ENVS (untested) Signed-off-by: pyranota <pyra@duck.com> * feat: self-signed certs + proxy support (tested) Signed-off-by: pyranota <pyra@duck.com> * flake: pin ruby to 3.4 * merge follow-ups Signed-off-by: pyranota <pyra@duck.com> * remove obsolete libc crate Signed-off-by: pyranota <pyra@duck.com> * cleanup parser Signed-off-by: pyranota <pyra@duck.com> * fix java ci Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix nits Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/universal_pkg_installer.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/ruby_executor.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * add nsjail for lock stage Signed-off-by: pyranota <pyra@duck.com> * fix merge Signed-off-by: pyranota <pyra@duck.com> * Update common.rs * regen windmill-api-client Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * fix tests again Signed-off-by: pyranota <pyra@duck.com> * reference published ruby parser Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
claude[bot]
|
161dbbc6d6 |
feat: add native result streaming (#6242)
* feat: add stream output feature to SSE job updates Adds stream_output field to JobUpdate struct that extracts log lines starting with '[wm_stream]:' from job logs. Regular logs now exclude stream lines, which are captured separately for specialized handling. - Added stream_output: Option<String> field to JobUpdate struct - Created extract_stream_output_from_logs() function to filter stream lines - Modified get_job_update_data() to use stream extraction logic - SSE clients now receive both new_logs and stream_output in job updates Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * feat: rename stream_output to stream and handle newlines as \n - Renamed `stream_output` field to `stream` in JobUpdate struct - Updated extract_stream_output_from_logs to extract_stream_from_logs - Changed stream output to join with literal \n instead of actual newlines - Stream lines are properly excluded from regular new_logs 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * decision tree nits * push ee ref * push ee ref * fix: fix id renaming in apps * remove duplicate caching (#6285) * feat: migrate audit log ids to bigints (blocking migration for EE) * fix(mcp): add proper check for mcp routes (#6282) * add proper check for mcp routes * cleaner * apply to flow * fix add checks scopes --------- Co-authored-by: dieriba <dieriba.pro@gmail.com> * chore(main): release 1.514.0 (#6283) * chore(main): release 1.514.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix: pin tokio to 1.46.1 and aws-sdks-ts * pin rustls to 0.23.29 + pin aws-sdk * chore(main): release 1.514.1 (#6288) * chore(main): release 1.514.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix: improve docker logs collection in docker mode * support $res: string in form inputs of arrays * fix import nit * fix: fix DynSelect * nits * fix: resource-type-ts-parser (#6289) * fix: resource types as arg in typescript handle imported defined types * Update nix flake (#6291) * merge * Small UI fixes (#6294) * fix step history not refreshing with staticInputs * fix array of obj not showing up in json editor in test this step * datatable scales correctly in DisplayResult and scrolling is much more usable * avoid next button disapearing and changing layout / hurting ux * nits * fix bug when renaming module A to B then module C to A, C takes the schema of A * fix bug with comments in sql repl * fix aggrid theme randomly not loading * bindable script * better delete button in db manager * property select doesnt exist * fix all warnings * delete $flowStateStore[id] on delete * feat(cli): generate cursor rules on init (#6270) * create cursor rules on init * change gen * add missing resource-type command * add resource type command in guidance * add schema option * revert * nit * nit * add flow guidance * nit * chore(main): release 1.515.0 (#6292) * chore(main): release 1.515.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix: improved logs for script * nits logs * chore(main): release 1.515.1 (#6295) * chore(main): release 1.515.1 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * merge * even more indexer tracings * add more tracing logs * feat: prevent too large results (>500Mb) from OOMing database * nit naming * feat: add CA certificate update at startup via environment variable (#6280) * feat: add CA certificate update at startup via environment variable Add support for running 'update-ca-certificates' at binary startup when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true". - Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup - Execute update-ca-certificates command if env var is set to "true" - Log success/failure appropriately with tracing - Continue startup even if CA certificate update fails - Non-blocking implementation with proper error handling Fixes #6279 Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * refactor: extract CA certificate update logic into separate function Extract the CA certificate update logic from windmill_main() into a dedicated update_ca_certificates_if_requested() function for better code organization and maintainability. Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * improvements --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com> Co-authored-by: Alexander Petric <alex@windmill.dev> * fix: indexer collection of job logs before indexing (#6300) * Add flume as dependecy for indexer * Update ee-repo-ref * Remove flags from cargo.toml * Update ee-repo-ref * Update ee-repo-ref * fix rust sdk build error (#6305) Signed-off-by: pyranota <pyra@duck.com> * fix broken audit logs filter (#6304) * rename to from to * goto fix * default to false if field not present operator settings (#6301) * git sync UI improvements (#6303) * ui improvements round 1 * modal cleanup * init * UI refactor * UI cleanup + refactor * legacy cleanup * success model -> github actions, non-ee warnings * sqlx * npm check * ee warning everywhere * last comments * formatting * no hardcoded theme * claude review improvemenets * fix: no process relative imports for scripts with codebase * fix: sqs oidc authentication disconnect #6307 * handle metadata for new scripts happen after commit * handle_deployment_metadata in a task * nits * chore: add windmill-utils-internal package (#6299) * add utils package * naming * cleaning * add docs * remove log * use autogenerated types * remove old * fix * cleaning * add docs * chore(main): release 1.516.0 (#6298) * chore(main): release 1.516.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * merge * indexer improvements * upgrade tantivy to 0.24.2 * use tantivy fork * nit warnings * fix oss build * improve indexer * chore: use windmill-utils-internal for cli (#6297) * add utils package * naming * cleaning * simplify assignPath * rename old files * same for locks * create on confirm * default true * use replaceinlinescripts from utils * use extractscriptfromflows * make it compile * cleaning * use argsigtojson * fix * fix missing await * cleaner * cleaning * cleaning * use in frontend * add docs * testing * remove log * use autogenerated types * remove old * fix * cleaning * adapt usage * draft * better build script * fix build * revert to default creation * add docs * remove and rename * make everything work * add await * only if not installed * add vs code setting * add to publish action * fix bc * safer use of sep * fix * do not rename on push * no publish on release * use published package on frontend * nit * Add dependencies to run sqlx prepare to nix flake (#6309) * feat(cli): wmill-lock.yaml v2 for easier git merge diffs * merge * merge * all * all * rm warnings * fix styling on aichatinput (#6312) * fix: use with_capacity back presusre for tantivy directory multipart writes (#6313) * use with capacity for tantivy directory multi part uploads * Update ee repo ref * Update ee-repo-ref * Update ee-repo-ref * chore(main): release 1.517.0 (#6310) * chore(main): release 1.517.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> * fix typo on cli build (#6314) * cleanup * feat(utils): add flow.yaml validation function (#6316) * add validateflow function * cleaner code * preprocess json * cleaning * create specific package * cleaning * add tests * fix: cleanup concurrency_counter automatically + remove orphans keys automatically * fix: add disabled support to resource picker in schema forms * fix: add wm_labels to tracing spans * all * merge * all * fix: delete empty git connection (#6318) * fix checks * bun handling * all * all? * all * all * update * all * update * check * fix history * all * all * all * Remove leftover debug tracing statements - Remove commented debug trace in jobs.rs for stream output - Remove commented debug trace in result_stream.rs for stream processing Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * fix test * all * handle iter * fix --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> Co-authored-by: centdix <40307056+centdix@users.noreply.github.com> Co-authored-by: dieriba <dieriba.pro@gmail.com> Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com> Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com> Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com> Co-authored-by: Alexander Petric <alex@windmill.dev> Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com> |
||
|
Ruben Fiszel
|
564e92e98d | cleanup warnings | ||
|
Ruben Fiszel
|
323ec9a994 | remove warnings | ||
|
Ruben Fiszel
|
f4073abef0 | test fix | ||
|
Diego Imbert
|
2ab5345e61 |
Assets refactor (#6217)
* Moved logic to FlowAssetsProvider * Remove assetsMap in flow * do not parse everything on mount + only check for missing assets fields * add assets field in backend * remove fallbackAccessTypes * better structure and less queries / parsing * Fix assets not showing when pulling raw_flow from jobs * flow assets ctx for job run * Fix transitive assets fetching * Fix input args asset node * enablePathScriptAndFlowAssets flag * edit btn for variable * untrack refresh * move parseInputArgsAssets * Assets tab in runs * Update FlowStatusViewerInner to svelte 5 + fix asset sync bug * avoid toast error on bad resource * fetch res metadata for input arg asset * Job assets viewer in run page * r/w selector * remove indigo badge * store alt_access_type state in ScriptEditor * Don't parse assets in flow script editor * Add alt_access_type in backend * show Read as selected by default to avoid giving the feeling of having made a decision * keep alt_access_type when reparsing in flow raw scripts * Remove variable asset kind, and save assets for scripts * remove all backend asset parsing * R/W/RW selector button nits * fix insert into assets not saving alt access type * support named arguments in python asset parser * improve asset usage drawer R/W indicator * update legacy $res: syntax * reactivity issue * remove last variable asset stuff * sqlx prepare * tooltip explainer * deprecated variable asset nit |
||
|
HugoCasa
|
328ef605ad |
feat: triggers error handler and retry (#6138)
* feat: http triggers error handling and retries * feat: handle sync http triggers * feat: triggers retry and error handling * feat: frontend and preprocessor * fix build * minimum allow dead code * check workspace tags in schedules * fix flow preprocesor + hub urls * update ee ref * nit * nit * final nits * schedule UI nits * migrate workspace settings to svelte 5 |
||
|
Diego Imbert
|
433341b295 |
feat: assets as a primary concept (#6125)
* assets migration * parse assets (duckdb) * iterate on assets * S3 object Preview * remove pagination * filterText * better occurence list * tweak * assets in JobPreview * clone impl * AssetsDetectedBadge * improve DbManagerButton + asset dropdown button * edit resource btn * warning when incorrect resource * +Resource in DuckDB * +S3 Object editor bar * nit fix rename * flow asset badge * More Generic OnChange * Highlight assets used in modules * Show occurence count in flow * Better UX, avoid moving parts * nit * Asset nodes * move to dedicated Asset ctx * fix layoutNodes not handling first assetsMap * explore asset btn in flow asset node * correct offset * single computeAssetNodes function * Fix y positioning of nodes with assets * resource editor * write mode node (ui) * accessType in ctx + fix insert button positioning * right positioning when mixing read and write nodes * right positioning when mixing R and W assets * Better layout fix algorithm * listAssetsByUsage and asset nodes on transitive usages * refactor + remove linkAssets * Refactor to allow for custom R/W modes * AssetsDropdownButton in flow script editor * R/W/RW selection and changes node pos in flow * layoutNodes doesnt need recompute now * fix wrong assumption that nodes recompute when assets change * r/w/rw multi toggle * MultiToggle cool animation + clearable * rename + 1px nit * remove mini toggle button group, use ToggleButtonGroup * Combinator parser that detects R / W asset context * nit fix missing flex-1 * missing order by * better ui indication for access type * special x offset case when only one asset node for clarity * parse getResource in TS with swc ecma parser * support load and write s3 detection in TS * Python asset parser * support wmill api calls without special $res: or s3:// syntax * detect out of context asset uris python * do not use access type override when not ambiguous in flow graph * parse_assets match case in rust * AsRef<str> refactor * From impl * Save flow assets * Save script asset usages + fixes + save fallback access types * asset sub icon * max total asset node width to avoid overlap * small refactor * don't parse comments in duckdb assets * fix assets clearing on parse error * fix script asset save in wrong place * load initial asset fallback access types * support variables * ui fixes * Support S3Object as URI in TS client * support new syntax in python client * Support +S3Object in EditorBar for TS and python * Reduce resource requests in assets page * import windmill client when necessary * update s3Types.d.ts * nit fix * Show input resources and s3 objects as assets * improve asset icons * DarkModeObserver refactor * asset page tabs * Moved resource variables and s3object pages to assets tabs * fetch resource usages * Get variables usages * move assets usage dropdown to component * Revert "move assets usage dropdown to component" This reverts commit |
||
 wendrul
|
835645643e |
feat: Better tracing for audit logs, including a graph to visualize them (#6078)
* Migrate audit log page to svelte 5 * Add email and span cols to audit table * Add token_prefixs to audit logs (into AuditAuthorable trait) * Add audit logs graph (wip) * Add audit span on push and jwt * Unify same job audit into the same audit span * Improve the graph visually * Fix typo * functioning graph with svelte issue * Fix leak * feat: migrate AuditLogsTable from DataTable to VirtualList for performance - Replace DataTable component with VirtualList for handling thousands of rows - Migrate to Svelte 5 runes ($props, $bindable, $derived, $state) - Implement flattenLogs() for virtual scrolling with grouped date headers - Add sticky indices and dynamic height calculation - Update parent component to use callback prop pattern instead of events - Preserve all existing functionality: filtering, selection, pagination - Follows RunsTable.svelte implementation pattern Resolves performance issues when displaying large audit log datasets. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> * Fix remaining virtual list issues * WIP graph * Fix chart styling * Fix npm check * Fix missing audit_span arguments * Update sqlx * use varchar 255 for email as in other tables * Remove syntax inconsistency * Match struct with ee crate * Update ee-repo-ref.txt * Update worker_flow.rs * Remove redefinition of trait to prevent shadowing * Re add trait on oss but only when no `private` flag --------- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com> Co-authored-by: GitHub Action <action@github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |