Ruben Fiszel
ea52a8b8ce
refactor: move integration tests to subcrates to reduce recompilation
...
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-11 14:04:54 +00:00
Ruben Fiszel
4343b73485
fix: reduce DB pool contention by eliminating dual-connection patterns ( #7861 )
2026-02-09 22:48:29 +00:00
Ruben Fiszel
ed4ea41c9c
test nit
2026-02-08 18:47:26 +00:00
Ruben Fiszel
c3812e41e1
ci: limit test threads to prevent pool exhaustion ( #7852 )
...
* ci: limit test threads to 16 to prevent postgres pool exhaustion
Running all tests with unlimited parallelism exhausts postgres
max_connections (default 100), causing sqlx::test databases to fail
setup and producing spurious RowNotFound errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
* fix: CI ruby env vars and mysql test binary coercion
- Ruby executor reads RUBY_BUNDLE_PATH and RUBY_GEM_PATH but CI was
setting BUNDLE_PATH and GEM_PATH, causing "Executable bundle not
found on worker" errors.
- MySQL test CAST(CONCAT(...) AS CHAR) returns binary type when param
is bound as bytes. Use CONVERT(? USING utf8mb4) to ensure character
result type.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-08 18:25:10 +00:00
Ruben Fiszel
477832dbee
fix: adapt mysql and ruby test assertions
...
MySQL CONCAT with binary params returns VARBINARY, causing base64
encoding. Use CAST(... AS CHAR) to force character type output.
Ruby executor doesn't support keyword parameters (name:), use
positional parameters instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-08 15:27:33 +00:00
Ruben Fiszel
f22b62d838
test: add integration tests for all feasible language runtimes in CI ( #7849 )
...
* test: add integration tests for all feasible language runtimes in CI
Add integration tests and CI setup for languages that were previously
untested. Each test runs a simple "hello world" job through the full
worker pipeline to verify end-to-end execution.
New language tests added to worker.rs:
- Nativets (4 tests): basic string, numeric args, object return, datetime
- Bunnative: TypeScript execution via Bun native runtime
- CSharp: .NET compilation and execution (feature-gated)
- PHP: PHP script execution (feature-gated)
- Ruby: Ruby script execution (feature-gated)
- MySQL: SQL query via async MySQL client (feature-gated)
- PowerShell: pwsh script execution
- PostgreSQL: SQL query against test database
CI changes (backend-test.yml):
- Add MySQL 8.0 service container
- Add setup-php (8.3 + composer), setup-ruby (3.3), pwsh install
- Enable feature flags: csharp, php, ruby, mysql
- Pass language binary paths: PHP_PATH, COMPOSER_PATH, RUBY_PATH,
BUNDLE_PATH, GEM_PATH, POWERSHELL_PATH, DOTNET_PATH
- Uncomment and modernize CSharp test (was commented out)
- Increase test timeout 16m -> 20m for additional runtimes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
* sqlx
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-08 09:11:53 +00:00
Ruben Fiszel
2eafe6df36
test: add comprehensive test coverage for extracted backend crates
...
Add 296 tests across unit and integration test suites to cover
the newly extracted crates from the recent refactor commits.
Unit tests (270):
- windmill-trigger-postgres (96): hex codec, bool parsing, type
conversion, relation tracking, replication message parsing,
publication data validation
- windmill-trigger-http (92): HMAC signature verification for
GitHub/Slack/Stripe/TikTok/Twitch/Zoom webhooks, API key auth,
Basic Auth, route validation, HTTP method/request type serde
- windmill-api-jobs (39): SQL query builder for job listing/counting
with filters, pagination, label handling
- windmill-trigger (31): TriggerMode serde, query pagination,
BaseTriggerData backward compat, HandlerAction, ServerState
- windmill-common webhook (7): WebhookMessage serialization tags
- worker nativets/postgresql (5): nativets job execution with
args/objects/datetime, postgresql query execution
Integration tests (26):
- backend/tests/triggers.rs: capture config CRUD, capture payload
operations, capture API endpoints, HTTP trigger CRUD with mode
filtering, all trigger types DB schema validation (websocket,
kafka, postgres, nats, sqs), schedule operations
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com >
2026-02-08 08:58:24 +00:00
Ruben Fiszel
b6aa05d39f
fix tests
2026-01-09 07:23:45 +00:00
claude[bot]
bfe35a8421
fix: check tag availability for flow substeps ( #7468 )
2025-12-29 17:24:49 +00:00
hugocasa
a6993823af
feat: restart flow from step with different flow version ( #7409 )
...
* feat: restart flow from step with different flow version
* fix tests
* fix tests
2025-12-18 20:27:51 +00:00
Pyra
fe56191422
feat(internal): runnable settings ( #7298 )
...
* rework everything again
Signed-off-by: pyranota <pyra@duck.com >
* updcate sqlx
Signed-off-by: pyranota <pyra@duck.com >
* update ref
Signed-off-by: pyranota <pyra@duck.com >
* fix things
Signed-off-by: pyranota <pyra@duck.com >
* fix function
Signed-off-by: pyranota <pyra@duck.com >
* final fixes
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx
Signed-off-by: pyranota <pyra@duck.com >
* fix script creation
Signed-off-by: pyranota <pyra@duck.com >
* address todo
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix
Signed-off-by: pyranota <pyra@duck.com >
* fixups
Signed-off-by: pyranota <pyra@duck.com >
* fix cargo.toml
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
* ref
Signed-off-by: pyranota <pyra@duck.com >
* fix
Signed-off-by: pyranota <pyra@duck.com >
* ee repo
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* chore: update ee-repo-ref to 505eadbff32d102ea5245a2bef88ce6f1bb95395
This commit updates the EE repository reference after PR #348 was merged in windmill-ee-private.
Previous ee-repo-ref: 195243e56cc0eab55f8890fa57297206bfe2c18c
New ee-repo-ref: 505eadbff32d102ea5245a2bef88ce6f1bb95395
Automated by sync-ee-ref workflow.
* ci: force runnable settings
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-16 21:17:06 +00:00
Diego Imbert
9bbab3321e
feat: Data tables ( #7226 )
...
* data tables settings ui
* install runed
* zod 4 fixes
* use new toJSONSchema
* Migrate ducklake catalogs to more generic custom instance databases
* fix compilation
* Safety conversion for old duckdb ffi
* data tables settings
* ts client basis
* inline run works
* datatables work
* Revert "datatables work"
This reverts commit 6e1588d59e .
* datatables work (without leaking pg credentials)
* println
* separate sqlUtils.ts
* nit
* Separate custom instance db Select and Wizard components
* nit
* nit wording
* add tags to custom instance dbs
* error when trying to use ducklake as datatable or opposite
* show status in dropdown
* data table instance setup works
* sqk function for ducklake
* factorize logic
* fix temp reactivity
* Data table assetexplore
* Migrate S3 permissions to modal
* Revert "Migrate S3 permissions to modal"
This reverts commit 0631d03cb0 .
* nit query -> fetch
* Custom instance setup new look
* run_language_executor separate fn
* run_inline param
* nit wording
* Better typed client
* Data tables display as assets in frontend
* asset db icon
* nit
* cleaner errors
* nit
* Fix sed calls in mac
* run_inline_script_preview in python client
* basic python datatable client
* datatable and datalake parser in python
* ducklake client python
* nit fix
* Fix migration producing NULL instead of {} when no custom databases
* merge conflict fail
* python ducklake client arg fix
* parse or infer sql types in ts client
* ts asset parser, detect datatable & ducklake R/W
* fix sql repl for other read ops than select
* export type SqlTemplateFunction
* rename list_custom_instance_pg_databases
* typecheck datatable and ducklake name in Typescript
* Fix typecheck datatable and ducklake in TS
* declare module overriding instead of extending
* infer_sql_type in python client
* SqlQuery object in python
* fix merge conflicts
* update const_format
* CI fix
* factor out to var_identifiers
* sqlx prepare
* unnecessary security (admin is required)
* clearer comment
* ee repo ref
* nit snake case
* claude step 1: detect var declarations
* move detect_sql_access_type to common mod
* claude step 2: detect when saved vars are queried
* Revert "claude step 2: detect when saved vars are queried"
This reverts commit 1e1f930568 .
* Revert "claude step 1: detect var declarations"
This reverts commit f866f4819d .
* remove ducklake/datatable and default
* detect data table assigns in var_identifiers
* Python parser successfully infers R/W/RW from ducklake / datatable
* still register ducklake/datatable if not used as unknown R/W
* Go to settings button in Assets Dropdown on not found
* nit
* sqlx prepare fail
* manual fix, somehow sqlx prepare won't do it
* fix frontend ci
* ee repo ref
* ducklake_user doesnt exist in unit tests
* nit fix
* ui nit
* nit
* nit missing clone
* fork ducklakes and datatables
* fix surface hover bug
* stupid mistake
* better deeply reactive mutable derived
* Ducklake picker
* Editor bar data tables
* DuckDB supports datatables
* datatable in duckdb asset parser
* duckdb asset parser var_identifiers
* Revert "duckdb asset parser var_identifiers"
This reverts commit 88068b1a77 .
* sqlx prepare
* Box pin in test_workflow_as_code to fix stack overflow
* go to settings button
* ee repo ref
* fix compilation
* wording nit
2025-12-05 23:08:58 +00:00
Diego Imbert
7b1a8b0dce
fix: improve S3 etag caching ( #7301 )
2025-12-05 18:01:27 +00:00
Pyra
1dfb0f0078
refactor: move concurrency and debouncing settings to their own structs ( #7286 )
...
* move concurrency and debouncing settings to it's own structs
Signed-off-by: pyranota <pyra@duck.com >
* cleanup queue::jobs::pull
Signed-off-by: pyranota <pyra@duck.com >
* done
Signed-off-by: pyranota <pyra@duck.com >
* migrated the rest
Signed-off-by: pyranota <pyra@duck.com >
* implement todo!
* impmenet todo! v2
* more cleanup
* fix all
Signed-off-by: pyranota <pyra@duck.com >
* remove unrelated code
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* do not serialize rawcode
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixes
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* Update ee-repo-ref.txt
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-04 10:11:54 +00:00
hugocasa
93c34c6d76
feat: flow for loop squashing ( #7107 )
...
* feat: flow for loop optimization
* fix: make dedicated flows work with flow nodes
* fix: transform dedicated inputs + better error handling + differentiate squash from same worker
* fix builds
* fix build
* fix build
* make dedicated worker private
* update ee ref
* frontend nits
* nit
* add trace to dedicated worker
* update ee ref
* Update ee-repo-ref.txt
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-25 17:25:34 +01:00
Diego Imbert
fa1bc3c711
DuckDB test to ensure FFI doesn't crash on simple query ( #7147 )
...
* test_duckdb_ffi
* build dev duckdb lib
* cache
2025-11-15 11:56:45 +00:00
dieriba
09cdfb4556
feat: webhook by flow version ( #7062 )
...
* ok
* done
* update sqlx and repo ref
* fix
2025-11-14 14:36:24 +00:00
Ramtin Mesgari
5da494b197
refactor: remove legacy database views v2_as_queue and v2_as_completed_job ( #6689 )
...
* refactor: remove legacy database views v2_as_queue and v2_as_completed_job
Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com >
* fix tests
* fix jobs.rs
* end
* fix
* improvement
* improvement
---------
Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-11-04 13:08:02 +00:00
Ruben Fiszel
5b1ab610b7
improve stack size usage ( #6936 )
...
* oom
* final
* final
2025-10-24 22:08:54 +00:00
Ruben Fiszel
04890a7a42
tmp overflow fix
2025-10-24 19:38:35 +00:00
Pyra
edece035f8
feat: job debouncing ( #6878 )
...
* feat(EE): job debouncing
Signed-off-by: pyranota <pyra@duck.com >
* remove 'script' file
Signed-off-by: pyranota <pyra@duck.com >
* more work
Signed-off-by: pyranota <pyra@duck.com >
* properly gate it behind enterprise
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* change ee repo ref again
Signed-off-by: pyranota <pyra@duck.com >
* remove unused variable
Signed-off-by: pyranota <pyra@duck.com >
* feat(EE): implement TODOs and enhance tracing for job debouncing
- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* create -> crate
Signed-off-by: pyranota <pyra@duck.com >
* remove index
Signed-off-by: pyranota <pyra@duck.com >
* some updates
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* fix it once more
Signed-off-by: pyranota <pyra@duck.com >
* Remove flow step debouncing, keep top-level flow debouncing
- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixup claude's work
Signed-off-by: pyranota <pyra@duck.com >
* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments
- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
- Debouncing feature purpose and mechanics
- Database schema for debounce_key and debounce_stale_data tables
- Version check logic and guard functions
- Improved code clarity and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* improve fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove comments from old migration
Signed-off-by: pyranota <pyra@duck.com >
* fix pull
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* Update frontend/src/lib/components/ScriptBuilder.svelte
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* add sqlx cache
Signed-off-by: pyranota <pyra@duck.com >
* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing
- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte
All frontend checks now passing (0 errors, 0 warnings)
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* remove unused import
Signed-off-by: pyranota <pyra@duck.com >
* fix ci again
Signed-off-by: pyranota <pyra@duck.com >
* udpate ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* CI doesn't want to be fixed but I still try
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* ci...
Signed-off-by: pyranota <pyra@duck.com >
* Update ee-repo-ref.txt
* safer migration
Signed-off-by: pyranota <pyra@duck.com >
* reduce noise in logs
Signed-off-by: pyranota <pyra@duck.com >
* fix cli for scripts
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
Pyra
defb6c9694
feat: dependency job debouncing ( #6769 )
...
* v0
Signed-off-by: pyranota <pyra@duck.com >
* optimize relocks
* make it work with relative relative imports
Signed-off-by: pyranota <pyra@duck.com >
* use fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg and todos
Signed-off-by: pyranota <pyra@duck.com >
* future proof a bit
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove final TODO
Signed-off-by: pyranota <pyra@duck.com >
* do not use bytemuck
Signed-off-by: pyranota <pyra@duck.com >
* optimize hashing
Signed-off-by: pyranota <pyra@duck.com >
* implementation 1
Signed-off-by: pyranota <pyra@duck.com >
* almost v0
Signed-off-by: pyranota <pyra@duck.com >
* v0
Signed-off-by: pyranota <pyra@duck.com >
* add comments and use fallback
Signed-off-by: pyranota <pyra@duck.com >
* call dissolve for apps
Signed-off-by: pyranota <pyra@duck.com >
* add comms
Signed-off-by: pyranota <pyra@duck.com >
* refactor v0 (partially tested + dirty)
Signed-off-by: pyranota <pyra@duck.com >
* finishing
Signed-off-by: pyranota <pyra@duck.com >
* remove TODO
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* silence unused argument
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* implement rebuild_map endpoint
Signed-off-by: pyranota <pyra@duck.com >
* update windmill api client
Signed-off-by: pyranota <pyra@duck.com >
* almost finish with tests
Signed-off-by: pyranota <pyra@duck.com >
* add proper testing
Signed-off-by: pyranota <pyra@duck.com >
* remove unused fixtures
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* partial cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/scoped_dependency_map.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* evil doings
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
* Update SQLx metadata
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixing CI
Signed-off-by: pyranota <pyra@duck.com >
* remove python from default features
Signed-off-by: pyranota <pyra@duck.com >
* feat: dependency job debouncing
* checkpoint
Signed-off-by: pyranota <pyra@duck.com >
* more improvements
Signed-off-by: pyranota <pyra@duck.com >
* refactor: clean up dependency job debouncing implementation
- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
- extract_to_relock_from_args() for extracting nodes/components
- accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-queue/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* better error handling for helper
Signed-off-by: pyranota <pyra@duck.com >
* update ee-ref
Signed-off-by: pyranota <pyra@duck.com >
* test: add race condition test for dependency job debouncing
- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment
The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.
This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* test: implement race condition test for dependency job debouncing
- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* implement fallback
Signed-off-by: pyranota <pyra@duck.com >
* make it mostly work
* all tests are almost working
Signed-off-by: pyranota <pyra@duck.com >
* add comments a bit
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat: improve debouncing documentation, tracing, and error handling
- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* do some work for future improvements
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update ee ref
Signed-off-by: pyranota <pyra@duck.com >
* flag tests behind the feature, add timeout
Signed-off-by: pyranota <pyra@duck.com >
* fix timeout + cleanup
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* row lock debounce_key
Signed-off-by: pyranota <pyra@duck.com >
* addressing TODOs
Signed-off-by: pyranota <pyra@duck.com >
* fix test
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* ee repo
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-10-16 16:49:35 +00:00
Alexander Petric
ae8d37fc34
feat: add dynamic skip for schedules ( #6739 )
...
* feat: add dynamic skip for schedules
* npm check
* accidental rename
* fixing tests
* Update SQLx metadata
* simplify api / queries
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
Ruben Fiszel
2047073c03
improve tests
2025-09-28 19:30:50 +00:00
Pyra
ed806bf9d0
fix(backend): rework dependency_map handling ( #6598 )
...
* v0
Signed-off-by: pyranota <pyra@duck.com >
* optimize relocks
* make it work with relative relative imports
Signed-off-by: pyranota <pyra@duck.com >
* use fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg and todos
Signed-off-by: pyranota <pyra@duck.com >
* future proof a bit
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove final TODO
Signed-off-by: pyranota <pyra@duck.com >
* do not use bytemuck
Signed-off-by: pyranota <pyra@duck.com >
* optimize hashing
Signed-off-by: pyranota <pyra@duck.com >
* implementation 1
Signed-off-by: pyranota <pyra@duck.com >
* almost v0
Signed-off-by: pyranota <pyra@duck.com >
* v0
Signed-off-by: pyranota <pyra@duck.com >
* add comments and use fallback
Signed-off-by: pyranota <pyra@duck.com >
* call dissolve for apps
Signed-off-by: pyranota <pyra@duck.com >
* add comms
Signed-off-by: pyranota <pyra@duck.com >
* refactor v0 (partially tested + dirty)
Signed-off-by: pyranota <pyra@duck.com >
* finishing
Signed-off-by: pyranota <pyra@duck.com >
* remove TODO
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* silence unused argument
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* implement rebuild_map endpoint
Signed-off-by: pyranota <pyra@duck.com >
* update windmill api client
Signed-off-by: pyranota <pyra@duck.com >
* almost finish with tests
Signed-off-by: pyranota <pyra@duck.com >
* add proper testing
Signed-off-by: pyranota <pyra@duck.com >
* remove unused fixtures
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* partial cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/scoped_dependency_map.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* evil doings
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
* Update SQLx metadata
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixing CI
Signed-off-by: pyranota <pyra@duck.com >
* remove python from default features
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-09-24 13:40:39 +00:00
Ruben Fiszel
cf70265ee9
rust tests cleanup ( #6573 )
...
* test improvements
* improve tests
* improve tests
2025-09-10 14:43:10 +00:00
hugocasa
b69478cf2b
feat: better early stop in flows ( #6534 )
...
* feat: better early stop
* style
* nits
* nits
* fix test
2025-09-05 06:25:53 +00:00
Ruben Fiszel
d3f6a0da7e
test logs
2025-09-02 21:32:21 +00:00
hugocasa
fbd942f179
fix(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs ( #6490 )
...
* feat(backend): retrieve root_job for MiniPulledJob + fix root job for flow jobs
* nit
* nits
* set root_job only if additional information meaning if not equal to innermost or parent
* nit
* feat: add root job env var
* nits
* fix build
* nit
* fix build
* nit
* sqlx
2025-09-02 09:44:09 +00:00
pyranota
006f32602c
fix: fix relative imports cache invalidation ( #6468 )
...
* fix: new script on deps job for secondary scripts
Signed-off-by: pyranota <pyra@duck.com >
* make it work (dirty)
Signed-off-by: pyranota <pyra@duck.com >
* reduce db calls
* remove `triggered_by_relative_import`
Signed-off-by: pyranota <pyra@duck.com >
* add comment to common_dependency_path
Signed-off-by: pyranota <pyra@duck.com >
* add fallback to old behavior
Signed-off-by: pyranota <pyra@duck.com >
* remove TODOs
Signed-off-by: pyranota <pyra@duck.com >
* pass deployed hash to git sync handler function
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
2025-08-27 17:44:35 +00:00
hugocasa
958e8af782
feat: ai agent steps ( #6393 )
...
* feat: ai agent steps base
* better backend and graph
* feat: anthropic, log viewer
* nit
* fix(frontend): hide tool nodes from timeline
* move ai agent actions from flow status to flow status module
* nits and workspace/hub scripts support
* tmp ref
* fix merge
* feat: display agent tools status in the graph
* fix reactivity
* fix flow status
* nit
2025-08-20 22:40:57 +00:00
pyranota
11dd4118ce
feat: add ruby support ( #5939 )
...
* implement single line pin
* make panic-safe
* use pin even if multiple modules imported withing single statement
* add repins and make imports respect pins
* keep all pins
* Allow multiple pins
* add comments + handle stuff more safely
* fix fully qualified imports
* remove ignore
* sort nested
* apply unique to output requirements list
* fix typo
* remove mut
* update sqlx
* sort imports
* sort imports
* fix formatter and format
* feat: add ruby support
* fix
* update sqlx
* add ruby to flake.nix
Signed-off-by: pyranota <pyra@duck.com >
* upgrade par_install_language_dependencies helper
rename fields + allow passing custom payload
Signed-off-by: pyranota <pyra@duck.com >
* change migrations
Signed-off-by: pyranota <pyra@duck.com >
* make full pipeline work!
Signed-off-by: pyranota <pyra@duck.com >
* do a bit fixing
* stage
Signed-off-by: pyranota <pyra@duck.com >
* add ruby in dockerfiles
Signed-off-by: pyranota <pyra@duck.com >
* implement auto-require + 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* create 'windmill/inline' only if does not exist
Signed-off-by: pyranota <pyra@duck.com >
* add nsjail
Signed-off-by: pyranota <pyra@duck.com >
* update global settings
Signed-off-by: pyranota <pyra@duck.com >
* add ruby icon
* generate lockfile on deploy (but broken)
Signed-off-by: pyranota <pyra@duck.com >
* fix(frontend): display deployed script lockfile and other info
Signed-off-by: pyranota <pyra@duck.com >
* feat: add mini-wmill client
Methods:
- get_resource
- get_variable
Additionally add shortcuts on frontend
Signed-off-by: pyranota <pyra@duck.com >
* add nit warning
Signed-off-by: pyranota <pyra@duck.com >
* support private repos credentials
Signed-off-by: pyranota <pyra@duck.com >
* fix compilation error
Signed-off-by: pyranota <pyra@duck.com >
* found rustc bug + refactor universal installer
Signed-off-by: pyranota <pyra@duck.com >
* more refactor + fixes
Signed-off-by: pyranota <pyra@duck.com >
* bug fixing + polishing
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* change 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* delegate env_clear to universal installer
Signed-off-by: pyranota <pyra@duck.com >
* update init script
Signed-off-by: pyranota <pyra@duck.com >
* add cli support
Signed-off-by: pyranota <pyra@duck.com >
* unhardcode frontend
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* fix init script
Signed-off-by: pyranota <pyra@duck.com >
* make it work on ms windows
Signed-off-by: pyranota <pyra@duck.com >
* do not clean env in the end
Signed-off-by: pyranota <pyra@duck.com >
* fix windows (again)
Signed-off-by: pyranota <pyra@duck.com >
* include PROXY_ENVS (untested)
Signed-off-by: pyranota <pyra@duck.com >
* feat: self-signed certs + proxy support (tested)
Signed-off-by: pyranota <pyra@duck.com >
* flake: pin ruby to 3.4
* merge follow-ups
Signed-off-by: pyranota <pyra@duck.com >
* remove obsolete libc crate
Signed-off-by: pyranota <pyra@duck.com >
* cleanup parser
Signed-off-by: pyranota <pyra@duck.com >
* fix java ci
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix nits
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/universal_pkg_installer.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/ruby_executor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* add nsjail for lock stage
Signed-off-by: pyranota <pyra@duck.com >
* fix merge
Signed-off-by: pyranota <pyra@duck.com >
* Update common.rs
* regen windmill-api-client
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* fix tests again
Signed-off-by: pyranota <pyra@duck.com >
* reference published ruby parser
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 13:03:32 +00:00
claude[bot]
161dbbc6d6
feat: add native result streaming ( #6242 )
...
* feat: add stream output feature to SSE job updates
Adds stream_output field to JobUpdate struct that extracts log lines
starting with '[wm_stream]:' from job logs. Regular logs now exclude
stream lines, which are captured separately for specialized handling.
- Added stream_output: Option<String> field to JobUpdate struct
- Created extract_stream_output_from_logs() function to filter stream lines
- Modified get_job_update_data() to use stream extraction logic
- SSE clients now receive both new_logs and stream_output in job updates
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* feat: rename stream_output to stream and handle newlines as \n
- Renamed `stream_output` field to `stream` in JobUpdate struct
- Updated extract_stream_output_from_logs to extract_stream_from_logs
- Changed stream output to join with literal \n instead of actual newlines
- Stream lines are properly excluded from regular new_logs
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* decision tree nits
* push ee ref
* push ee ref
* fix: fix id renaming in apps
* remove duplicate caching (#6285 )
* feat: migrate audit log ids to bigints (blocking migration for EE)
* fix(mcp): add proper check for mcp routes (#6282 )
* add proper check for mcp routes
* cleaner
* apply to flow
* fix add checks scopes
---------
Co-authored-by: dieriba <dieriba.pro@gmail.com >
* chore(main): release 1.514.0 (#6283 )
* chore(main): release 1.514.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: pin tokio to 1.46.1 and aws-sdks-ts
* pin rustls to 0.23.29 + pin aws-sdk
* chore(main): release 1.514.1 (#6288 )
* chore(main): release 1.514.1
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: improve docker logs collection in docker mode
* support $res: string in form inputs of arrays
* fix import nit
* fix: fix DynSelect
* nits
* fix: resource-type-ts-parser (#6289 )
* fix: resource types as arg in typescript handle imported defined types
* Update nix flake (#6291 )
* merge
* Small UI fixes (#6294 )
* fix step history not refreshing with staticInputs
* fix array of obj not showing up in json editor in test this step
* datatable scales correctly in DisplayResult and scrolling is much more usable
* avoid next button disapearing and changing layout / hurting ux
* nits
* fix bug when renaming module A to B then module C to A, C takes the schema of A
* fix bug with comments in sql repl
* fix aggrid theme randomly not loading
* bindable script
* better delete button in db manager
* property select doesnt exist
* fix all warnings
* delete $flowStateStore[id] on delete
* feat(cli): generate cursor rules on init (#6270 )
* create cursor rules on init
* change gen
* add missing resource-type command
* add resource type command in guidance
* add schema option
* revert
* nit
* nit
* add flow guidance
* nit
* chore(main): release 1.515.0 (#6292 )
* chore(main): release 1.515.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix: improved logs for script
* nits logs
* chore(main): release 1.515.1 (#6295 )
* chore(main): release 1.515.1
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* merge
* even more indexer tracings
* add more tracing logs
* feat: prevent too large results (>500Mb) from OOMing database
* nit naming
* feat: add CA certificate update at startup via environment variable (#6280 )
* feat: add CA certificate update at startup via environment variable
Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".
- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling
Fixes #6279
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* refactor: extract CA certificate update logic into separate function
Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* improvements
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com >
Co-authored-by: Alexander Petric <alex@windmill.dev >
* fix: indexer collection of job logs before indexing (#6300 )
* Add flume as dependecy for indexer
* Update ee-repo-ref
* Remove flags from cargo.toml
* Update ee-repo-ref
* Update ee-repo-ref
* fix rust sdk build error (#6305 )
Signed-off-by: pyranota <pyra@duck.com >
* fix broken audit logs filter (#6304 )
* rename to from to
* goto fix
* default to false if field not present operator settings (#6301 )
* git sync UI improvements (#6303 )
* ui improvements round 1
* modal cleanup
* init
* UI refactor
* UI cleanup + refactor
* legacy cleanup
* success model -> github actions, non-ee warnings
* sqlx
* npm check
* ee warning everywhere
* last comments
* formatting
* no hardcoded theme
* claude review improvemenets
* fix: no process relative imports for scripts with codebase
* fix: sqs oidc authentication disconnect #6307
* handle metadata for new scripts happen after commit
* handle_deployment_metadata in a task
* nits
* chore: add windmill-utils-internal package (#6299 )
* add utils package
* naming
* cleaning
* add docs
* remove log
* use autogenerated types
* remove old
* fix
* cleaning
* add docs
* chore(main): release 1.516.0 (#6298 )
* chore(main): release 1.516.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* merge
* indexer improvements
* upgrade tantivy to 0.24.2
* use tantivy fork
* nit warnings
* fix oss build
* improve indexer
* chore: use windmill-utils-internal for cli (#6297 )
* add utils package
* naming
* cleaning
* simplify assignPath
* rename old files
* same for locks
* create on confirm
* default true
* use replaceinlinescripts from utils
* use extractscriptfromflows
* make it compile
* cleaning
* use argsigtojson
* fix
* fix missing await
* cleaner
* cleaning
* cleaning
* use in frontend
* add docs
* testing
* remove log
* use autogenerated types
* remove old
* fix
* cleaning
* adapt usage
* draft
* better build script
* fix build
* revert to default creation
* add docs
* remove and rename
* make everything work
* add await
* only if not installed
* add vs code setting
* add to publish action
* fix bc
* safer use of sep
* fix
* do not rename on push
* no publish on release
* use published package on frontend
* nit
* Add dependencies to run sqlx prepare to nix flake (#6309 )
* feat(cli): wmill-lock.yaml v2 for easier git merge diffs
* merge
* merge
* all
* all
* rm warnings
* fix styling on aichatinput (#6312 )
* fix: use with_capacity back presusre for tantivy directory multipart writes (#6313 )
* use with capacity for tantivy directory multi part uploads
* Update ee repo ref
* Update ee-repo-ref
* Update ee-repo-ref
* chore(main): release 1.517.0 (#6310 )
* chore(main): release 1.517.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
* fix typo on cli build (#6314 )
* cleanup
* feat(utils): add flow.yaml validation function (#6316 )
* add validateflow function
* cleaner code
* preprocess json
* cleaning
* create specific package
* cleaning
* add tests
* fix: cleanup concurrency_counter automatically + remove orphans keys automatically
* fix: add disabled support to resource picker in schema forms
* fix: add wm_labels to tracing spans
* all
* merge
* all
* fix: delete empty git connection (#6318 )
* fix checks
* bun handling
* all
* all?
* all
* all
* update
* all
* update
* check
* fix history
* all
* all
* all
* Remove leftover debug tracing statements
- Remove commented debug trace in jobs.rs for stream output
- Remove commented debug trace in result_stream.rs for stream processing
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* fix test
* all
* handle iter
* fix
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
Co-authored-by: dieriba <dieriba.pro@gmail.com >
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com >
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com >
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com >
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com >
Co-authored-by: Alexander Petric <alex@windmill.dev >
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com >
2025-08-06 22:40:42 +00:00
Ruben Fiszel
564e92e98d
cleanup warnings
2025-07-26 13:48:53 +00:00
Ruben Fiszel
323ec9a994
remove warnings
2025-07-26 13:28:51 +00:00
Ruben Fiszel
f4073abef0
test fix
2025-07-24 12:08:20 +00:00
Diego Imbert
2ab5345e61
Assets refactor ( #6217 )
...
* Moved logic to FlowAssetsProvider
* Remove assetsMap in flow
* do not parse everything on mount + only check for missing assets fields
* add assets field in backend
* remove fallbackAccessTypes
* better structure and less queries / parsing
* Fix assets not showing when pulling raw_flow from jobs
* flow assets ctx for job run
* Fix transitive assets fetching
* Fix input args asset node
* enablePathScriptAndFlowAssets flag
* edit btn for variable
* untrack refresh
* move parseInputArgsAssets
* Assets tab in runs
* Update FlowStatusViewerInner to svelte 5 + fix asset sync bug
* avoid toast error on bad resource
* fetch res metadata for input arg asset
* Job assets viewer in run page
* r/w selector
* remove indigo badge
* store alt_access_type state in ScriptEditor
* Don't parse assets in flow script editor
* Add alt_access_type in backend
* show Read as selected by default to avoid giving the feeling of having made a decision
* keep alt_access_type when reparsing in flow raw scripts
* Remove variable asset kind, and save assets for scripts
* remove all backend asset parsing
* R/W/RW selector button nits
* fix insert into assets not saving alt access type
* support named arguments in python asset parser
* improve asset usage drawer R/W indicator
* update legacy $res: syntax
* reactivity issue
* remove last variable asset stuff
* sqlx prepare
* tooltip explainer
* deprecated variable asset nit
2025-07-17 22:15:01 +00:00
HugoCasa
328ef605ad
feat: triggers error handler and retry ( #6138 )
...
* feat: http triggers error handling and retries
* feat: handle sync http triggers
* feat: triggers retry and error handling
* feat: frontend and preprocessor
* fix build
* minimum allow dead code
* check workspace tags in schedules
* fix flow preprocesor + hub urls
* update ee ref
* nit
* nit
* final nits
* schedule UI nits
* migrate workspace settings to svelte 5
2025-07-11 18:55:23 +02:00
Diego Imbert
433341b295
feat: assets as a primary concept ( #6125 )
...
* assets migration
* parse assets (duckdb)
* iterate on assets
* S3 object Preview
* remove pagination
* filterText
* better occurence list
* tweak
* assets in JobPreview
* clone impl
* AssetsDetectedBadge
* improve DbManagerButton + asset dropdown button
* edit resource btn
* warning when incorrect resource
* +Resource in DuckDB
* +S3 Object editor bar
* nit fix rename
* flow asset badge
* More Generic OnChange
* Highlight assets used in modules
* Show occurence count in flow
* Better UX, avoid moving parts
* nit
* Asset nodes
* move to dedicated Asset ctx
* fix layoutNodes not handling first assetsMap
* explore asset btn in flow asset node
* correct offset
* single computeAssetNodes function
* Fix y positioning of nodes with assets
* resource editor
* write mode node (ui)
* accessType in ctx + fix insert button positioning
* right positioning when mixing read and write nodes
* right positioning when mixing R and W assets
* Better layout fix algorithm
* listAssetsByUsage and asset nodes on transitive usages
* refactor + remove linkAssets
* Refactor to allow for custom R/W modes
* AssetsDropdownButton in flow script editor
* R/W/RW selection and changes node pos in flow
* layoutNodes doesnt need recompute now
* fix wrong assumption that nodes recompute when assets change
* r/w/rw multi toggle
* MultiToggle cool animation + clearable
* rename + 1px nit
* remove mini toggle button group, use ToggleButtonGroup
* Combinator parser that detects R / W asset context
* nit fix missing flex-1
* missing order by
* better ui indication for access type
* special x offset case when only one asset node for clarity
* parse getResource in TS with swc ecma parser
* support load and write s3 detection in TS
* Python asset parser
* support wmill api calls without special $res: or s3:// syntax
* detect out of context asset uris python
* do not use access type override when not ambiguous in flow graph
* parse_assets match case in rust
* AsRef<str> refactor
* From impl
* Save flow assets
* Save script asset usages + fixes + save fallback access types
* asset sub icon
* max total asset node width to avoid overlap
* small refactor
* don't parse comments in duckdb assets
* fix assets clearing on parse error
* fix script asset save in wrong place
* load initial asset fallback access types
* support variables
* ui fixes
* Support S3Object as URI in TS client
* support new syntax in python client
* Support +S3Object in EditorBar for TS and python
* Reduce resource requests in assets page
* import windmill client when necessary
* update s3Types.d.ts
* nit fix
* Show input resources and s3 objects as assets
* improve asset icons
* DarkModeObserver refactor
* asset page tabs
* Moved resource variables and s3object pages to assets tabs
* fetch resource usages
* Get variables usages
* move assets usage dropdown to component
* Revert "move assets usage dropdown to component"
This reverts commit 622ea4ab12 .
* Revert "Get variables usages"
This reverts commit b11ced4e29 .
* Revert "fetch resource usages"
This reverts commit aa5187ad4b .
* Revert "Moved resource variables and s3object pages to assets tabs"
This reverts commit 4430487be4 .
* Revert "asset page tabs"
This reverts commit dacc2f0da5 .
* move assets usage dropdown to component
* asset icon in asset pages
* tooltip
* details
* Storage selector in S3 File Picker
* make edge less opaque
* Refactor computeAssetNodes to separate in and out nodes
* AssetsOverflowedNode
* nits
* fix assets not being parsed in flows sometimes
* show asset kind and resource_type
* ui nits
* support res:// in duckdb
* add banner for old deployments
* Fix permissionning
* fix broken disable /enable all
* assets page view permission for operators
* Disable ExploreAssetButton for operators
* asset kind as subtitle
* do not spam getResource in assets page. prob. revert fail
* update assets page on workspace change
* reload storage names on ws change
* delete assets on archive / deletion
* sqlx prepare
* missing update when updating user
* add indexes on asset
* better message
* missing loadInit: false
* dead code
* use transaction
* typo
* update package.json
* update package.json
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-07-11 09:56:53 +00:00
wendrul
835645643e
feat: Better tracing for audit logs, including a graph to visualize them ( #6078 )
...
* Migrate audit log page to svelte 5
* Add email and span cols to audit table
* Add token_prefixs to audit logs (into AuditAuthorable trait)
* Add audit logs graph (wip)
* Add audit span on push and jwt
* Unify same job audit into the same audit span
* Improve the graph visually
* Fix typo
* functioning graph with svelte issue
* Fix leak
* feat: migrate AuditLogsTable from DataTable to VirtualList for performance
- Replace DataTable component with VirtualList for handling thousands of rows
- Migrate to Svelte 5 runes ($props, $bindable, $derived, $state)
- Implement flattenLogs() for virtual scrolling with grouped date headers
- Add sticky indices and dynamic height calculation
- Update parent component to use callback prop pattern instead of events
- Preserve all existing functionality: filtering, selection, pagination
- Follows RunsTable.svelte implementation pattern
Resolves performance issues when displaying large audit log datasets.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Fix remaining virtual list issues
* WIP graph
* Fix chart styling
* Fix npm check
* Fix missing audit_span arguments
* Update sqlx
* use varchar 255 for email as in other tables
* Remove syntax inconsistency
* Match struct with ee crate
* Update ee-repo-ref.txt
* Update worker_flow.rs
* Remove redefinition of trait to prevent shadowing
* Re add trait on oss but only when no `private` flag
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-07-03 21:02:45 +00:00
pyranota
a2334629d3
ci: add tests for python's global-site-packages ( #6112 )
...
Signed-off-by: pyranota <pyra@duck.com >
2025-07-01 23:38:26 +02:00
pyranota
50a5c1f56a
fix(python): assign PATCH version to python runtime only when needed ( #5866 )
...
* fix
* clean up
* nit
* fix integration tests
* nit
* another nit
* remove dublicate test
2025-06-04 17:15:46 +02:00
pyranota
26222539e6
feat(python): inline script metadata (PEP 723) ( #5712 )
...
* make resolver
* more updates
* fix build
* fix raw_dependencies job type
* compat with http agent workers
* refactor
* rename
* more refactor
* cleanup
* more tests
* fix s3
* small fixes
* more fixing
* fix endpoint
* nit: update comment
* update ee ref
* update ee ref
* update ee ref
* implement safer `list_available_python_versions`
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783 )
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix build
* update ee ref
* ignore versions <=3.9
* fix windows build
* correct versions filter
* fix windows build (this time for real)
* inject error to debug CI
* update CI
* undo debug of CI
* fix tests
* remove outdated comment
* update ee repo ref
* Update ee-repo-ref.txt
* Update backend/parsers/windmill-parser-py-imports/src/lib.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update InstanceSetting.svelte
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
HugoCasa
254c3cf8ef
feat: preprocessor refactor ( #5629 )
...
* feat: preprocessor refactor
* nit
* oneOf kind support + new capture format + fix build
* fix build
* fix build
* update preprocessor templates
* cache runnable version info
* update ee ref
* update hub scripts && ee ref
2025-05-09 09:50:01 +02:00
Ruben Fiszel
afc9cf5940
update deps versions ( #5703 )
2025-05-05 16:19:39 +02:00
Ruben Fiszel
d2ad2cdf33
nit feature flag better backend tests
2025-05-04 17:58:06 +02:00
pyranota
1ab41603f4
fix: make #(extra_)requirements: work better with pins ( #5680 )
2025-04-29 19:59:51 +02:00
centdix
1625524311
feat: Add MCP only mode ( #5661 )
...
* add mcp mode
* fix flag usage
* change mcp path
* do not migrate db in mcp mode
* handle null values
2025-04-25 14:22:45 +02:00
pyranota
0b6d017fed
feat(python): per import requirement pin ( #5520 )
...
* implement single line pin
* make panic-safe
* use pin even if multiple modules imported withing single statement
* add repins and make imports respect pins
* keep all pins
* Allow multiple pins
* add comments + handle stuff more safely
* fix fully qualified imports
* remove ignore
* sort nested
* apply unique to output requirements list
* fix typo
* remove mut
* update sqlx
* sort imports
* sort imports
* fix formatter and format
* refactor
* fix comptime error
* write tests
* perf: do not capture if string is empty
2025-04-11 21:31:51 +00:00
Ruben Fiszel
63fa499015
feat: agent workers v2 using http ( #5588 )
2025-04-10 00:56:11 +00:00