4451a37999
feat: debuggers for python and bun v0 ( #7546 )
2026-01-13 15:20:06 +00:00
a3da19dbc3
fix(backend): remove itered from parallel for-loop status ( #7424 )
...
* fix(backend): remove itered from parallel for-loop
* remove fk on v2_job_queue
2026-01-05 20:52:48 +00:00
4807a429c9
fix(backend): run flow error handler for flow step oom ( #7484 )
2026-01-05 12:09:33 +00:00
7fbc70add0
fix: add help subcommand ( #7221 )
...
* add help subcommand
* imporve env variables
2025-12-26 13:15:57 +00:00
9d698dabb4
feat: v2 job debouncing ( #7411 )
...
* rework everything again
Signed-off-by: pyranota <pyra@duck.com >
* updcate sqlx
Signed-off-by: pyranota <pyra@duck.com >
* update ref
Signed-off-by: pyranota <pyra@duck.com >
* fix things
Signed-off-by: pyranota <pyra@duck.com >
* fix function
Signed-off-by: pyranota <pyra@duck.com >
* final fixes
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx
Signed-off-by: pyranota <pyra@duck.com >
* fix script creation
Signed-off-by: pyranota <pyra@duck.com >
* address todo
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix
Signed-off-by: pyranota <pyra@duck.com >
* fixups
Signed-off-by: pyranota <pyra@duck.com >
* fix cargo.toml
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
* ref
Signed-off-by: pyranota <pyra@duck.com >
* fix
Signed-off-by: pyranota <pyra@duck.com >
* ee repo
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* feat: v2 job debouncing
Signed-off-by: pyranota <pyra@duck.com >
* v2 debouncing
Signed-off-by: pyranota <pyra@duck.com >
* compat
Signed-off-by: pyranota <pyra@duck.com >
* fix all
Signed-off-by: pyranota <pyra@duck.com >
* remove legacy logic
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
* finish
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* stage import
Signed-off-by: pyranota <pyra@duck.com >
* remove unused import
Signed-off-by: pyranota <pyra@duck.com >
* remove comments
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-queue/src/jobs.rs
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/runnable_settings/settings.rs
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* Update frontend/src/lib/components/flows/DebounceLimit.svelte
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* fix nits
Signed-off-by: pyranota <pyra@duck.com >
* force no compat in ci
Signed-off-by: pyranota <pyra@duck.com >
* better default key handling
Signed-off-by: pyranota <pyra@duck.com >
* simplify
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* optimize
Signed-off-by: pyranota <pyra@duck.com >
* sqlx
Signed-off-by: pyranota <pyra@duck.com >
* fix flow step
Signed-off-by: pyranota <pyra@duck.com >
* add openapi
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* rename WMDEBUG flag
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* remove gate for is_flow_step on pull
Signed-off-by: pyranota <pyra@duck.com >
* remove dead code
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* chore: update ee-repo-ref to 71d872d27de44227b82f75a9c0879d8428684880
This commit updates the EE repository reference after PR #369 was merged in windmill-ee-private.
Previous ee-repo-ref: 4c7642dc9f7f4467fc055136bd56558b66ef2d54
New ee-repo-ref: 71d872d27de44227b82f75a9c0879d8428684880
Automated by sync-ee-ref workflow.
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-23 13:24:04 +00:00
5f182bc28a
nit warns
2025-12-19 12:57:06 +00:00
ff0d1d444e
cgroups only on linux
2025-12-19 12:48:15 +00:00
806024403e
fix: disable oomgroup by default
2025-12-19 12:45:21 +00:00
e197e1b064
unsafe set var to rustc latest compat
2025-12-08 19:27:38 +00:00
9bbab3321e
feat: Data tables ( #7226 )
...
* data tables settings ui
* install runed
* zod 4 fixes
* use new toJSONSchema
* Migrate ducklake catalogs to more generic custom instance databases
* fix compilation
* Safety conversion for old duckdb ffi
* data tables settings
* ts client basis
* inline run works
* datatables work
* Revert "datatables work"
This reverts commit 6e1588d59e0631d03cb01e1f930568f866f4819d88068b1a77
2025-12-05 23:08:58 +00:00
e15af78d93
feat: agent workers min version support ( #7284 )
...
* feat: agent workers min version support
Signed-off-by: pyranota <pyra@duck.com >
* update oapi
Signed-off-by: pyranota <pyra@duck.com >
* fix what claude wrote
Signed-off-by: pyranota <pyra@duck.com >
* final cleanup
Signed-off-by: pyranota <pyra@duck.com >
* change a bit
Signed-off-by: pyranota <pyra@duck.com >
* remove unsed import
Signed-off-by: pyranota <pyra@duck.com >
* run update_min_version periodically
Signed-off-by: pyranota <pyra@duck.com >
* Update ee-repo-ref.txt
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-12-03 13:48:07 +00:00
d38c96db36
feat: workspace dependencies ( #7124 )
...
* commit raw requirements
Signed-off-by: pyranota <pyra@duck.com >
* raw requirements
Signed-off-by: pyranota <pyra@duck.com >
* implement `parse_annotation`
Signed-off-by: pyranota <pyra@duck.com >
* more progress on wdeps
Signed-off-by: pyranota <pyra@duck.com >
* more progress
Signed-off-by: pyranota <pyra@duck.com >
* fixes
Signed-off-by: pyranota <pyra@duck.com >
* more progress
Signed-off-by: pyranota <pyra@duck.com >
* fixes
Signed-off-by: pyranota <pyra@duck.com >
* cli improvements + raw deps
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix python versions
Signed-off-by: pyranota <pyra@duck.com >
* progress
Signed-off-by: pyranota <pyra@duck.com >
* update :)
Signed-off-by: pyranota <pyra@duck.com >
* add MaybeLock
Signed-off-by: pyranota <pyra@duck.com >
* go WIP
Signed-off-by: pyranota <pyra@duck.com >
* fix python ignoring py version from requirements
Signed-off-by: pyranota <pyra@duck.com >
* optimize php
Signed-off-by: pyranota <pyra@duck.com >
* require admin to alter
Signed-off-by: pyranota <pyra@duck.com >
* fix(cli): flow generateLocks raw deps
Signed-off-by: pyranota <pyra@duck.com >
* progress in checklist
Signed-off-by: pyranota <pyra@duck.com >
* fix agent workers
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* nit: remove default features
Signed-off-by: pyranota <pyra@duck.com >
* oh-wow
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg!
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* add indexes
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* remove todos
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* add debug flag
Signed-off-by: pyranota <pyra@duck.com >
* cli: remove noise
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* remove todos
Signed-off-by: pyranota <pyra@duck.com >
* trigger deps correctly
Signed-off-by: pyranota <pyra@duck.com >
* fix frontend
Signed-off-by: pyranota <pyra@duck.com >
* fix frontend again
Signed-off-by: pyranota <pyra@duck.com >
* finally fix frontend
Signed-off-by: pyranota <pyra@duck.com >
* ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* fix all
Signed-off-by: pyranota <pyra@duck.com >
* more fixes...
Signed-off-by: pyranota <pyra@duck.com >
* remove test
Signed-off-by: pyranota <pyra@duck.com >
* Update backend-test.yml
* comment out legacy test
Signed-off-by: pyranota <pyra@duck.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* fix ci?
Signed-off-by: pyranota <pyra@duck.com >
* comment out thing
Signed-off-by: pyranota <pyra@duck.com >
* ignore test
Signed-off-by: pyranota <pyra@duck.com >
* ci
Signed-off-by: pyranota <pyra@duck.com >
* base fixture
Signed-off-by: pyranota <pyra@duck.com >
* fix regression
Signed-off-by: pyranota <pyra@duck.com >
* fix docs links
Signed-off-by: pyranota <pyra@duck.com >
* update min version
Signed-off-by: pyranota <pyra@duck.com >
* simplify
* implement cache for get_latest
Signed-off-by: pyranota <pyra@duck.com >
* move to workspace settings
Signed-off-by: pyranota <pyra@duck.com >
* sqlx + migration
Signed-off-by: pyranota <pyra@duck.com >
* more migrations
Signed-off-by: pyranota <pyra@duck.com >
* use box pin
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-11-28 16:00:11 +00:00
93c34c6d76
feat: flow for loop squashing ( #7107 )
...
* feat: flow for loop optimization
* fix: make dedicated flows work with flow nodes
* fix: transform dedicated inputs + better error handling + differentiate squash from same worker
* fix builds
* fix build
* fix build
* make dedicated worker private
* update ee ref
* frontend nits
* nit
* add trace to dedicated worker
* update ee ref
* Update ee-repo-ref.txt
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-25 17:25:34 +01:00
3ae3b40cc7
nit style
2025-11-21 07:43:23 +00:00
e9691c9eb0
feat(ee): support iamrds
2025-11-20 18:58:38 +00:00
ceeff5f76c
feat: support to run windows binary as service ( #7153 )
...
* feat: support to run windows binary as service
* ee ref
* ee ref
* flags
* Update backend/src/main.rs
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* fix ai commit
* fix
* ee ref
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-11-17 11:48:31 +00:00
137adbf813
nit
2025-11-14 11:32:38 +00:00
0c9d268e2f
nit
2025-11-14 11:28:17 +00:00
ec4998d866
nits license ee check
2025-11-14 10:41:03 +00:00
5da494b197
refactor: remove legacy database views v2_as_queue and v2_as_completed_job ( #6689 )
...
* refactor: remove legacy database views v2_as_queue and v2_as_completed_job
Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com >
* fix tests
* fix jobs.rs
* end
* fix
* improvement
* improvement
---------
Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-11-04 13:08:02 +00:00
2d1c1d81ba
internal(backend): mini completed job + improvements ( #7042 )
...
* mini completed job
* mini completed job
* sqlx
* restore dedi
* restore dedi
* nits
* fix
2025-11-04 09:31:11 +00:00
d86ce2e3e3
fix: better handle same worker zombie job
2025-10-31 07:36:38 +00:00
edece035f8
feat: job debouncing ( #6878 )
...
* feat(EE): job debouncing
Signed-off-by: pyranota <pyra@duck.com >
* remove 'script' file
Signed-off-by: pyranota <pyra@duck.com >
* more work
Signed-off-by: pyranota <pyra@duck.com >
* properly gate it behind enterprise
Signed-off-by: pyranota <pyra@duck.com >
* update ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* change ee repo ref again
Signed-off-by: pyranota <pyra@duck.com >
* remove unused variable
Signed-off-by: pyranota <pyra@duck.com >
* feat(EE): implement TODOs and enhance tracing for job debouncing
- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* create -> crate
Signed-off-by: pyranota <pyra@duck.com >
* remove index
Signed-off-by: pyranota <pyra@duck.com >
* some updates
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* fix it once more
Signed-off-by: pyranota <pyra@duck.com >
* Remove flow step debouncing, keep top-level flow debouncing
- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixup claude's work
Signed-off-by: pyranota <pyra@duck.com >
* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments
- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
- Debouncing feature purpose and mechanics
- Database schema for debounce_key and debounce_stale_data tables
- Version check logic and guard functions
- Improved code clarity and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* improve fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove comments from old migration
Signed-off-by: pyranota <pyra@duck.com >
* fix pull
Signed-off-by: pyranota <pyra@duck.com >
* fix once more
Signed-off-by: pyranota <pyra@duck.com >
* Update frontend/src/lib/components/ScriptBuilder.svelte
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* add sqlx cache
Signed-off-by: pyranota <pyra@duck.com >
* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing
- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte
All frontend checks now passing (0 errors, 0 warnings)
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* fix ci
Signed-off-by: pyranota <pyra@duck.com >
* remove unused import
Signed-off-by: pyranota <pyra@duck.com >
* fix ci again
Signed-off-by: pyranota <pyra@duck.com >
* udpate ee repo ref
Signed-off-by: pyranota <pyra@duck.com >
* CI doesn't want to be fixed but I still try
Signed-off-by: pyranota <pyra@duck.com >
* nits
Signed-off-by: pyranota <pyra@duck.com >
* ci...
Signed-off-by: pyranota <pyra@duck.com >
* Update ee-repo-ref.txt
* safer migration
Signed-off-by: pyranota <pyra@duck.com >
* reduce noise in logs
Signed-off-by: pyranota <pyra@duck.com >
* fix cli for scripts
Signed-off-by: pyranota <pyra@duck.com >
* nit
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
caa21fcb33
fix: debounce_key automatic deletion ( #6885 )
...
* fix: debounce_key automatic deletion
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* better solution
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-21 18:31:05 +00:00
f581ce6248
fix: monitor less frequent cleanup process periodicity
2025-10-17 18:27:11 +00:00
defb6c9694
feat: dependency job debouncing ( #6769 )
...
* v0
Signed-off-by: pyranota <pyra@duck.com >
* optimize relocks
* make it work with relative relative imports
Signed-off-by: pyranota <pyra@duck.com >
* use fallback
Signed-off-by: pyranota <pyra@duck.com >
* remove dbg and todos
Signed-off-by: pyranota <pyra@duck.com >
* future proof a bit
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* remove final TODO
Signed-off-by: pyranota <pyra@duck.com >
* do not use bytemuck
Signed-off-by: pyranota <pyra@duck.com >
* optimize hashing
Signed-off-by: pyranota <pyra@duck.com >
* implementation 1
Signed-off-by: pyranota <pyra@duck.com >
* almost v0
Signed-off-by: pyranota <pyra@duck.com >
* v0
Signed-off-by: pyranota <pyra@duck.com >
* add comments and use fallback
Signed-off-by: pyranota <pyra@duck.com >
* call dissolve for apps
Signed-off-by: pyranota <pyra@duck.com >
* add comms
Signed-off-by: pyranota <pyra@duck.com >
* refactor v0 (partially tested + dirty)
Signed-off-by: pyranota <pyra@duck.com >
* finishing
Signed-off-by: pyranota <pyra@duck.com >
* remove TODO
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* silence unused argument
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* implement rebuild_map endpoint
Signed-off-by: pyranota <pyra@duck.com >
* update windmill api client
Signed-off-by: pyranota <pyra@duck.com >
* almost finish with tests
Signed-off-by: pyranota <pyra@duck.com >
* add proper testing
Signed-off-by: pyranota <pyra@duck.com >
* remove unused fixtures
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* partial cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/scoped_dependency_map.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* evil doings
Signed-off-by: pyranota <pyra@duck.com >
* more cleanup
* Update SQLx metadata
* more cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fixing CI
Signed-off-by: pyranota <pyra@duck.com >
* remove python from default features
Signed-off-by: pyranota <pyra@duck.com >
* feat: dependency job debouncing
* checkpoint
Signed-off-by: pyranota <pyra@duck.com >
* more improvements
Signed-off-by: pyranota <pyra@duck.com >
* refactor: clean up dependency job debouncing implementation
- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
- extract_to_relock_from_args() for extracting nodes/components
- accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* Update SQLx metadata
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-common/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-queue/src/jobs.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* better error handling for helper
Signed-off-by: pyranota <pyra@duck.com >
* update ee-ref
Signed-off-by: pyranota <pyra@duck.com >
* test: add race condition test for dependency job debouncing
- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment
The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.
This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* test: implement race condition test for dependency job debouncing
- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* implement fallback
Signed-off-by: pyranota <pyra@duck.com >
* make it mostly work
* all tests are almost working
Signed-off-by: pyranota <pyra@duck.com >
* add comments a bit
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-common/src/scripts.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/worker_lockfiles.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* feat: improve debouncing documentation, tracing, and error handling
- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
* do some work for future improvements
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update sqlx
Signed-off-by: pyranota <pyra@duck.com >
* clippy
Signed-off-by: pyranota <pyra@duck.com >
* update ee ref
Signed-off-by: pyranota <pyra@duck.com >
* flag tests behind the feature, add timeout
Signed-off-by: pyranota <pyra@duck.com >
* fix timeout + cleanup
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* row lock debounce_key
Signed-off-by: pyranota <pyra@duck.com >
* addressing TODOs
Signed-off-by: pyranota <pyra@duck.com >
* fix test
Signed-off-by: pyranota <pyra@duck.com >
* ee ref
Signed-off-by: pyranota <pyra@duck.com >
* ee repo
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com >
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-10-16 16:49:35 +00:00
aeb6829011
use excluded where relevant
2025-10-16 09:19:44 +00:00
e1d2eb2870
feat: support for protected private hub ( #6762 )
...
* feat: support for protected private hub
* nits
* nits
2025-10-08 09:33:28 +00:00
ae8d37fc34
feat: add dynamic skip for schedules ( #6739 )
...
* feat: add dynamic skip for schedules
* npm check
* accidental rename
* fixing tests
* Update SQLx metadata
* simplify api / queries
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-08 09:18:15 +00:00
4bbbeb956f
feat: powershell private repo support ( #6684 )
...
* feat: powershell private repo support
* nits
2025-09-29 13:24:41 +00:00
d56dea4969
fix: show more autoscaling events ( #6704 )
...
* all
* all
2025-09-29 13:24:10 +00:00
8f4fef9804
feat(backend): job result stream optimization ( #6673 )
...
* feat(backend): job result stream optimization
* get offset locally instead of from db
* fix: agent worker result stream
* update ref
* nit
* remove foreign key on job
* fix build
2025-09-26 11:26:50 +00:00
30b4ef9044
nit compile
2025-09-22 16:22:03 +00:00
f6dd78cb11
fix: improve reliability of exits in case graceful handler didn't exit as expected
2025-09-22 10:27:57 +00:00
56cdb69e59
fix: add termination handler earlier in lifecycle
2025-09-20 00:22:15 +00:00
e98bde6be6
feat: allow operator to use script/flow with dynselect input ( #6616 )
...
* done
* update sqlx
* fix openapi spec
2025-09-17 10:56:39 +00:00
20f48e6ded
feat(ai agent): handle images in ai agent ( #6572 )
...
* add in frontend
* draft openai handling
* upload to s3
* simpler output
* return s3 directly if any
* low quality
* implement for gemini
* handle imagen model
* handle image input
* cleaning
* remove base64 from output
* cleaning
* fix timeout
* handle openrouter
* remove log
* allow image input when creating image
* cleaning
* increase stack size
* inline everything
* revert stack size
* cleaning
* fix for openai
* better mime type
* add descriptions
2025-09-15 07:51:09 +00:00
6b34123aee
fix: add OTEL_ENVIRONMENT to force otel environment
2025-09-09 14:21:06 +00:00
20f4086a77
feat: add worker_group_job_stats table for efficient job metrics aggregation ( #6527 )
...
* feat: add worker_group_job_stats table for job metrics aggregation
- Add new table with hour timestamp, worker group, script lang, workspace_id, job count and total duration
- Workers accumulate stats in memory and update hourly via sum aggregation
- Monitor.rs cleans up rows older than 60 days periodically
- Stats are flushed on worker shutdown to prevent data loss
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* all
* all
* ee-repo-ref
* nits
* nits
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-09-08 17:27:58 +00:00
d3f6a0da7e
test logs
2025-09-02 21:32:21 +00:00
593da3213f
add variable caching api
2025-09-02 14:34:46 +00:00
f3fd1e90b0
feat: refactor trigger crud ( #6472 )
...
* base
* base crate
* websocket updated
* crud refactored
* fix and nits
* fix compiler warning, nits and update repo ref
* fix oss
* fix compilation
* update ref
* fix
* update feature
* listener base
* add listener
* refactor logic done and implemented for postgres
* fix capture
* websocket
* implem for all triggers
* update sqlx,repo ref and fix
* fix oss
* unify struct fix websocket
* nits and fix oss runtime axum error
* perf cache query
* update .sqlx
* update .sqlx
* fix
* fix unused
* fix
* fix
* nits and fix http handler update endpoint
* update .sqlx
* update repo ref
* nits
* fix
* update ref
* fix
* update .sqlx
2025-09-02 11:28:06 +00:00
db36a323a7
feat(perf): add 60-second local cache for variables and resources( #6511 )
...
* feat: add 60-second cache for variables and resources with PostgreSQL invalidation
- Add new var_resource_cache module with 60-second TTL
- Implement PostgreSQL NOTIFY/LISTEN for immediate cache invalidation
- Cache get_variable() for non-secret, non-encrypted reads
- Cache get_resource_value() for all read operations
- Add database triggers on variable/resource table changes
- Initialize cache system in main.rs after database connection
- Add Clone derive to ListableVariable for cache compatibility
Performance benefits:
- Avoids database queries on cache hits
- Immediate invalidation ensures data consistency
- Selective caching respects security constraints
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Claude <noreply@anthropic.com >
* refactor: move cache to API layer with allow_cache query param
- Move variable/resource cache from windmill-common to windmill-api
- Add allow_cache query parameter to variables and resources endpoints
- Follow raw script cache pattern with timestamp + value structure
- Create proper database migration for notification triggers
- Include encrypted values in cache when appropriate
- Only activate caching when explicitly requested
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* refactor: move cache to API layer with allow_cache query param
Move PostgreSQL LISTEN logic to main.rs following established pattern
- Remove custom listener initialization from cache module
- Add cache invalidation channels to centralized notification handler
- Simplified cache module to only handle cache operations
- Follow raw script cache pattern for notification handling
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
2025-09-02 07:48:03 +00:00
5510b1dd95
typo env nit
2025-08-31 12:23:37 +00:00
8f46066a64
feat(perf): add 300-second local cache for variable crypt retrieval ( #6483 )
...
* feat: add 60-second cache for workspace key retrieval
This implements a cache with 60-second staleness for the get_workspace_key
function to reduce database queries for workspace encryption keys. The cache
follows the same pattern as the existing CUSTOM_ENVS_CACHE but with a
shorter expiration time.
Requested by @rubenfiszel
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Extend cache staleness from 60 to 300 seconds
* feat: add cache invalidation notifications for workspace keys
Add PostgreSQL LISTEN/NOTIFY mechanism to invalidate workspace key cache
across all servers and workers when workspace keys change.
- Add database migration with trigger function for workspace_key changes
- Add notification handler in main.rs to remove from WORKSPACE_KEY_CACHE
- Follow same pattern as existing workspace environment cache invalidation
- Ensures distributed cache consistency for workspace encryption keys
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* finish
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
2025-08-31 12:19:59 +00:00
2ce11cea3b
prevent past due team plans to overuse ( #6487 )
...
* prevent past due team plans to overuse
* nit
* update ee ref
2025-08-28 16:56:57 +00:00
4ad0d255f3
feat: autovacuum or high intensity tables
2025-08-27 17:29:06 +00:00
99666426ec
nits
2025-08-23 07:54:59 +00:00
49ed757424
feat: add prometheus metric queue_running_count ( #6413 )
...
* feat: add prometheus metric queue_running_count
Adds a new Prometheus metric queue_running_count that tracks the number
of currently running jobs per tag, similar to the existing queue_count
metric but filtered for running=true instead of running=false.
Changes:
- Added get_queue_running_counts() function in windmill-common/src/queue.rs
- Added QUEUE_RUNNING_COUNT Prometheus metric in monitor.rs
- Added /workers/queue_running_counts API endpoint
- Updated OpenAPI specification
- Added SQLx query cache entry
Requested by @rubenfiszel
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Remove database insertion for queue_running_count metrics
Keep Prometheus metrics and API endpoint functionality while removing
the database INSERT statements as requested.
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* sqlx
* improve logic
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-08-19 22:12:36 +00:00
11dd4118ce
feat: add ruby support ( #5939 )
...
* implement single line pin
* make panic-safe
* use pin even if multiple modules imported withing single statement
* add repins and make imports respect pins
* keep all pins
* Allow multiple pins
* add comments + handle stuff more safely
* fix fully qualified imports
* remove ignore
* sort nested
* apply unique to output requirements list
* fix typo
* remove mut
* update sqlx
* sort imports
* sort imports
* fix formatter and format
* feat: add ruby support
* fix
* update sqlx
* add ruby to flake.nix
Signed-off-by: pyranota <pyra@duck.com >
* upgrade par_install_language_dependencies helper
rename fields + allow passing custom payload
Signed-off-by: pyranota <pyra@duck.com >
* change migrations
Signed-off-by: pyranota <pyra@duck.com >
* make full pipeline work!
Signed-off-by: pyranota <pyra@duck.com >
* do a bit fixing
* stage
Signed-off-by: pyranota <pyra@duck.com >
* add ruby in dockerfiles
Signed-off-by: pyranota <pyra@duck.com >
* implement auto-require + 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* create 'windmill/inline' only if does not exist
Signed-off-by: pyranota <pyra@duck.com >
* add nsjail
Signed-off-by: pyranota <pyra@duck.com >
* update global settings
Signed-off-by: pyranota <pyra@duck.com >
* add ruby icon
* generate lockfile on deploy (but broken)
Signed-off-by: pyranota <pyra@duck.com >
* fix(frontend): display deployed script lockfile and other info
Signed-off-by: pyranota <pyra@duck.com >
* feat: add mini-wmill client
Methods:
- get_resource
- get_variable
Additionally add shortcuts on frontend
Signed-off-by: pyranota <pyra@duck.com >
* add nit warning
Signed-off-by: pyranota <pyra@duck.com >
* support private repos credentials
Signed-off-by: pyranota <pyra@duck.com >
* fix compilation error
Signed-off-by: pyranota <pyra@duck.com >
* found rustc bug + refactor universal installer
Signed-off-by: pyranota <pyra@duck.com >
* more refactor + fixes
Signed-off-by: pyranota <pyra@duck.com >
* bug fixing + polishing
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* change 'windmill/inline'
Signed-off-by: pyranota <pyra@duck.com >
* delegate env_clear to universal installer
Signed-off-by: pyranota <pyra@duck.com >
* update init script
Signed-off-by: pyranota <pyra@duck.com >
* add cli support
Signed-off-by: pyranota <pyra@duck.com >
* unhardcode frontend
Signed-off-by: pyranota <pyra@duck.com >
* fix cli
Signed-off-by: pyranota <pyra@duck.com >
* fix init script
Signed-off-by: pyranota <pyra@duck.com >
* make it work on ms windows
Signed-off-by: pyranota <pyra@duck.com >
* do not clean env in the end
Signed-off-by: pyranota <pyra@duck.com >
* fix windows (again)
Signed-off-by: pyranota <pyra@duck.com >
* include PROXY_ENVS (untested)
Signed-off-by: pyranota <pyra@duck.com >
* feat: self-signed certs + proxy support (tested)
Signed-off-by: pyranota <pyra@duck.com >
* flake: pin ruby to 3.4
* merge follow-ups
Signed-off-by: pyranota <pyra@duck.com >
* remove obsolete libc crate
Signed-off-by: pyranota <pyra@duck.com >
* cleanup parser
Signed-off-by: pyranota <pyra@duck.com >
* fix java ci
Signed-off-by: pyranota <pyra@duck.com >
* cleanup
Signed-off-by: pyranota <pyra@duck.com >
* fix nits
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/universal_pkg_installer.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/ruby_executor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* add nsjail for lock stage
Signed-off-by: pyranota <pyra@duck.com >
* fix merge
Signed-off-by: pyranota <pyra@duck.com >
* Update common.rs
* regen windmill-api-client
Signed-off-by: pyranota <pyra@duck.com >
* fix tests
Signed-off-by: pyranota <pyra@duck.com >
* fix tests again
Signed-off-by: pyranota <pyra@duck.com >
* reference published ruby parser
Signed-off-by: pyranota <pyra@duck.com >
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 13:03:32 +00:00
Ruben Fiszel
hugocasa
Aananth K
Pyra
Diego Imbert
Alexander Petric
Ramtin Mesgari
dieriba
centdix
claude[bot]