Commit Graph

6608 Commits

Author SHA1 Message Date
Ruben Fiszel 200242aa3b chore(main): release 1.563.2 (#6862)
* chore(main): release 1.563.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 18:30:16 +00:00
Ruben Fiszel f581ce6248 fix: monitor less frequent cleanup process periodicity 2025-10-17 18:27:11 +00:00
Ruben Fiszel 4fdbd9184e chore(main): release 1.563.1 (#6861)
* chore(main): release 1.563.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 17:21:32 +00:00
Ruben Fiszel c8d2a036b6 re-route agent workers job if needed prior to applying concurrency limits 2025-10-17 17:15:05 +00:00
Ruben Fiszel 6faea9adad fix: fix concurrency limit behavior with remote agents dep jobs 2025-10-17 16:59:29 +00:00
Ruben Fiszel c3299e8b8b chore(main): release 1.563.0 (#6856)
* chore(main): release 1.563.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-17 14:27:49 +00:00
hugocasa 8cd0006498 feat: http routes streaming (#6834)
* feat: http routes streaming

* improve webhook config section for SSE

* feat: add request type config to http routes

* fix check

* sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-17 14:05:17 +00:00
Ruben Fiszel d2ae0533f4 fix select grant 2025-10-17 13:59:58 +00:00
wendrul e719be54e0 Fix ansible delegate mode only working with ssh (#6858) 2025-10-17 13:58:00 +00:00
Ruben Fiszel 60e17e5068 fix: add grant select on debounce_key 2025-10-17 13:55:02 +00:00
Pyra 2868eeda26 perf: remove unnecessary db call (#6853)
* perf: remove unnecessary db call

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-17 11:51:55 +00:00
Ruben Fiszel 5c02af11c3 remove benchmark from all_features_oss 2025-10-17 08:28:06 +00:00
Ruben Fiszel 34cd68676b fix migration error 2025-10-17 03:42:41 +00:00
Ruben Fiszel cc64f8acef chore(main): release 1.562.0 (#6841)
* chore(main): release 1.562.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-16 21:54:50 +00:00
Diego Imbert 72c6bad0dd create ducklake_user_pg_pwd if deleted by CLI (#6845) 2025-10-16 21:54:00 +00:00
Ruben Fiszel ea8c9cc766 remove safety migration 2025-10-16 21:29:44 +00:00
hugocasa c86b3448b8 feat: add support for sage intacct oauth (#6794)
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 20:40:25 +00:00
Diego Imbert a4a502cf3a ducklake safety for instance_settings.yaml users (#6844) 2025-10-16 20:39:11 +00:00
Ruben Fiszel e5a11a7bc6 split main async with a box pin 2025-10-16 20:31:46 +00:00
Ruben Fiszel 3dd75ad18d fix rare stack overflow bc of async size 2025-10-16 20:05:09 +00:00
Ruben Fiszel 6543a83d9f fix rare stack overflow bc of async size 2025-10-16 20:01:30 +00:00
wendrul 83337b2035 Ansible repo update hubPath + fixes (#6843)
* Allow ssh auth

* ssh changes pt2

* Fix unused

* Update package.json and hubPaths
2025-10-16 18:24:41 +00:00
Pyra 952a15a877 remove dbg! leftovers (#6842)
Signed-off-by: pyranota <pyra@duck.com>
2025-10-16 17:29:57 +00:00
Ruben Fiszel 0c4e14e81c fix compile 2025-10-16 17:21:09 +00:00
Pyra defb6c9694 feat: dependency job debouncing (#6769)
* v0

Signed-off-by: pyranota <pyra@duck.com>

* optimize relocks

* make it work with relative relative imports

Signed-off-by: pyranota <pyra@duck.com>

* use fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg and todos

Signed-off-by: pyranota <pyra@duck.com>

* future proof a bit

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* remove final TODO

Signed-off-by: pyranota <pyra@duck.com>

* do not use bytemuck

Signed-off-by: pyranota <pyra@duck.com>

* optimize hashing

Signed-off-by: pyranota <pyra@duck.com>

* implementation 1

Signed-off-by: pyranota <pyra@duck.com>

* almost v0

Signed-off-by: pyranota <pyra@duck.com>

* v0

Signed-off-by: pyranota <pyra@duck.com>

* add comments and use fallback

Signed-off-by: pyranota <pyra@duck.com>

* call dissolve for apps

Signed-off-by: pyranota <pyra@duck.com>

* add comms

Signed-off-by: pyranota <pyra@duck.com>

* refactor v0 (partially tested + dirty)

Signed-off-by: pyranota <pyra@duck.com>

* finishing

Signed-off-by: pyranota <pyra@duck.com>

* remove TODO

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* silence unused argument

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* implement rebuild_map endpoint

Signed-off-by: pyranota <pyra@duck.com>

* update windmill api client

Signed-off-by: pyranota <pyra@duck.com>

* almost finish with tests

Signed-off-by: pyranota <pyra@duck.com>

* add proper testing

Signed-off-by: pyranota <pyra@duck.com>

* remove unused fixtures

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* partial cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/scoped_dependency_map.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* evil doings

Signed-off-by: pyranota <pyra@duck.com>

* more cleanup

* Update SQLx metadata

* more cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixing CI

Signed-off-by: pyranota <pyra@duck.com>

* remove python from default features

Signed-off-by: pyranota <pyra@duck.com>

* feat: dependency job debouncing

* checkpoint

Signed-off-by: pyranota <pyra@duck.com>

* more improvements

Signed-off-by: pyranota <pyra@duck.com>

* refactor: clean up dependency job debouncing implementation

- Add comprehensive comments explaining the debouncing mechanism
- Replace debug statements (dbg!) with proper tracing calls
- Extract helper functions to reduce code duplication:
  - extract_to_relock_from_args() for extracting nodes/components
  - accumulate_debounce_stale_data() for updating stale data
- Improve code readability and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* Update SQLx metadata

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-common/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-queue/src/jobs.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* better error handling for helper

Signed-off-by: pyranota <pyra@duck.com>

* update ee-ref

Signed-off-by: pyranota <pyra@duck.com>

* test: add race condition test for dependency job debouncing

- Implement test_2 for scripts to test the race condition edge case
- Add comprehensive documentation comments to all test functions
- Remove empty test_2 stubs for flows (not needed)
- Keep test_2 stub for apps with TODO comment

The race condition test simulates the scenario where a job is marked
as running but debounce_key hasn't been cleaned up yet, forcing the
system to create a new job while reusing the existing debounce_key.

This edge case can occur due to the lack of transactions in the pull
function for performance reasons (see jobs.rs:4415-4425).

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* test: implement race condition test for dependency job debouncing

- Add comprehensive test_2 for script module that tests the race condition edge case
- Remove empty test_2 stubs from flows and apps modules
- Fix unused variable warning in worker_lockfiles.rs
- Add detailed comments explaining the race condition scenario and test logic

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* implement fallback

Signed-off-by: pyranota <pyra@duck.com>

* make it mostly work

* all tests are almost working

Signed-off-by: pyranota <pyra@duck.com>

* add comments a bit

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-common/src/scripts.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/worker_lockfiles.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat: improve debouncing documentation, tracing, and error handling

- Add comprehensive 3-phase debouncing documentation explaining push/accumulation/pull
- Enhance all tracing with structured logging (job_id, workspace_id, node_count, etc.)
- Add proper error handling with .map_err() and contextual messages
- Replace dbg!() with proper tracing::debug!()
- Replace todo!() with proper error handling
- Fix typos: 'and edge case' → 'an edge case', 'bc' → 'because'
- Fix debug variable name: 'debounce_job_id_0' → 'debounce_job_id_o'
- Add documentation for debounce cleanup and stale data retrieval
- Add trace-level logging for non-error paths to reduce noise

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* do some work for future improvements

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update sqlx

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* flag tests behind the feature, add timeout

Signed-off-by: pyranota <pyra@duck.com>

* fix timeout + cleanup

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* row lock debounce_key

Signed-off-by: pyranota <pyra@duck.com>

* addressing TODOs

Signed-off-by: pyranota <pyra@duck.com>

* fix test

Signed-off-by: pyranota <pyra@duck.com>

* ee ref

Signed-off-by: pyranota <pyra@duck.com>

* ee repo

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Pyra <92104930+pyranye@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 16:49:35 +00:00
Ruben Fiszel 0c1fbc6f0d fix compile 2025-10-16 16:28:04 +00:00
Ruben Fiszel d16cc56f58 nit compile 2025-10-16 16:24:36 +00:00
Ruben Fiszel f723a1fb72 fix: add configurable timeout sse stream 2025-10-16 15:29:49 +00:00
Ruben Fiszel 0fe81f5b98 chore(main): release 1.561.0 (#6838)
* chore(main): release 1.561.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-16 15:07:59 +00:00
hugocasa 3b5c962473 fix(backend): revert flow node opti for ai agents (#6840)
* Revert "feat(backend): use flow nodes opti for ai agent steps (#6808)"

This reverts commit 8d5acda340.

* fix(backend): revert flow node opti for ai agents

* keep standard base64
2025-10-16 14:05:06 +00:00
Ruben Fiszel aeb6829011 use excluded where relevant 2025-10-16 09:19:44 +00:00
wendrul 32fae7a10c feat: ansible playbook execution git repo mode (repo viewer + UI utils) (#6831)
* Improve minio on flake.nix

* Add first asset parsing logic for ansible

* Correct html gt sign

* Decouple s3 file picker from drawer

* Factor duplicate code into snippet

* Update S3FilePickerInner to be compatible

* Fix pane shrinking issue

* Git repo viewer

* Change GitRepoViewer

* Endpoints for git repo visualizer

* Move git repo viewer to its own component

* Add button to populate git repo viewer

* Update parser yaml for new ansisble features (repo viewer)

* Reflect parser changes for ansible

* Add button to add the git repo mode of declaration for ansible

* Factor function

* Playbook + inventories into the drawer

* Add button to add inventories from s3

* Move tests to lib.rs

* Inventory loading from s3

* Move get github app token logic to be reused by ansible

* Update parser and ansible executor

* Use the correct path for inventories

* Add nushell to flake for wasm builds

* Add published parser

* Update hubPaths with clone and upload to s3

* Update ee-repo to the branch ref

* Fix npm run check

* Update cargo.lock

* Change labels on buttons

* Remove debug log

* Update ee-repo-ref

* Fix ee issues

* Update ee-repo ref

* Fix typo

* Fix ee

* Update ee-repo-ref

* Fix missing imports

* Unused var

* Fix typo

* Layout improvents

* Fix typos

* Remove unused function and log

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-10-16 09:08:18 +00:00
Ruben Fiszel 06c05200cd chore(main): release 1.560.0 (#6823)
* chore(main): release 1.560.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-15 22:39:10 +00:00
Alexander Petric 48acc57823 fix: always create instance groups with uuid (#6826)
* fix: always create instance groups with uuid

* Update SQLx metadata

* repo ref

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-15 22:30:09 +00:00
Ruben Fiszel 892ce64ea8 fix: fix concurrency key filter 2025-10-15 22:24:11 +00:00
Ruben Fiszel 3d5631938f error 2025-10-15 22:12:09 +00:00
hugocasa 8d5acda340 feat(backend): use flow nodes opti for ai agent steps (#6808) 2025-10-14 18:06:27 +00:00
hugocasa 9d9c29fdfa feat: add support for zoho oauth (#6809) 2025-10-14 17:32:24 +00:00
dieriba 256522273e fix: bug for loop flow inconsistent state (#6815)
* fix

* Update frontend/src/lib/components/flows/content/FlowLoop.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-10-14 16:00:15 +00:00
Ruben Fiszel 2736586298 chore(main): release 1.559.0 (#6796)
* chore(main): release 1.559.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-14 15:27:13 +00:00
centdix 784f988b07 chore(mcp): display API endpoint tools in MCP URL generation (#6811)
* chore(mcp): display API endpoint tools in MCP URL generation

- Add list of 24 API endpoint tools that are always available via MCP
- Display API endpoints as green badges below scripts/flows list
- Update tooltip to mention 'scripts, flows, and API endpoints'
- Show count of available API endpoints in the UI

API endpoints include operations for variables, resources, scripts, flows,
jobs, schedules, and workers.

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* better

* nit

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-14 09:22:58 +00:00
Pyra e2feba391c feat(rust): add resource types (#5843)
* feat(rust): add resource types

* write tests

* copy from go-resources

* implement frontend for rust

* update

* update cli

* update tests

* update

Signed-off-by: pyranota <pyra@duck.com>

* fix package-lock

* fix ci

* update parser

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/Editor.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-10-13 19:15:44 +00:00
wendrul 9149faf305 fix: Safeguard prevents button from deleting non-fork workspaces (#6795)
* Fix: Safeguard to prevent `delete fork` button from deleting non-fork workspaces

* fix typo

* Consistent prefix
2025-10-10 18:53:30 +00:00
Alexander Petric e0d9017036 fix: show workspace prefix to non superadmins for app deploy custom path (#6793) 2025-10-10 18:52:56 +00:00
hugocasa b972eb9721 feat: add support for contextual vars in SQL (#6791) 2025-10-10 18:52:24 +00:00
Ruben Fiszel 7ebd1bc56c chore(main): release 1.558.1 (#6789)
* chore(main): release 1.558.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-09 21:08:10 +00:00
Alexander Petric 1c9faf9d03 fix: support pg jsonb array (#6788) 2025-10-09 21:01:43 +00:00
Ruben Fiszel 1faba5b097 make js expr more resilient in case where previous result is error 2025-10-09 20:59:00 +00:00
Ruben Fiszel 877286193e chore(main): release 1.558.0 (#6787)
* chore(main): release 1.558.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-10-09 19:46:04 +00:00
Alexander Petric 705a177005 feat: allow setting custom cors header on http trigger (#6786)
* feat: allow setting custom cors header on http trigger

* preflight

* headers one by one

* perf: optimize conditional_cors_middleware by checking existing headers first

Improves performance by iterating through existing headers once and using
flags to track which CORS headers need to be inserted, avoiding unnecessary
header lookups for the common case where headers are not present

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: use not_insert flags in conditional_cors_middleware for clarity

Changed the conditional_cors_middleware logic to use not_insert_* flags
instead of needs_* flags as suggested, which better represents the intent
when iterating through existing headers first.

Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
2025-10-09 19:41:54 +00:00