c4adaeeabd
fix: improve piptar upload - sequential uploads via background task queue ( #5994 )
...
* feat: improve piptar upload with sequential background task queue
Replace simultaneous piptar uploads with sequential processing via background job queue while keeping dependency installation parallelized.
Key changes:
- Add PiptarUpload job kind and database migration
- Queue piptar uploads instead of using tokio::spawn
- Implement job handler for sequential S3 uploads
- Maintain parallel dependency installation as requested
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Remove PiptarUpload job kind migration files
Refactoring piptar uploads to use tokio channel instead of Windmill jobs.
Migration files are no longer needed.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Refactor piptar uploads from Windmill jobs to tokio channel
Replace the complex job queue infrastructure for piptar uploads with a simple
tokio channel approach as requested. This maintains sequential upload behavior
while keeping dependency installation parallel.
Key changes:
- Add PIPTAR_UPLOAD_CHANNEL global channel for sequential processing
- Replace JobPayload::PiptarUpload with simple channel send
- Remove PiptarUpload from JobKind enum and all job handling code
- Remove job dispatcher case from worker.rs
- Simplify upload logic while maintaining same functionality
Benefits:
- Reduced complexity by removing unnecessary job infrastructure
- Sequential uploads without blocking dependency installation
- Better separation of concerns
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Update worker.rs
* Update python_executor.rs
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-06-19 15:32:21 +02:00
d2dfd27b88
nit error message
2025-06-10 15:33:48 +02:00
2c689e0666
usage query on cloud is in bg
2025-06-05 17:00:39 +02:00
4c4d8a655c
usage query on cloud is in bg
2025-06-05 15:36:15 +02:00
f2526571a3
feat: ssh repl like direct to workers hosts machine ( #5809 )
...
* feat(backend): add repl for worker
* feat(frontend): add repl to interact with worker and missing packages
* nits
* Update backend/windmill-worker/src/result_processor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/result_processor.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update frontend/src/lib/components/WorkerRepl.svelte
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* fix typo
* fix cd
* nits
* refactoring
* feat: handle other agent worker to also access repl feature, fix bugs
* update repo ref
* nits and match function args
* nits
* typo
* remove struct AgentWorkerData
* update repo ref
* nits
* impl new for authed client and revert to async
* update ref
* updage ee repo ref
* fix missing method/imports small bugs
* update ee repo ref
* update .sqlx
* test
* clean wait for interactive_shell future
* free call stack
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-06-03 18:41:14 +02:00
0e316239dd
EE Refactor ( #5844 )
...
* app compiles with every ee substituted
* Replace all oss files content
* Revert "Replace all oss files content"
This reverts commit ea4017d59f
2025-06-02 22:12:33 +02:00
26222539e6
feat(python): inline script metadata (PEP 723) ( #5712 )
...
* make resolver
* more updates
* fix build
* fix raw_dependencies job type
* compat with http agent workers
* refactor
* rename
* more refactor
* cleanup
* more tests
* fix s3
* small fixes
* more fixing
* fix endpoint
* nit: update comment
* update ee ref
* update ee ref
* update ee ref
* implement safer `list_available_python_versions`
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783 )
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix build
* update ee ref
* ignore versions <=3.9
* fix windows build
* correct versions filter
* fix windows build (this time for real)
* inject error to debug CI
* update CI
* undo debug of CI
* fix tests
* remove outdated comment
* update ee repo ref
* Update ee-repo-ref.txt
* Update backend/parsers/windmill-parser-py-imports/src/lib.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update InstanceSetting.svelte
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
d662e18f97
add more labels to traces
2025-05-22 12:07:21 +02:00
30edcdfe0e
fix: workspace preprocessor improvements ( #5784 )
...
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783 )
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix: workspace preprocessor fixes
* tmp ee ref
* fix build
* update ee ref
* fix: hub script preprocessor handling
* fix build
* good ref
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
2025-05-21 17:49:21 +02:00
59afa493fa
fix: improve handling of custom concurrency key/tag with preprocessors ( #5762 )
...
* fix
* rm spurrious diff
* handle failing preprocessors
* nits
* update
* update
2025-05-17 10:52:28 +02:00
3441996b00
add more duration warnings to logs
2025-05-14 10:27:55 +02:00
254c3cf8ef
feat: preprocessor refactor ( #5629 )
...
* feat: preprocessor refactor
* nit
* oneOf kind support + new capture format + fix build
* fix build
* fix build
* update preprocessor templates
* cache runnable version info
* update ee ref
* update hub scripts && ee ref
2025-05-09 09:50:01 +02:00
0ac8e477d6
feat: handle . in interpolated args
2025-05-07 13:57:05 +02:00
f662cf5d75
fix(python): handle better relative imports with requirements or extra_requirements
2025-04-29 15:23:44 +02:00
654e3e857e
slower pull if no tag is available
2025-04-28 00:34:30 +02:00
3c68fef2ab
ui code builder v0 (at secret path) ( #4964 )
2025-04-20 01:19:02 +02:00
7a4f64b92d
improve perf of push
2025-04-12 15:26:56 +00:00
12ba15c928
improve benchmarks
2025-04-12 11:09:21 +00:00
6339775404
feat: add gcp trigger ( #5501 )
...
* update: add migration for gcp_trigger table, add cli method gcp and add gcp_trigger args for github ci
* update: add gcp module and routing it in lib.rs
* update: added gcp type to TRIGGER KIND type in db, add new feature condition to try_get_fn and added gcp_trigger feature in cargo file
* update: add ui for gcp trigger, added missing triggers type for job kin, add gcp trigger in deploy to functionallity
* adding google cloud crate
* update: handle pull and push delivery
* update: handle pull and push delivery
* update: handle push event, changed front ui
* update: ui
* fix: capture for gcp
* update: automatically manage pub sub subscription and refactoring
* capture done
* update cli types
* fix: wrong func argument and type openapi
* fix: missing import
* update .sqlx
* update: svg color and remove pulse button for capture push gcp
* update: handle deployto
* update script helper and link to hub gcp script/flow
* update gcp representation
* update: add confirmation modal
* add unique index to mitigate same subscriber id, fix `zombie worker`, update test connection logic
* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update cli/gen/services.gen.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* nits
* update repo ref
* fix feature function
* update dependencies
* update .sqlx
* fix missing import
* update: handle existing subscription and creating new one or update it
* fix ee build
* handle existing config properly
* update .sqlx
* update .sqlx
* remove unused
* update documentation link fix ci
* use on instance of empty_string_as_none fn
* update refo ref
* updat script_helper
* update ui
* update migration
* add missing arguments
* fix and nits
* update repo ref
* remove unused
* nits
* add doc links
* nits
* Update frontend/src/lib/components/triggers/gcp/GcpTriggerPanel.svelte
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* use trigger path as default route path
* update .sqlx
* update repo ref
* update ref
* update route path
* update types
* fix deploy to
* nits
* update repo ref
* update .sqlx
* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorConfigSection.svelte
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* nits
* nits
* nits
* remove dupliacte fn and update repo ref
* remove unused import
* fix missing import
* fix: wong name var
* update script helper and template script
* Update frontend/src/lib/script_helpers.ts
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update ee-repo-ref.txt
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch >
2025-04-10 20:39:01 +00:00
63fa499015
feat: agent workers v2 using http ( #5588 )
2025-04-10 00:56:11 +00:00
1bb402bcfc
fix benchmarks
2025-04-03 21:17:13 +00:00
3d3250194d
fix: correctly run empty flow with preprocessor from UI ( #5537 )
2025-03-28 11:26:26 -07:00
f3a0af8fe6
bump versions ( #5526 )
...
* all
* version bump
* ci
* fix bump
* fix
2025-03-27 23:57:09 +01:00
2e74595a37
nit, add scheduled_for in logs
2025-03-27 21:28:21 +01:00
63fe9c1852
fix: add missing privileged hub script for app slack reports ( #5515 )
2025-03-26 23:15:03 +01:00
5573d88695
perf: cache workspace premium check
2025-03-25 02:00:51 +01:00
ec11d577c6
fix: improve cancel for flows with many substeps
2025-03-24 22:00:15 +01:00
3edca4bc91
perf: optimize number of queries needed for job run ( #5504 )
...
* optPerf
* update sqlx
* update sqlx
2025-03-24 20:12:26 +01:00
551c0ecd6a
perf: improve perf of get completed flow node ( #5418 )
...
* perf: improve perf of get completed flow node
* better
* nit
* nits
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-03-19 10:51:30 +01:00
fb532e0b85
nit move away from using views
2025-03-19 10:48:43 +01:00
880db319e8
feat: filter by worker + backend perf opt ( #5489 )
...
* all
* all
* all
* refactor
* sqlx
* ref
* impr
* fix
* fix
* fix
* ee private
2025-03-19 09:55:53 +01:00
8a7b119d4d
improve concurrency limits II ( #5485 )
2025-03-17 17:24:50 +01:00
eee7d33bd8
fix: improve concurrency limit check performances
2025-03-17 00:53:14 +01:00
8d71124c20
DISABLE_CONCURRENCY_LIMIT 2
2025-03-16 18:22:41 +01:00
85d13e650d
DISABLE_CONCURRENCY_LIMIT
2025-03-16 17:45:40 +01:00
e8bb307502
fix: improve row lock contention on concurrency counter
2025-03-16 17:20:39 +01:00
11ed81a6c7
nit zombie job monitoring
2025-03-11 07:08:28 +01:00
fba9e7ef03
fix: improve cancel performance
2025-03-03 23:50:13 +01:00
c90fe387e8
fix: schedules do not accept 5 units cron syntax on update/create anymore
2025-02-20 17:59:32 +01:00
588ff79364
make benchmarks even more stable
2025-02-18 08:43:03 +01:00
953082681e
fix: improve que job indices for faster performances
2025-02-17 21:26:46 +01:00
bf206515e8
fix(backend): improve schedule queries plan to leverage indices better for performance ( #5273 )
...
* backend: fix schedule queries plan
* rework
* fix skipped as non success
* update sqlx
* update
* all
* fix npm run check
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-02-12 16:34:03 +01:00
4828a77f21
fix: fix concurrent limit jobs non restarting
2025-02-10 11:33:19 +01:00
403826fca9
fix: worker name in job + better timeout handling for same_worker jobs ( #5248 )
2025-02-08 22:20:06 +01:00
4849af04de
backend: fix workflow as code ( #5239 )
...
* backend: improve `/get_job_update` after v2
* backend: insert missing `workflow_as_code_status` on completion
also insert `flow_status` from so we can remove the query when `_duration` is above 500
* backend: fix workflow_as_code after v2
* backend: add `workflow_as_code` worker test
2025-02-07 12:27:43 +01:00
d51c3080f7
backend: use v2 tables through views where possible (v2 phase 3) ( #5119 )
2025-02-06 12:43:24 +01:00
e0d99ab703
backend: check queries to queue and completed_job where possible (v2 phase 2) ( #5125 )
2025-02-06 12:39:32 +01:00
3e8e0c653e
backend: migrate job tables to v2 schema (v2 phase 1) ( #5084 )
2025-02-06 12:37:11 +01:00
abf094bcbc
fix uv install for nsjail ( #5222 )
2025-02-06 00:00:33 +01:00
90ba65ae20
improve error messages for internal err
2025-02-05 16:32:01 +01:00
claude[bot]
Ruben Fiszel
dieriba
Diego Imbert
pyranota
HugoCasa
Lucas Abel