Commit Graph

6608 Commits

Author SHA1 Message Date
Ruben Fiszel dfa090f9bd update ee ref 2024-11-29 19:23:47 +01:00
HugoCasa 8b20530584 apply jwt ext tag scopes to single job endpoints (#4824)
* apply jwt ext tag scopes to single job endpoints

* fix sqlx
2024-11-29 19:23:20 +01:00
wendrul 088c666a7f feat: indexer extra settings + parallel downloads + many improvements (#4822)
* Nits on the service logs page

* Show all the hosts returned by query + sumOtherDocCount warning

* Remove from index endpoint

* Fix tests

* Prepare sqlx
2024-11-29 19:14:05 +01:00
Ruben Fiszel aa1c3702d7 fix empty branchall subflow 2024-11-29 19:02:47 +01:00
Ruben Fiszel 45d434c875 prettify cache msg 2024-11-29 13:01:54 +01:00
Ruben Fiszel b1603690f3 nit pip install 2024-11-29 12:37:52 +01:00
Ruben Fiszel d36e2a2647 add total time 2024-11-29 12:35:10 +01:00
Ruben Fiszel 9db86306a9 small uv s3 pull fix 2024-11-29 12:31:50 +01:00
Ruben Fiszel f6eeef0dc6 chore(main): release 1.433.0 (#4819)
* chore(main): release 1.433.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-29 10:21:39 +01:00
Ruben Fiszel 0a95b6f9d2 fix: fix windows build 2024-11-29 09:53:05 +01:00
Lucas Abel 3fbb2bfc8a feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Ruben Fiszel f7908682d3 chore(main): release 1.432.0 (#4818)
* chore(main): release 1.432.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-29 09:12:46 +01:00
Ruben Fiszel 8954b0b68c skip s3 python cache on windows and pro 2024-11-29 08:35:08 +01:00
pyranota 96371bc89d feat: Parallelize uv install (#4774)
* Implement MVP of Parallel uv installation

* Implement PY_CONCURRENT_DOWNLOADS

* Remove Flock for uv installs

* Make S3 pull/push parallel

* Refactor and allow to Cancel installation

* Dont print S3 in output if disabled

* Implement better error handling

* Polishing

* More polishing

* Implement error-handler for kill_tx_2.send()

* Fix and Format prev merge

* Presubscribe to all kill_tx's

We do it now before first event could fire
Meaning no events can be lost anymore

* Early print errors and safer error handling

* Return Err if installation failed

* Final changes

* Return error instead of just printing it

* Safer the way to acquire permit

* Fix compilation error

* Remove double error logs
2024-11-29 08:21:51 +01:00
Ruben Fiszel 12dbbab401 chore(main): release 1.431.1 (#4816)
* chore(main): release 1.431.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-28 17:36:52 +01:00
Lucas Abel 390ee3113b fix: invalid null comparison while inserting flow node (#4815) 2024-11-28 17:29:16 +01:00
Ruben Fiszel 8a814d4699 chore(main): release 1.431.0 (#4788)
* chore(main): release 1.431.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-28 17:22:24 +01:00
Ruben Fiszel 8f56456701 fix: global timeout not set correctly on load 2024-11-28 16:02:52 +01:00
Lucas Abel 77937a494a fix: invalid jsonb comparison while inserting flow node (#4814) 2024-11-28 14:43:54 +01:00
pyranota a853c8d6a5 Remove flock (#4809) 2024-11-28 12:42:25 +01:00
Ruben Fiszel ac8d359dfb fix apply migration to current_user 2024-11-27 19:07:31 +01:00
wendrul 09fe535bdc fix: service logs: aggregation query + killpill on index pull + retention period for logs on s3 (#4795)
* Search across all hosts in index. killpill on index pull

* Add retention period for service logs

* Remove console log

* monitor log_file when older than biggest retention period (job / log)

* Retention period deletes service and job logs

* Add setting to monitor logs out of s3

* Prepare sqlx

* Update ee repo ref

* Prepare sqlx

* Unused variables
2024-11-27 18:57:23 +01:00
Ruben Fiszel c998102cf3 simplify logic to check for license key on workers 2024-11-27 18:51:06 +01:00
Alexander Petric 2668e37574 adding sqlx migration to grant windmill permissions to db user (#4806) 2024-11-27 18:27:06 +01:00
Ruben Fiszel c25c638bdf add more links to operators menu 2024-11-27 17:53:45 +01:00
Lucas Abel 2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
pyranota 34609a7d8c Fix tar prefix (#4781)
* Fix tar prefix

* Use TAR_PIP(PYTHON_311)_CACHE_DIR instead

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-27 15:54:47 +01:00
Lucas Abel d86ea30221 enable is_simple optimization for FlowScript (#4799)
* backend: remove some allocations in `worker_flow.rs`

* feat(backend): enable `is_simple` optimization for `FlowScript`

* fix: fix node retrieval for `flowscript` inner step
2024-11-27 10:06:19 +01:00
Lucas Abel 150ca33f0d fix: disable flow_version_lite in dedicated workers (#4800) 2024-11-27 10:02:44 +01:00
pyranota e876ae4f42 fix(python): Merge to site-packages packages with same name (#4793)
* (python) Initially implement merge

Iterate overall all python paths and if same folder has same name multiple times,
then merge the content and put to <job_dir>/site-packages

Solves problem with imports for some dependencies.

Default layout (/windmill/cache/):

dep==x.y.z
└── X
   └── A
dep-ext==x.y.z
└── X
   └── B

In this case python would be confused with finding B module.

This function will convert it to (/<job_id>):

site-packages
└── X
   ├── A
   └── B

This way python has no problems with finding correct module

* Better error-handling

* Optimize and fix bug

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Polish

* Add path to PathBuf

* Add & before PathBuf

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-11-26 18:24:26 +01:00
Ruben Fiszel 2427d64aa8 nit 2024-11-25 23:32:14 +01:00
Lucas Abel e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Ruben Fiszel dd5081592f email quotas only apply to top-level job 2024-11-25 16:20:55 +01:00
Ruben Fiszel e81e239f43 fix: fix getting logs locally for ee without instance settings 2024-11-25 12:02:35 +01:00
Ruben Fiszel cd7385f38d improve error message for log file not found on ee 2024-11-25 11:07:48 +01:00
Ruben Fiszel 1baf9d008b chore(main): release 1.430.2 (#4782)
* chore(main): release 1.430.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-25 09:42:04 +01:00
Lucas Abel 29faefe856 fix: rename job workspace as well (#4785) 2024-11-25 08:09:30 +01:00
Lucas Abel 39b902c118 backend: keep tracking minimal worker version (#4786)
Be sure to also track the min version of workers that have been started
after the min version variable has been set. e.g. All workers are up to
date and `MIN_VERSION_IS_AT_LEAST_1_427` is set to true, a worker is
then powered on with an anterior version, leading to a false positive
value. This commit fix that.
2024-11-25 08:08:16 +01:00
Ruben Fiszel 6768e5bbbd fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
Ruben Fiszel 5ed0ae697b improve backoff for schedules 2024-11-24 18:48:01 +01:00
Ruben Fiszel 89456f502f remove rsmq 2024-11-24 11:04:54 +01:00
Ruben Fiszel 278f593358 fix: retry on pushing next scheduled job of schedule 2024-11-24 08:52:51 +01:00
Ruben Fiszel e4f1ac1f7c chore(main): release 1.430.1 (#4780)
* chore(main): release 1.430.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-23 08:52:59 +01:00
Ruben Fiszel 495d4487bc fix: expose DISABLE_DENO_LOCK 2024-11-23 08:39:57 +01:00
Ruben Fiszel c194e124da chore(main): release 1.430.0 (#4764)
* chore(main): release 1.430.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-22 16:41:12 +01:00
wendrul 4facf3ca3e feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Patrick Dobbs e854318f29 Update lib.rs with python-taiga import alias (#4776)
* Update lib.rs with python-taiga import alias

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-21 23:18:38 +01:00
Alexander Petric 8909bea935 feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
HugoCasa b49ba59da7 fix: infer python list inner type from default if unknown inner (#4771) 2024-11-21 14:44:19 +01:00
wendrul 99ab5471fc indexer: Add early exit on receiving of handoff signal (#4773) 2024-11-21 14:44:08 +01:00