Commit Graph
3554 Commits
Author SHA1 Message Date
Ruben Fiszel b81bf99cc2 prettify cache msg 2024-11-29 13:01:54 +01:00
Ruben Fiszel 6891bf604c nit pip install 2024-11-29 12:37:52 +01:00
Ruben Fiszel 1b74cbaebb add total time 2024-11-29 12:35:10 +01:00
Ruben Fiszel e700016780 small uv s3 pull fix 2024-11-29 12:31:50 +01:00
Ruben Fiszelandrubenfiszel 7dc6328bf6 chore(main): release 1.433.0 (#4819)
* chore(main): release 1.433.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-29 10:21:39 +01:00
Ruben Fiszel dbfc8a2783 fix: fix windows build 2024-11-29 09:53:05 +01:00
Lucas Abel 0a65b90049 feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Ruben Fiszelandrubenfiszel e04a2069d9 chore(main): release 1.432.0 (#4818)
* chore(main): release 1.432.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-29 09:12:46 +01:00
Ruben Fiszel 8338efa685 skip s3 python cache on windows and pro 2024-11-29 08:35:08 +01:00
pyranota a1dd641466 feat: Parallelize uv install (#4774)
* Implement MVP of Parallel uv installation

* Implement PY_CONCURRENT_DOWNLOADS

* Remove Flock for uv installs

* Make S3 pull/push parallel

* Refactor and allow to Cancel installation

* Dont print S3 in output if disabled

* Implement better error handling

* Polishing

* More polishing

* Implement error-handler for kill_tx_2.send()

* Fix and Format prev merge

* Presubscribe to all kill_tx's

We do it now before first event could fire
Meaning no events can be lost anymore

* Early print errors and safer error handling

* Return Err if installation failed

* Final changes

* Return error instead of just printing it

* Safer the way to acquire permit

* Fix compilation error

* Remove double error logs
2024-11-29 08:21:51 +01:00
Ruben Fiszelandrubenfiszel 4fa65c1e02 chore(main): release 1.431.1 (#4816)
* chore(main): release 1.431.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-28 17:36:52 +01:00
Lucas Abel 995edad334 fix: invalid null comparison while inserting flow node (#4815) 2024-11-28 17:29:16 +01:00
Ruben Fiszelandrubenfiszel 874881d9e7 chore(main): release 1.431.0 (#4788)
* chore(main): release 1.431.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-28 17:22:24 +01:00
Ruben Fiszel 98eae8a990 fix: global timeout not set correctly on load 2024-11-28 16:02:52 +01:00
Lucas Abel 7079b030eb fix: invalid jsonb comparison while inserting flow node (#4814) 2024-11-28 14:43:54 +01:00
pyranota 1323adb345 Remove flock (#4809) 2024-11-28 12:42:25 +01:00
Ruben Fiszel db5f01371f fix apply migration to current_user 2024-11-27 19:07:31 +01:00
wendrul db4172439c fix: service logs: aggregation query + killpill on index pull + retention period for logs on s3 (#4795)
* Search across all hosts in index. killpill on index pull

* Add retention period for service logs

* Remove console log

* monitor log_file when older than biggest retention period (job / log)

* Retention period deletes service and job logs

* Add setting to monitor logs out of s3

* Prepare sqlx

* Update ee repo ref

* Prepare sqlx

* Unused variables
2024-11-27 18:57:23 +01:00
Ruben Fiszel 66fdd8723f simplify logic to check for license key on workers 2024-11-27 18:51:06 +01:00
Alexander Petric 4f83684ed6 adding sqlx migration to grant windmill permissions to db user (#4806) 2024-11-27 18:27:06 +01:00
Ruben Fiszel 838c68600a add more links to operators menu 2024-11-27 17:53:45 +01:00
Lucas Abel 4e52e54314 feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
pyranotaandRuben Fiszel 5e20824367 Fix tar prefix (#4781)
* Fix tar prefix

* Use TAR_PIP(PYTHON_311)_CACHE_DIR instead

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-27 15:54:47 +01:00
Lucas Abel 60f2e9e242 enable is_simple optimization for FlowScript (#4799)
* backend: remove some allocations in `worker_flow.rs`

* feat(backend): enable `is_simple` optimization for `FlowScript`

* fix: fix node retrieval for `flowscript` inner step
2024-11-27 10:06:19 +01:00
Lucas Abel 41a7fbb80f fix: disable flow_version_lite in dedicated workers (#4800) 2024-11-27 10:02:44 +01:00
pyranotaandellipsis-dev[bot] 41c8a22830 fix(python): Merge to site-packages packages with same name (#4793)
* (python) Initially implement merge

Iterate overall all python paths and if same folder has same name multiple times,
then merge the content and put to <job_dir>/site-packages

Solves problem with imports for some dependencies.

Default layout (/windmill/cache/):

dep==x.y.z
└── X
   └── A
dep-ext==x.y.z
└── X
   └── B

In this case python would be confused with finding B module.

This function will convert it to (/<job_id>):

site-packages
└── X
   ├── A
   └── B

This way python has no problems with finding correct module

* Better error-handling

* Optimize and fix bug

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Polish

* Add path to PathBuf

* Add & before PathBuf

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-11-26 18:24:26 +01:00
Ruben Fiszel e3ee592a2f nit 2024-11-25 23:32:14 +01:00
Lucas AbelandRuben Fiszel ce121fcf37 feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Ruben Fiszel a268bbf2cd email quotas only apply to top-level job 2024-11-25 16:20:55 +01:00
Ruben Fiszel 1dbcae953d fix: fix getting logs locally for ee without instance settings 2024-11-25 12:02:35 +01:00
Ruben Fiszel ac525ec73c improve error message for log file not found on ee 2024-11-25 11:07:48 +01:00
Ruben Fiszelandrubenfiszel 5275ad9242 chore(main): release 1.430.2 (#4782)
* chore(main): release 1.430.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-25 09:42:04 +01:00
Lucas Abel d84d578181 fix: rename job workspace as well (#4785) 2024-11-25 08:09:30 +01:00
Lucas Abel 44997cea78 backend: keep tracking minimal worker version (#4786)
Be sure to also track the min version of workers that have been started
after the min version variable has been set. e.g. All workers are up to
date and `MIN_VERSION_IS_AT_LEAST_1_427` is set to true, a worker is
then powered on with an anterior version, leading to a false positive
value. This commit fix that.
2024-11-25 08:08:16 +01:00
Ruben Fiszel e54e8ab405 fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
Ruben Fiszel 48d1f61307 improve backoff for schedules 2024-11-24 18:48:01 +01:00
Ruben Fiszel fd7bf27f27 remove rsmq 2024-11-24 11:04:54 +01:00
Ruben Fiszel 9c6502b17b fix: retry on pushing next scheduled job of schedule 2024-11-24 08:52:51 +01:00
Ruben Fiszelandrubenfiszel 91b8127fc6 chore(main): release 1.430.1 (#4780)
* chore(main): release 1.430.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-23 08:52:59 +01:00
Ruben Fiszel 6901db5412 fix: expose DISABLE_DENO_LOCK 2024-11-23 08:39:57 +01:00
Ruben Fiszelandrubenfiszel 02593badf8 chore(main): release 1.430.0 (#4764)
* chore(main): release 1.430.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-11-22 16:41:12 +01:00
wendrul 9f34712bd7 feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Patrick DobbsandRuben Fiszel 94d12185e8 Update lib.rs with python-taiga import alias (#4776)
* Update lib.rs with python-taiga import alias

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-21 23:18:38 +01:00
Alexander Petric 54b419d8ab feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
HugoCasa 3a056a1e94 fix: infer python list inner type from default if unknown inner (#4771) 2024-11-21 14:44:19 +01:00
wendrul 6742756008 indexer: Add early exit on receiving of handoff signal (#4773) 2024-11-21 14:44:08 +01:00
Lucas Abel 002f0b92ea backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
Lucas Abel 40cc9cc007 fix: garbage collect job table + delete leaked ones (#4767) 2024-11-21 09:02:24 +01:00
Ruben Fiszel eb0562ad08 fix main 2024-11-20 23:36:31 +01:00
wendrul 6e70df1db8 feat: Indexer improvements: s3 backup logic reworked, settings on the frontend (#4763)
* Add settings page + common code for indexer

* Prepare sqlx
2024-11-20 22:40:36 +01:00