Commit Graph

476 Commits

Author SHA1 Message Date
Ruben Fiszel 669a1ff8e6 fix: improve workspace specific tags behavior 2025-01-21 18:15:34 +01:00
Ruben Fiszel e81c2c983d fix: improve workspace specific tags behavior 2025-01-21 16:24:50 +01:00
Ruben Fiszel 0d153a0a2d fix: fix workspace specific custom tags 2025-01-17 19:48:38 +01:00
HugoCasa 742d04df38 fix: preprocessor script kind option (#5081)
* fix: preprocessor script kind option

* oups

* oups
2025-01-17 14:10:39 +01:00
HugoCasa 4a68ce9ac3 fix: improve runnable permissions (#5063) 2025-01-15 18:25:12 +01:00
HugoCasa bc3d93b935 fix(backend): better deleted user error (#5060)
* fix(backend): better deleted user error

* nit
2025-01-15 17:02:10 +01:00
wendrul 77d825540f feat: add oracle db support on ee (#5062) 2025-01-14 21:21:24 +01:00
HugoCasa 04fbda28e8 feat: on behalf of (#5058)
* feat: on behalf of

* fix

* revert mistakke

* temp ee ref

* renaming + CLI

* sqlx

* nits + ee ref
2025-01-14 18:32:25 +01:00
Lucas Abel f316045483 cache: clear the cache between tests (#4958)
otherwise tests aren't correctly isolated
2024-12-20 20:03:56 +09:00
Lucas Abel 83d24cb9ab cache: better cache system (#4951)
* cache: consider source as untrusted

* cache: better scoped cache

* cache: non-lazy flow value parse error

* fixup! cache: consider source as untrusted

* fixup! cache: consider source as untrusted

* fixup! cache: better scoped cache
2024-12-20 11:19:57 +01:00
Ruben Fiszel 13be0cd1c8 fix: fix relative imports in cached flow scripts 2024-12-18 11:34:07 +01:00
Lucas Abel a040513791 cache: disable caching for tests (#4943)
As `#[sqlx::test]` spawn a database per test, the cache
could yield unexpected results
2024-12-18 09:57:31 +01:00
Lucas Abel 51b74e713c cache: improve error reporting (#4940) 2024-12-17 19:44:51 +00:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
Ruben Fiszel 34392ef66f split workspace and auth logic 2024-12-15 13:13:09 +01:00
Ruben Fiszel c320862c51 make mail-send optional 2024-12-15 10:31:44 +01:00
Ruben Fiszel 51c0373b4c feature gates more feature for faster compilation (#4928)
* cleanup

* cleanup

* all

* all

* all

* all
2024-12-15 00:00:37 +01:00
Lucas Abel 74a89552c7 cleanup is_flow usage and impl (#4923) 2024-12-13 18:08:01 +01:00
wendrul c85d2a4957 feat: add C# support (#4908)
* Add csharp stuff and merge after the rewrite

    * Add csharp, boilerplate + minimal execution
    * Add initial parser
    * Frontend + wasm export of the tree sitter parser
    * Arg spread and use cache

* More types and req parsing

* Add logo

* Parse async and void to fit the wrapper to the main sig

* Csharp editor resource and var buttons

* Lockfile generation

* Add default parsing using serde_json::from_str

* Update init code for c#

* Nsjail for c#

* Remove c_variadic feature

* Remove unused imports

* Add c_variadic only for wasm32 assembly

* Fix c# test

* Add dotnet to the images

* Update dockerfiles

* Update dockerfile again

* Touch Dockerfile

* npm install c# parser

* Add opt mount in nsjail

* Update ee repo ref

* Add csharp to backend image for tests

* Add setting to set nuget package

* Windows compatibility

* Disable csharp test

* ADd feature flag
2024-12-13 00:29:01 +01:00
Lucas Abel af5cca1b00 feat(cache): re-work job results cache (#4898) 2024-12-11 12:01:48 +01:00
Ruben Fiszel 1639bb2b35 improve debug logs 2024-12-10 12:35:56 +01:00
Ruben Fiszel 2c5d07f3d9 feat: docker runtime 2024-12-09 18:02:18 +01:00
Ruben Fiszel 465ec09f96 improve otel 2024-12-08 11:06:35 +01:00
Ruben Fiszel fccc889e14 add hostname to logs 2024-12-08 09:49:51 +01:00
Ruben Fiszel 5bd0d556dd add tls to tonic 2024-12-08 00:34:46 +01:00
Ruben Fiszel 6d047449e2 feat: add otlp support (#4869)
* all

* all

* add otel

* add otel

* update docker-image

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update

* update
2024-12-07 16:29:20 +01:00
Lucas Abel 2bc4934c4f feat: add db storage for app inline scripts (#4837) 2024-12-06 13:37:45 +01:00
Lucas Abel 3c4408e3db feat(cache): refurbish fs backed cache (#4863)
* feat(cache): refurbish fs backed cache

* add `cached` exemple
2024-12-06 12:26:36 +01:00
Ruben Fiszel e5efefc7e8 improve retries max value handling 2024-12-06 01:03:46 +01:00
Lucas Abel c443c2b3a8 nit: format cache.rs (#4845) 2024-12-05 11:10:30 +01:00
Ruben Fiszel db62a2c53c simplify warn_after_seconds 2024-12-04 18:28:28 +01:00
Ruben Fiszel bb9dd7d368 simplify warn_after_seconds 2024-12-04 18:24:31 +01:00
Ruben Fiszel b534ef6074 improve transparency for slow queries 2024-12-04 17:30:58 +01:00
wendrul 088c666a7f feat: indexer extra settings + parallel downloads + many improvements (#4822)
* Nits on the service logs page

* Show all the hosts returned by query + sumOtherDocCount warning

* Remove from index endpoint

* Fix tests

* Prepare sqlx
2024-11-29 19:14:05 +01:00
Lucas Abel 3fbb2bfc8a feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
wendrul 09fe535bdc fix: service logs: aggregation query + killpill on index pull + retention period for logs on s3 (#4795)
* Search across all hosts in index. killpill on index pull

* Add retention period for service logs

* Remove console log

* monitor log_file when older than biggest retention period (job / log)

* Retention period deletes service and job logs

* Add setting to monitor logs out of s3

* Prepare sqlx

* Update ee repo ref

* Prepare sqlx

* Unused variables
2024-11-27 18:57:23 +01:00
Lucas Abel 2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Lucas Abel d86ea30221 enable is_simple optimization for FlowScript (#4799)
* backend: remove some allocations in `worker_flow.rs`

* feat(backend): enable `is_simple` optimization for `FlowScript`

* fix: fix node retrieval for `flowscript` inner step
2024-11-27 10:06:19 +01:00
pyranota e876ae4f42 fix(python): Merge to site-packages packages with same name (#4793)
* (python) Initially implement merge

Iterate overall all python paths and if same folder has same name multiple times,
then merge the content and put to <job_dir>/site-packages

Solves problem with imports for some dependencies.

Default layout (/windmill/cache/):

dep==x.y.z
└── X
   └── A
dep-ext==x.y.z
└── X
   └── B

In this case python would be confused with finding B module.

This function will convert it to (/<job_id>):

site-packages
└── X
   ├── A
   └── B

This way python has no problems with finding correct module

* Better error-handling

* Optimize and fix bug

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/python_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Polish

* Add path to PathBuf

* Add & before PathBuf

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-11-26 18:24:26 +01:00
Lucas Abel e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Lucas Abel 39b902c118 backend: keep tracking minimal worker version (#4786)
Be sure to also track the min version of workers that have been started
after the min version variable has been set. e.g. All workers are up to
date and `MIN_VERSION_IS_AT_LEAST_1_427` is set to true, a worker is
then powered on with an anterior version, leading to a false positive
value. This commit fix that.
2024-11-25 08:08:16 +01:00
wendrul 4facf3ca3e feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Alexander Petric 8909bea935 feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
Lucas Abel e6c7fe7027 backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
wendrul 8f198ba68c feat: Indexer improvements: s3 backup logic reworked, settings on the frontend (#4763)
* Add settings page + common code for indexer

* Prepare sqlx
2024-11-20 22:40:36 +01:00
Ruben Fiszel 45ce5ff8c6 add FORCE_HOSTNAME env var 2024-11-20 16:07:24 +01:00
HugoCasa 6924370f11 refactor sql result formatting (#4760) 2024-11-20 14:53:39 +01:00
HugoCasa b3a7cb0583 feat: pdf file preview (#4753)
also fixes s3 missing content-length header
2024-11-19 19:09:20 +01:00
Lucas Abel 97457a5679 feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00
wendrul 4d9ec909d1 fix: playbook files permission mode incompatible with Windows (#4740) 2024-11-18 22:14:19 +01:00