Commit Graph

919 Commits

Author SHA1 Message Date
HugoCasa b70ca6e3ed skip step better UI (#4465) 2024-10-01 12:28:46 +02:00
HugoCasa 0df169e3f9 feat: flow step skipping (#4461) 2024-09-30 16:30:59 +02:00
Ruben Fiszel fcc51bc5a4 optimize flow update parallel queries 2024-09-29 13:17:11 +02:00
Ruben Fiszel 0df35e5787 add urandom to nsjail 2024-09-29 12:31:55 +02:00
Ruben Fiszel 004bcb2b07 quote req and target for nsjail python download 2024-09-29 12:13:54 +02:00
Ruben Fiszel 8fb3637183 optimize early stop query 2024-09-29 01:19:06 +02:00
Ruben Fiszel 5acfc83521 optimize skip failure query 2024-09-29 00:38:43 +02:00
Ruben Fiszel 30d9c40193 optimize more queries 2024-09-28 17:34:37 +02:00
Ruben Fiszel 363e5e1727 improve suspend_first frequency behavior 2024-09-28 15:50:12 +02:00
Ruben Fiszel b5e226b977 fix: improve suspend_first behavior and frequency 2024-09-28 15:46:50 +02:00
HugoCasa 0b9ec83036 fix: allow preprocessor to write to args.json on nsjail (#4455) 2024-09-27 21:12:38 +02:00
Ruben Fiszel cfaea38e92 improve benchmarking tools (#4450) 2024-09-27 10:48:04 +02:00
Ruben Fiszel 9f91b1995a feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
HugoCasa 3ce5587faa feat: add return_last_result annotation to sql (#4443) 2024-09-25 18:11:11 +02:00
wendrul 398a09b741 feat: add static variable and resources support to ansible (#4435)
* Add static variable and resources to ansible

* Add wasm parser to cli

* Fix pulling of resources, rs and playbooks

* Remove unused import, add publish pkg-yaml

* Update package.json

* Update generate-metadata

* Fix null fields being serialized in script metadata

* npm install
2024-09-25 15:05:53 +02:00
HugoCasa 590321fd3c feat: add tag filtering to external JWT authentication (#4425)
* feat: tag filtering jwt ext auth

* move tags to scopes

* fix symlink

* update ee ref
2024-09-25 11:21:19 +02:00
Ruben Fiszel 93fa6a0e97 improve init script error handling 2024-09-24 19:27:00 +02:00
HugoCasa 61f27acbd5 fix: time handling in bun and deno (#4423)
* fix: time handling in bun and deno

* put dates conversion in try catch
2024-09-23 18:00:36 +02:00
wendrul a500994cc2 fix: various improvements for ansible (#4419)
* Improve error messages, fix dependency job issue

* Add nsjail support for ansible

* Remove invalid comment syntax

* Change nsjail config and ansible controller config to be ephemeral

* Move ansible cfg to later to overwrite possible files named ansible.cfg

* Fix ansible.cfg env var

* Fix tmp dir issue
2024-09-23 16:44:37 +02:00
HugoCasa 304dac3447 feat: http routing (#4339)
* feat: http routing

* all

* feat: improve UI

* final stuff

* fix: sqlx

* fix: nit

* fix: nits

* fix: error handler display

* fix: routes panel perms

* all

* fix: improve ability to paste from macos in vscode extension

* fix lock-write for deno

* all

* cleaning

* fix

* cli preprocessor

* nits

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-20 15:38:38 +02:00
Ruben Fiszel 69b8754aef fix(bun): never pre-bundle BASE_URL env variable 2024-09-20 09:48:00 +02:00
Ruben Fiszel 623ece8c67 fix(go): put shared directory in job dir and not go parent 2024-09-19 10:05:08 +02:00
wendrul 6855b8da9a feat: ansible playbook support (#4399)
* Ansible execution and parsing

* Working collections and pip dependencies

* Remove unused vars

* File resources logic for ansible

* Make frontend for ansible and resource file

* Format file

* Change naming of file resource and inventory fields

* Add autocomplete for file extension resource creation

* Add endpoint to list file formats for resource types

* Add CLI functionality to pull/push file resources

* Add beta tag to ansible

* Add Full image dockerfiles containing ansible

* Prepare sqlx

* Update cargo.lock

* Update ansible path

* Remove unused imports

* Add back import removed by rust-analyzer

* Improve ansible init code

* Prepare sqlx

* Change dockerfile to make the full windmill image

* Remove old dockerfile

* Improve autocomplete file resource extension select

* Remove comment

* Validate file extension

* Add icons to text file resources

* Remove editability of file resource types

* Remove unused import

* Missing space

* Add yaml parser
2024-09-19 00:21:15 +02:00
Ruben Fiszel 37d152feeb fix: postgres scripts that take longer than 20s do not timeout anymore 2024-09-18 14:32:16 +02:00
pyranota d6d4756b7a feat: Allow setProgress and getProgress from within the script (#4400)
* Allow setting progress explicitly from script body.

This feature exposes:
 * `getProgress`
 * `setProgress`
 * `incProgress`

API in TypeScript client (python is coming soon).

NOTE: Progress cannot be out of range 0..100 and cannot decrease.

With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.

For optimization reasons, jobs start to ask for progress only after N-seconds of execution.

* feat: Add `shell.nix`

If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies

NOTE: You will still need docker

* feat: Add `dev.nu` to typescript client

Little helper function, allowing developer to work on ts client easier.

To use:
  `./dev.nu watch`

Now add import of windmill in body of your script and `//nobundle` on top of the file

Edit ts client in your favourite editor and hit save. Script will do the rest.

* Cleanup files

* Fix: Failed to deserialize query string: missing field `get_progress`

* perf: Implement non-naive polling mechanism for getting job progress

* Add independant delay for getProgress

Problem in `TestJobLoader`:

There should be 2 delays:

    One until we find our first progress (every 5s)
    Once we found our first progress, we can do it every second

* nit: Use `query_scalar!` instead of `query_as`

* Fix: Sql error, no rows returned by a query that expected to return at least one row

* refactor: Remove global CSS for JobProgressBar

* Change UI for progress of flow subjobs

* Replace `Step 1` with `Running` in ProgressBar for individual jobs

* Remove `incProgress`

incProgress is not very usefull and error-prone

* perf: Set metric only for jobs that are actually using it

(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)

* Offload registering progress from clients to server

* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`

Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.

Could be used by SDK.

* Add `Error::MetricNotFound` for better error handling

* Fix: Make `JobProgressBar` display in red when failed

* Add persistant progress bar

Now you can reload the page after job is done and progress will be still there

* Allow succeeded individual job's progress bar stick to 100%

* Add python support 

* nit: Remove usage of undefined variable in python-client

* Add `async` in ts client (for error handling)

* nit(frontend): Remove unused import

* Dont load JobProgressBar when it is not needed

* nit: npm check fix

* cargo sqlx prepare

* fix sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-18 01:16:19 +02:00
Ruben Fiszel f7454e621c fix lock-write for deno 2024-09-18 00:57:53 +02:00
Ruben Fiszel 277d085dbe fix(deno): replace lock-write with frozen=false 2024-09-17 08:33:33 +02:00
Ruben Fiszel d57b139df2 fix: cannot create duplicate apps and raw_apps 2024-09-16 21:15:29 +02:00
HugoCasa de78f6c192 feat: failed key renewal alert + renew on start if no recent renewal (#4387)
* feat: failed key renewal alert + renew on start if no recent renewal

* Update ee-repo-ref.txt
2024-09-16 16:54:35 +02:00
Ruben Fiszel 396258f637 fix: improve first time setup experience 2024-09-15 21:02:36 +02:00
Ruben Fiszel 4dc9ca7f4e fix: timeout in pg executor on postgresql connection after 20s 2024-09-12 18:49:43 +02:00
Ruben Fiszel 0e64380473 fix: add tag_override for script in flows 2024-09-11 23:02:45 +02:00
Ruben Fiszel 042a2bf917 fix: graceful worker exits for same worker jobs (#4371)
* all

* fix compile
2024-09-11 18:32:08 +02:00
Ruben Fiszel 4720237091 fix: handle better same_worker flow monitor 2024-09-11 01:29:40 +02:00
Ruben Fiszel 4a6c3c8cc5 fix: bun scripts cached in docker image have their dependencies pre-loaded 2024-09-09 13:52:12 +02:00
Ruben Fiszel 6f0af6b8c6 synchronize hub script is moved to new windmill cli 2024-09-07 19:05:34 +02:00
Ruben Fiszel 50f32c4e12 fix: nativets correct transform resources in args 2024-09-06 20:41:42 +02:00
Ruben Fiszel cdd7349f67 fix: improve dependency map to handle recusrive loops + handle better flow relative imports 2024-09-02 21:38:47 +02:00
Ruben Fiszel 64dbf6fce4 refactor main tokio select loop (#4312) 2024-09-01 21:58:37 +02:00
Ruben Fiszel b05717b820 nits job still running format 2024-09-01 12:28:03 +02:00
Ruben Fiszel 3e774fe51d nits log formats for delayed job 2024-09-01 12:22:14 +02:00
Ruben Fiszel 2fe48df720 feat: service logs (#4244) 2024-08-31 14:57:35 +02:00
wendrul a2beed9d73 feat: add rust (#4253)
* Add rust on frontend

* Add parser for rust

* Handle rust job execution

* Main has to return a result that is serializable

* Update parser: parse many rust types

* Rust parser dependencies

* Frontend wasm parser

* Add windmill parser rust to windmill parser wasm (temporarily)

* Add rust as a tag on worker__default

* Change init code for rust

* Cleanup rust_executor.rs

* Remove tree-sitter dep and fix unused imports

* Add lockfile and build logic

* Add cargo and rustup to windmill image

* Fix env var to work on docker image and dev env

* modify package.json

* Deps after parser publish

* Add stashed migration

* Unify rust versions

* Add error message when php or cargo are not present to use another image

* Error message conditionally on feature flag

* all ci/cd changes

* all ci/cd changes

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-30 17:01:04 +02:00
Proesmans Bert 2ec4e95b8f Take interpreter paths from environment (bash executor) (#4304)
* Take bash exe path from environment

- Changed BIN_BASH type to owned String
- Replaced hardcoded path with BIN_BASH
- Updated references of BIN_BASH to match new String type

* Replaced hardcoded pwsh call with POWERSHELL_PATH
2024-08-30 14:56:32 +02:00
HugoCasa 84ce3d819d feat: show last job instead of current job on workers page (#4293)
* feat: show last job instead of current job on workers page

* fix: keep old sql column name
2024-08-27 22:04:02 +02:00
Ruben Fiszel 16465e47c8 fix: cache hub scripts in more cases + pre-cache hub scripts deps in deno 2024-08-27 02:30:54 +02:00
Ruben Fiszel 1ec45e5ad6 feat: add vim support for monaco/webeditor 2024-08-26 22:31:23 +02:00
HugoCasa bcde2e62d7 feat: improve early stop (#4257) 2024-08-21 00:47:34 +02:00
Ruben Fiszel 1a0e32b40b fix(bun): disable large transpiling cache 2024-08-20 08:49:51 +02:00
HugoCasa 99f7828ebb feat: cache common hub scripts in image (#4249)
* feat: cache common hub scripts in image

* Delete backend/src/hubPaths.json

* fix: missing file

* fix: dockerfile

* fix: cache by path

* fix

* fix

* precreate cache folder
2024-08-16 17:30:44 +02:00