Commit Graph

77 Commits

Author SHA1 Message Date
pyranota ed61d97700 feat(rust): shared build directory (#5610)
* feat: rust incremental compilation

* update dockerfiles

* fix compilation error

* add windows flags

* init

* polishing

* update

* return mount-point

* Update backend/windmill-worker/src/rust_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* share worker dir if nsjail off

* final refactor

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-29 22:11:35 +02:00
Ruben Fiszel bb0ed2b112 update rust to 1.86.0 in build and full images 2025-05-09 09:43:24 +02:00
pyranota 59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
pyranota a3faea16e7 feat: add nu-lang support (#5217)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version
2025-03-20 16:45:21 +01:00
Ruben Fiszel 8e0963eec8 fix: update bun to 1.2.4 2025-03-12 17:10:40 +01:00
Ruben Fiszel 692de3e628 update bun to 1.2.3 2025-02-23 23:11:07 +01:00
Ruben Fiszel c81a13cdac bun to 1.2.2 while 1.2.3 is not released 2025-02-22 16:31:00 +01:00
Ruben Fiszel 4e477d1f58 fix: update bun t.1.43->1.2.3 2025-02-22 09:23:33 +01:00
Ruben Fiszel 6b31a6dee5 fix ansible for nsjail 2025-02-21 17:06:15 +01:00
Ruben Fiszel 3188bee46e fix: disable toggle is more consistently applied on arg inputs 2025-02-21 14:07:44 +01:00
Ruben Fiszel 03903d06bf nit ansible 2025-02-21 11:22:58 +01:00
Ruben Fiszel 22ff20b6ab nit ansible 2025-02-21 10:12:46 +01:00
Ruben Fiszel 420e65c71c nit ansible galaxy 2025-02-21 10:03:51 +01:00
Ruben Fiszel 90ceb6e4fd uv tool install ansible 2025-02-21 09:50:57 +01:00
Ruben Fiszel 5ad473e12e install ansible with uv pip 2025-02-21 09:44:40 +01:00
wendrul 6bfaa8c034 add missing TARGETPLATFORM arg on dockerfile (#5166) 2025-01-28 19:40:21 +01:00
wendrul a1579c1654 fix: oracledb tag for native workers + client not working on arm systems (#5162)
* New migration for oracledb

* fix oracledb on ARM64 systems
2025-01-28 17:57:40 +01:00
Alexander Petric 6f469ac9e0 rhel image build workaround (#5091)
* RHEL Dockerfile / no chef

* Update Dockerfile

* Update Dockerfile

* Update build-publish-rh-image.yml
2025-01-17 19:50:23 +01:00
Ruben Fiszel 797f46c680 chore: update bun to 1.43.0 and loaded from tsx to ts 2025-01-16 18:26:03 +01:00
wendrul 09dda483f4 fix: Oracle Database client libraries (#5072) 2025-01-15 23:54:30 +01:00
wendrul 77d825540f feat: add oracle db support on ee (#5062) 2025-01-14 21:21:24 +01:00
Tsvetomir Bonev 3a5b670fca chore: update rust to 1.81.0 (#5037) 2025-01-09 18:21:59 +01:00
pyranota 91257e5e2f Bump uv version to 0.5.15 (#5030) 2025-01-09 11:54:28 +01:00
Ruben Fiszel c4fdd2297e fix: update bun from 1.1.38 to 1.1.40 2024-12-19 01:38:35 +01:00
wendrul eb3ed7a2c7 fix: add DOTNET_ROOT env variable (#4921) 2024-12-13 17:54:56 +01:00
wendrul afac8a73f0 fix: c#: nsjail image, default langs and feature cage imports (#4917)
* fix: nsjail image, default langs and feature cage csharp imports

* Fix unused imports with csharp flag
2024-12-13 09:36:17 +01:00
wendrul c85d2a4957 feat: add C# support (#4908)
* Add csharp stuff and merge after the rewrite

    * Add csharp, boilerplate + minimal execution
    * Add initial parser
    * Frontend + wasm export of the tree sitter parser
    * Arg spread and use cache

* More types and req parsing

* Add logo

* Parse async and void to fit the wrapper to the main sig

* Csharp editor resource and var buttons

* Lockfile generation

* Add default parsing using serde_json::from_str

* Update init code for c#

* Nsjail for c#

* Remove c_variadic feature

* Remove unused imports

* Add c_variadic only for wasm32 assembly

* Fix c# test

* Add dotnet to the images

* Update dockerfiles

* Update dockerfile again

* Touch Dockerfile

* npm install c# parser

* Add opt mount in nsjail

* Update ee repo ref

* Add csharp to backend image for tests

* Add setting to set nuget package

* Windows compatibility

* Disable csharp test

* ADd feature flag
2024-12-13 00:29:01 +01:00
Ruben Fiszel 45d4fc2de7 bump deno to 2.1.2 and bun to 1.1.38 2024-12-06 01:24:36 +01:00
Ruben Fiszel 789b4f6442 bump deno to 2.0.4 and bun to 1.1.34 2024-11-06 00:38:37 +01:00
Ruben Fiszel 2cd46cc55f fix(bash): correctly propagate sigterm for cancelled bash scripts 2024-10-24 14:42:37 +02:00
Ruben Fiszel 1ae033841e fix: update bun to 1.1.32 (#4568)
* fix: update bun to 1.1.32

* Update DockerfileSlimEe

* Update Dockerfile
2024-10-22 20:08:26 +02:00
Ruben Fiszel d7b83274db fix: update bun to 1.1.31 and deno to 2.0.2 2024-10-21 11:29:14 +02:00
Alexander Petric acc5aa86da rhel9 build workflow (#4528)
* dockerfile for rhel9

* attach rh to release page workflow

* changing name

* updating new workflow

* updating new workflow

* pushing new workflow

* trigger workflow

* trigger workflow

* trigger workflow

* trigger workflow

* finished workflow

* ellipsis comment
2024-10-16 14:36:36 +02:00
Ruben Fiszel 00e7f0aedc usr local bin uv 2024-10-03 22:48:09 +02:00
pyranota 1e7a37ac8b feat: Replace pip-compile with uv (#4460)
* Update `shell.nix`

- Replace pip-compile with uv packages
- Pin rust version
- Add var to trigger windmill print more info in stdout

* Replace `pip-compile` with `uv` (dirty + untested)

* Fix arguments passed to uv

Some of the flags are included by default in UV and can be safely removed:
- --resolver=backtracking
- --no-emit-index-url

Also uv does not support `--pip-args` and suggests to directly pass args to uv.

* Remove extra `dbg!`

* Replace 'pip-compile' with 'uv' in Dockerfile

* Add fallback option to `pip-compile` (Disabled)

* Add `uv` to `docker/DockerfileSlim*`

* Add `get_annotation_python` and rename `get_annotation` to `get_annotation_ts`

* Add option to fallback to pip-compile

Put `# no_uv` on top the file for specific python script

Or set `USE_PIP_COMPILE` variable to `true`

* Put back `pip-tools` into shell.nix

* Make sure lockfile resolves again if `#no_uv` used

Add #no_uv to the end of requirements (requirements.in)
That way if something breaks for customer, then they put #no_uv and new lockfile will be resolved

* Put `pip install pip-tools` in original spot

* Fix compilation error

* Fix EE compilation error

error[E0658]: attributes on expressions are experimental
   --> windmill-worker/src/python_executor.rs:144:5
    |
144 |     #[cfg(feature = "enterprise")]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information

* Add `no_cache` annotation

Will force recalculation of lockfile
And block uv from using cached values

* Target uv cache to /tmp/windmill/cache

* Prohibit uv from managing python

* Add uv to DockerfileBackendTests

* Pin uv version to 0.4.18 in Dockerfiles

* Dont put `#no_uv` in requirements.in

Instead postfix hash for requirements.in with `-no_uv`

* Push Warning to logs if fallbacked to pip-compile

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 16:00:35 +02:00
Ruben Fiszel 768d4b752e fix: update pip to 24.2 and python 3.11.8->3.11.10 2024-09-23 14:01:31 +02:00
Ruben Fiszel 3804c1085f add ansible to default tags 2024-09-19 00:47:30 +02:00
wendrul b0cb1e0b27 feat: ansible playbook support (#4399)
* Ansible execution and parsing

* Working collections and pip dependencies

* Remove unused vars

* File resources logic for ansible

* Make frontend for ansible and resource file

* Format file

* Change naming of file resource and inventory fields

* Add autocomplete for file extension resource creation

* Add endpoint to list file formats for resource types

* Add CLI functionality to pull/push file resources

* Add beta tag to ansible

* Add Full image dockerfiles containing ansible

* Prepare sqlx

* Update cargo.lock

* Update ansible path

* Remove unused imports

* Add back import removed by rust-analyzer

* Improve ansible init code

* Prepare sqlx

* Change dockerfile to make the full windmill image

* Remove old dockerfile

* Improve autocomplete file resource extension select

* Remove comment

* Validate file extension

* Add icons to text file resources

* Remove editability of file resource types

* Remove unused import

* Missing space

* Add yaml parser
2024-09-19 00:21:15 +02:00
Ruben Fiszel ae6103569a fix: update bun to 1.1.27 2024-09-17 10:27:51 +02:00
Tsvetomir Bonev ad5363375d chore: build full docker image for arm64 (#4391)
* chore: build `windmill-full` for `arm64`

* fix: use non-enterprise docker image as base
2024-09-14 18:30:31 +02:00
Ruben Fiszel 6f72d99fef migrate dockerfile CLI to node 2024-09-07 21:34:14 +02:00
wendrul 91d328b132 feat: add rust (#4253)
* Add rust on frontend

* Add parser for rust

* Handle rust job execution

* Main has to return a result that is serializable

* Update parser: parse many rust types

* Rust parser dependencies

* Frontend wasm parser

* Add windmill parser rust to windmill parser wasm (temporarily)

* Add rust as a tag on worker__default

* Change init code for rust

* Cleanup rust_executor.rs

* Remove tree-sitter dep and fix unused imports

* Add lockfile and build logic

* Add cargo and rustup to windmill image

* Fix env var to work on docker image and dev env

* modify package.json

* Deps after parser publish

* Add stashed migration

* Unify rust versions

* Add error message when php or cargo are not present to use another image

* Error message conditionally on feature flag

* all ci/cd changes

* all ci/cd changes

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-30 17:01:04 +02:00
HugoCasa 904b5b4ea1 feat: cache common hub scripts in image (#4249)
* feat: cache common hub scripts in image

* Delete backend/src/hubPaths.json

* fix: missing file

* fix: dockerfile

* fix: cache by path

* fix

* fix

* precreate cache folder
2024-08-16 17:30:44 +02:00
HugoCasa 357ac9124f feat: caddy with l4 image (#4178) 2024-08-04 23:17:24 +02:00
HugoCasa 49f1e276c7 rhel9 binaries (#4086)
* feat: rhl9 img test

* fix: action

* fix: action

* fix: build

* don't use rustup

* fix: rustfmt

* fix: second apt-get

* fix

* fix: try xmlsec1 no-dev

* fix: remove scache

* fix: 1.75 rust support

* fix: rust 1.75

* fix: xmlsec1

* fix: nit

* fix: missing async recursion

* fix: missing dep + unregister from RH

* fix: add xmlsec1 openssl

* fix: check binary is running

* add link

* feat: clean rhel files

* fix: lock
2024-07-16 16:50:51 +02:00
Ruben Fiszel b5a435b387 add back deno in slim image 2024-06-27 23:14:30 +02:00
Ruben Fiszel 24679b291b build slim image 2024-06-26 19:36:35 +02:00
Ruben Fiszel 0a8dce7241 feat: add wmill cli image 2024-06-26 16:58:09 +02:00
Ruben Fiszel fc08025643 small toggle fix for list & objects 2024-06-26 16:58:09 +02:00
Ruben Fiszel 686bdd211e chore: split out nsjail from public images 2024-04-03 23:53:38 +02:00