Commit Graph

53 Commits

Author SHA1 Message Date
pyranota 0b89260540 feat(go): local go.mod (#5929)
* feat(go): local go.mod

* readability refactor

* remove dbg!

* ignore module

* remove space

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/go_executor.rs

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-06-13 10:17:31 +02:00
Ruben Fiszel 5b96bccedd feat: add aws oidc support for instance s3 storage (#5810)
* backend

* iterate

* all

* all

* all

* iterate

* revert

* all

* add tracing to get of authed client

* all

* all

* lal

* all

* update

* fix

* push

* all

* all

* revert

* frontend

* fix checks

* avoid deadlock

* safer

* fix

* fix
2025-05-25 14:03:38 +02:00
Ruben Fiszel 3f825ec77f reduce the pip resolution cache duration 2025-05-22 18:47:02 +02:00
Ruben Fiszel 63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
pyranota 59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
Ruben Fiszel 3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00
Ruben Fiszel 880db319e8 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel de1e1563ed remove go cache 2025-02-06 17:41:47 +01:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
Ruben Fiszel 5404ec9b48 fix: pass USERPROFILE on windows 2024-12-05 10:12:44 +01:00
Ruben Fiszel e11cc13c92 fix: apply NO_PROXY and HTTP_PROXY, HTTPS_PROXY more consistently 2024-11-02 13:57:30 +01:00
Alexander Petric 6eb7f8ce81 feat(worker): support workers to run natively on windows (#4446)
* minimal code change to get windmill worker on windows for bun and python + rustfmt

* adding support for powershell

* compiling error on unix

* rust linting comments

* comments hugo: PSModulePath

* comments ruben, refactor to simplify

* adding build workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* skip migration env, ee fixes

* improvements powershell

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* killing process tree in windows

* sqlx_offline

* install openssl for github windows runner

* used pre-installed openssl

* used pre-installed openssl

* build ee

* build ee

* build ee

* build ee

* adding commented out steps for artifact publishing

* build on tag matchinv v* pattern

* ren instead of mv on Windows

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix merging issue

* gate imports for windows

* fixing default cargo home path...

* fixing default cargo home path...

* comments ruben

* make pwsh default modules loading more robust on unix (#4448)

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 15:59:12 +02:00
Ruben Fiszel bd6ddc166b feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
Ruben Fiszel 57a6beaed2 fix(go): put shared directory in job dir and not go parent 2024-09-19 10:05:08 +02:00
Ruben Fiszel e1223a6f24 fix: cache hub scripts in more cases + pre-cache hub scripts deps in deno 2024-08-27 02:30:54 +02:00
Ruben Fiszel 155513cf07 fix tests 2024-08-07 17:51:41 +02:00
Ruben Fiszel 0ebdd70b35 keep bin in local cache for go even in cloud 2024-08-07 16:23:13 +02:00
Ruben Fiszel eedc39c100 feat: pre-bundle bun scripts (#4132) 2024-07-26 19:00:30 +02:00
Ruben Fiszel 710a0933bd update datafusion 2024-06-18 20:51:59 +02:00
Ruben Fiszel c8915063c4 feat: track dependency map for bun + inline script of flows 2024-06-08 16:43:48 +02:00
Ruben Fiszel fa0e7d0fde fix: fix writing binary file from global cache in go 2024-06-04 21:17:05 +02:00
Ruben Fiszel 1f64715043 feat: local typescript codebase as bundle (#3694)
* codebases

* codebases

* foo

* foo

* foo

* foo

* fix(frontend): Disable the insert button when required fields are empty strings (#3659)

* fix(frontend): use normal password mask for the sensitive fields of the resource editor

* handle super admins in password arg input

* chore(main): release 1.323.2 (#3660)

* chore(main): release 1.323.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>

* foo

* all

* s3_helpers move

* progress

* all

* progress

* progress

* progress

* progress

* codebase final

* update without ee

* sqlx

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* update

* all

* all

* all

---------

Co-authored-by: Faton Ramadani <faton.ramadani14@gmail.com>
Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2024-05-09 18:35:50 +02:00
Ruben Fiszel e2798ef7a3 rm go bin if cloud hosted 2024-04-26 13:40:52 +02:00
Ruben Fiszel bbc8bece4c fix backend tests 2024-04-26 13:27:24 +02:00
Ruben Fiszel 1ee928dd9e small fixes 2024-04-26 11:38:44 +02:00
Ruben Fiszel e27d2c0d35 feat: add distributed global cache for go 2024-04-26 11:31:21 +02:00
Ruben Fiszel 4788c393e9 avoid caching binary on cloud 2024-03-17 19:13:51 +01:00
Ruben Fiszel 958df4f62c feat: refactor logging to improve performance by order of magnitude for longer jobs 2024-03-16 17:55:34 +01:00
Guillaume Bouvignies 5866ca71f5 fix: main broken tests (#2652) 2023-11-19 22:41:07 +01:00
Ruben Fiszel 518a8396b3 feat: dedicated workers for flows (#2637)
* dw for flows

* dw for flows

* all

* sqlx

* sqlx

* dedicated workers

* more comments
2023-11-17 10:00:04 +01:00
Guillaume Bouvignies 54cc811405 feat: Add toggle to optionally mute error handler for cancelled jobs (#2567)
* fix: Fix canceled flag in completed_job table

* Add toggle to workspace settings
2023-11-06 18:24:51 +01:00
Ruben Fiszel 2ee57f3f84 feat: refactor entirely json processing in favor or rawjson to handle larger payloads (#2446)
* raw json

* raw json

* raw json

* raw json

* the big rawjson rewrite

* resolve merge conflicts

* all

* progress

* progress

* all compile

* remove mut

* remove mut

* remove mut

* arc queuedjob

* finish

* finish

* finish

* fixes

* progress

* fix result

* small fixes

* small fixes

* fix tests

* fix tests

* fix tests

* optimize eval timeout

* optimize eval timeout

* optimize eval timeout

* handle big args and result in get jobs

* improve args

* fix

* fix

* fix tests

* fix tests

* done

* add bigquery to native picker

* add bigquery

---------

Co-authored-by: gbouv <guillaume@windmill.dev>
2023-10-16 10:46:55 +02:00
Guillaume Bouvignies cfe45f7044 Better error when executable not found (#2407)
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2023-10-09 18:06:05 +02:00
HugoCasa 8319f5d0e5 fix: add reserved variables in args (#2371)
* fix: add reserved variables in args

* fix: only get job when necessary
2023-10-03 18:14:37 +02:00
Ruben Fiszel e8dd9bf1f0 feat: attempt to SIGTERM before SIGKILL for bash 2023-09-11 18:55:19 +02:00
Ruben Fiszel 0868717a73 fix: pass TZ from env to runtimes 2023-09-07 12:20:29 +02:00
Ruben Fiszel 69588cc7b1 fix: fix sqlx build 2023-09-06 12:36:54 +02:00
Ruben Fiszel 1930a473e4 feat: support partial go dependency pinning 2023-08-31 22:12:47 +02:00
Ruben Fiszel c2d766028e refactor worker-common 2023-08-20 16:44:22 +02:00
Ruben Fiszel 580c4cf6ae refactor workers 2023-08-20 15:58:43 +02:00
Ruben Fiszel 1652ac20c2 fix: set NETRC at init and not for every job 2023-08-17 12:00:19 +02:00
Ruben Fiszel 09c7b39017 fix: fix go envs passing 2023-08-15 21:49:49 +02:00
Ruben Fiszel 8ab4336c0d feat: implement binary caching for go 2023-08-08 20:54:31 +02:00
Ruben Fiszel b0660619c8 feat: add step's custom timeout 2023-07-30 20:42:24 +02:00
Ruben Fiszel 4908661800 fix typo 2023-07-13 18:30:09 +02:00
Ruben Fiszel 80051fd4c2 fix tests 2023-07-13 17:57:55 +02:00
Ruben Fiszel 4317f065f5 feat: bun support (#1800)
* feat: bun support

* feat: bun support

* add bun support

* add bun support

* add bun support

* add bun
2023-07-06 21:19:07 +02:00
Ruben Fiszel acd6a019c3 select + drawer fix + explicit hash 2023-07-05 10:01:07 +02:00
Ruben Fiszel 1d9be0c602 fix: fix go and python cache resolution conflict 2023-07-05 00:43:17 +02:00
Ruben Fiszel 8c47957a02 feat: support custom env variables (#1675)
* custom envs

* feat: custom envs

* only on non cloud

* remove unecessary imports
2023-06-05 17:42:57 +02:00