Commit Graph

50 Commits

Author SHA1 Message Date
pyranota 11dd4118ce feat: add ruby support (#5939)
* implement single line pin

* make panic-safe

* use pin even if multiple modules imported withing single statement

* add repins and make imports respect pins

* keep all pins

* Allow multiple pins

* add comments + handle stuff more safely

* fix fully qualified imports

* remove ignore

* sort nested

* apply unique to output requirements list

* fix typo

* remove mut

* update sqlx

* sort imports

* sort imports

* fix formatter and format

* feat: add ruby support

* fix

* update sqlx

* add ruby to flake.nix

Signed-off-by: pyranota <pyra@duck.com>

* upgrade par_install_language_dependencies helper

rename fields + allow passing custom payload

Signed-off-by: pyranota <pyra@duck.com>

* change migrations

Signed-off-by: pyranota <pyra@duck.com>

* make full pipeline work!

Signed-off-by: pyranota <pyra@duck.com>

* do a bit fixing

* stage

Signed-off-by: pyranota <pyra@duck.com>

* add ruby in dockerfiles

Signed-off-by: pyranota <pyra@duck.com>

* implement auto-require + 'windmill/inline'

Signed-off-by: pyranota <pyra@duck.com>

* create 'windmill/inline' only if does not exist

Signed-off-by: pyranota <pyra@duck.com>

* add nsjail

Signed-off-by: pyranota <pyra@duck.com>

* update global settings

Signed-off-by: pyranota <pyra@duck.com>

* add ruby icon

* generate lockfile on deploy (but broken)

Signed-off-by: pyranota <pyra@duck.com>

* fix(frontend): display deployed script lockfile and other info

Signed-off-by: pyranota <pyra@duck.com>

* feat: add mini-wmill client

Methods:
- get_resource
- get_variable

Additionally add shortcuts on frontend

Signed-off-by: pyranota <pyra@duck.com>

* add nit warning

Signed-off-by: pyranota <pyra@duck.com>

* support private repos credentials

Signed-off-by: pyranota <pyra@duck.com>

* fix compilation error

Signed-off-by: pyranota <pyra@duck.com>

* found rustc bug + refactor universal installer

Signed-off-by: pyranota <pyra@duck.com>

* more refactor + fixes

Signed-off-by: pyranota <pyra@duck.com>

* bug fixing + polishing

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* change 'windmill/inline'

Signed-off-by: pyranota <pyra@duck.com>

* delegate env_clear to universal installer

Signed-off-by: pyranota <pyra@duck.com>

* update init script

Signed-off-by: pyranota <pyra@duck.com>

* add cli support

Signed-off-by: pyranota <pyra@duck.com>

* unhardcode frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* fix init script

Signed-off-by: pyranota <pyra@duck.com>

* make it work on ms windows

Signed-off-by: pyranota <pyra@duck.com>

* do not clean env in the end

Signed-off-by: pyranota <pyra@duck.com>

* fix windows (again)

Signed-off-by: pyranota <pyra@duck.com>

* include PROXY_ENVS (untested)

Signed-off-by: pyranota <pyra@duck.com>

* feat: self-signed certs + proxy support (tested)

Signed-off-by: pyranota <pyra@duck.com>

* flake: pin ruby to 3.4

* merge follow-ups

Signed-off-by: pyranota <pyra@duck.com>

* remove obsolete libc crate

Signed-off-by: pyranota <pyra@duck.com>

* cleanup parser

Signed-off-by: pyranota <pyra@duck.com>

* fix java ci

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix nits

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/universal_pkg_installer.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/ruby_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* add nsjail for lock stage

Signed-off-by: pyranota <pyra@duck.com>

* fix merge

Signed-off-by: pyranota <pyra@duck.com>

* Update common.rs

* regen windmill-api-client

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* fix tests again

Signed-off-by: pyranota <pyra@duck.com>

* reference published ruby parser

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 13:03:32 +00:00
dieriba 10befb995d feat: local type references parsing support for main function args (#5995)
* add base struct

* feat resolve interface and type declarion in entrypoint param's function

* nits

* fix reset dependencies

* update package

* fix handle infinite recursion

* add depth level and handle enum for referenced type

* nits

* nits

* nits

* perf

* fix

* done

* fix schema form cache inconsistency

* fix default type and nits

* remove

* update Object typ for parser

* one level ref from from parent when resolving types and use format for resource

* update cli and use resource type

* nits

* update parsers

* fix: use specific parser versions

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-07-23 10:05:32 +02:00
Diego Imbert 2ab5345e61 Assets refactor (#6217)
* Moved logic to FlowAssetsProvider

* Remove assetsMap in flow

* do not parse everything on mount + only check for missing assets fields

* add assets field in backend

* remove fallbackAccessTypes

* better structure and less queries / parsing

* Fix assets not showing when pulling raw_flow from jobs

* flow assets ctx for job run

* Fix transitive assets fetching

* Fix input args asset node

* enablePathScriptAndFlowAssets flag

* edit btn for variable

* untrack refresh

* move parseInputArgsAssets

* Assets tab in runs

* Update FlowStatusViewerInner to svelte 5 + fix asset sync bug

* avoid toast error on bad resource

* fetch res metadata for input arg asset

* Job assets viewer in run page

* r/w selector

* remove indigo badge

* store alt_access_type state in ScriptEditor

* Don't parse assets in flow script editor

* Add alt_access_type in backend

* show Read as selected by default to avoid giving the feeling of having made a decision

* keep alt_access_type when reparsing in flow raw scripts

* Remove variable asset kind, and save assets for scripts

* remove all backend asset parsing

* R/W/RW selector button nits

* fix insert into assets not saving alt access type

* support named arguments in python asset parser

* improve asset usage drawer R/W indicator

* update legacy $res: syntax

* reactivity issue

* remove last variable asset stuff

* sqlx prepare

* tooltip explainer

* deprecated variable asset nit
2025-07-17 22:15:01 +00:00
Diego Imbert 433341b295 feat: assets as a primary concept (#6125)
* assets migration

* parse assets (duckdb)

* iterate on assets

* S3 object Preview

* remove pagination

* filterText

* better occurence list

* tweak

* assets in JobPreview

* clone impl

* AssetsDetectedBadge

* improve DbManagerButton + asset dropdown button

* edit resource btn

* warning when incorrect resource

* +Resource in DuckDB

* +S3 Object editor bar

* nit fix rename

* flow asset badge

* More Generic OnChange

* Highlight assets used in modules

* Show occurence count in flow

* Better UX, avoid moving parts

* nit

* Asset nodes

* move to dedicated Asset ctx

* fix layoutNodes not handling first assetsMap

* explore asset btn in flow asset node

* correct offset

* single computeAssetNodes function

* Fix y positioning of nodes with assets

* resource editor

* write mode node (ui)

* accessType in ctx + fix insert button positioning

* right positioning when mixing read and write nodes

* right positioning when mixing R and W assets

* Better layout fix algorithm

* listAssetsByUsage and asset nodes on transitive usages

* refactor + remove linkAssets

* Refactor to allow for custom R/W modes

* AssetsDropdownButton in flow script editor

* R/W/RW selection and changes node pos in flow

* layoutNodes doesnt need recompute now

* fix wrong assumption that nodes recompute when assets change

* r/w/rw multi toggle

* MultiToggle cool animation + clearable

* rename + 1px nit

* remove mini toggle button group, use ToggleButtonGroup

* Combinator parser that detects R / W asset context

* nit fix missing flex-1

* missing order by

* better ui indication for access type

* special x offset case when only one asset node for clarity

* parse getResource in TS with swc ecma parser

* support load and write s3 detection in TS

* Python asset parser

* support wmill api calls without special $res: or s3:// syntax

* detect out of context asset uris python

* do not use access type override when not ambiguous in flow graph

* parse_assets match case in rust

* AsRef<str> refactor

* From impl

* Save flow assets

* Save script asset usages + fixes + save fallback access types

* asset sub icon

* max total asset node width to avoid overlap

* small refactor

* don't parse comments in duckdb assets

* fix assets clearing on parse error

* fix script asset save in wrong place

* load initial asset fallback access types

* support variables

* ui fixes

* Support S3Object as URI in TS client

* support new syntax in python client

* Support +S3Object in EditorBar for TS and python

* Reduce resource requests in assets page

* import windmill client when necessary

* update s3Types.d.ts

* nit fix

* Show input resources and s3 objects as assets

* improve asset icons

* DarkModeObserver refactor

* asset page tabs

* Moved resource variables and s3object pages to assets tabs

* fetch resource usages

* Get variables usages

* move assets usage dropdown to component

* Revert "move assets usage dropdown to component"

This reverts commit 622ea4ab12.

* Revert "Get variables usages"

This reverts commit b11ced4e29.

* Revert "fetch resource usages"

This reverts commit aa5187ad4b.

* Revert "Moved resource variables and s3object pages to assets tabs"

This reverts commit 4430487be4.

* Revert "asset page tabs"

This reverts commit dacc2f0da5.

* move assets usage dropdown to component

* asset icon in asset pages

* tooltip

* details

* Storage selector in S3 File Picker

* make edge less opaque

* Refactor computeAssetNodes to separate in and out nodes

* AssetsOverflowedNode

* nits

* fix assets not being parsed in flows sometimes

* show asset kind and resource_type

* ui nits

* support res:// in duckdb

* add banner for old deployments

* Fix permissionning

* fix broken disable /enable all

* assets page view permission for operators

* Disable ExploreAssetButton for operators

* asset kind as subtitle

* do not spam getResource in assets page. prob. revert fail

* update assets page on workspace change

* reload storage names on ws change

* delete assets on archive / deletion

* sqlx prepare

* missing update when updating user

* add indexes on asset

* better message

* missing loadInit: false

* dead code

* use transaction

* typo

* update package.json

* update package.json

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-11 09:56:53 +00:00
Ruben Fiszel 9c2f6a757f fix: consistency of root job propagation fixing cases where runFlow in scripts would fail 2025-06-20 20:24:53 +02:00
Ruben Fiszel f70b6f3052 update python paser 2025-06-20 18:24:17 +02:00
wendrul 6634c82e20 feat: backend arg schema validation (#5455)
* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Only cache if not preview
2025-03-19 19:41:10 +01:00
HugoCasa 90ccc3aae5 feat: add support for python list[x] (#5486) 2025-03-19 10:49:34 +01:00
HugoCasa 9736355d5f feat: add support for | None and Optional in python (#5361)
* feat: add support for | None and Optional in python

* update python parser package
2025-02-24 14:12:29 +01:00
HugoCasa 28558e674f feat: parse script for preprocessor/no_main_func on deploy (#5292)
* feat: parse script for preprocessor/no_main_func on deploy

* fix substitute for linux
2025-02-14 15:42:18 +01:00
HugoCasa b49ba59da7 fix: infer python list inner type from default if unknown inner (#4771) 2024-11-21 14:44:19 +01:00
HugoCasa ae8f29a4cb feat: http routing (#4339)
* feat: http routing

* all

* feat: improve UI

* final stuff

* fix: sqlx

* fix: nit

* fix: nits

* fix: error handler display

* fix: routes panel perms

* all

* fix: improve ability to paste from macos in vscode extension

* fix lock-write for deno

* all

* cleaning

* fix

* cli preprocessor

* nits

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-20 15:38:38 +02:00
HugoCasa 2b9b4eedc3 feat: multi sql statement with pg fix (#4134)
* Revert "Revert "feat: multi statement sql (#4104)" (#4133)"

This reverts commit c578f05e2d.

* fix: sort pg args
2024-07-26 09:23:39 +02:00
Ruben Fiszel c578f05e2d Revert "feat: multi statement sql (#4104)" (#4133)
This reverts commit 5bc0e96171.
2024-07-26 01:24:03 +02:00
HugoCasa 5bc0e96171 feat: multi statement sql (#4104)
* feat: multi statement pg

* fix: add other flavors

* feat: make pg params start at 1 and sequential

* fix: improve sql statement parsing

* add tests

* fix: allow no semi in last statement

* fix: merge conflict

* fix: minor improvement

* fix: parser version
2024-07-25 20:46:04 +02:00
Ruben Fiszel 7b724f1394 feat: dynamic select (#4110) 2024-07-22 13:53:32 +02:00
HugoCasa a79e09d65f feat: improve parsers when no main func (#3805)
* feat: improve parsers when no main func

* chore: update parser version
2024-05-23 20:36:16 +02:00
Ruben Fiszel ff3aa8ab06 feat: workflow as code v0 2024-03-04 20:19:34 +01:00
Ruben Fiszel 3420faae45 fix: parse wmill.S3Object as S3Object 2024-02-24 19:36:19 +01:00
Ruben Fiszel 45a794b1b3 feat: add support for multiselect in python 2024-01-10 13:12:20 +01:00
Guillaume Bouvignies 7441e407d4 fix: concurrency limit EE feature warning (#2505)
* fix: concurrency limit EE feature warning

* fix deprecated usage

* skip serializing empty optional
2023-10-24 17:59:16 +02:00
Ruben Fiszel affaf57250 update rust-python to 3.0 (#2499) 2023-10-24 09:26:36 +02:00
Ruben Fiszel 74254af71a migrate all jsonschema parser to wasms 2023-06-06 10:52:04 +02:00
Danny Summerlin 6a45b16564 Open Added an import exception for pysmb #1615
adding pysmb to exceptions list
2023-05-21 17:09:36 +02:00
Ruben Fiszel 811501d277 fix: rework multiselect as app component (#1599) 2023-05-18 16:28:58 +02:00
Ruben Fiszel a6c73e4d0e assign static crate for rustpython-parser 2023-05-06 12:02:52 +02:00
Ruben Fiszel 36e338c461 feat(backend): extend cached resolution for go 2023-04-09 10:58:45 +02:00
Ruben Fiszel cd8ab2b52c use requests for . imports 2023-03-30 01:07:24 +02:00
Ruben Fiszel 1ae84357e3 fix(cli): plain secrets might be undefined 2023-03-28 23:32:41 +02:00
Ruben Fiszel 0acbcb9678 fix(cli): add --plain-secrets 2023-03-28 22:46:43 +02:00
Ruben Fiszel 8aaab71161 fix python imports 2023-03-28 21:51:12 +02:00
Ruben Fiszel 904d74187e feat(backend): allow relative imports for python 2023-03-28 20:27:40 +02:00
Ruben Fiszel fd08c40893 update rustpython parser 2023-02-22 23:43:06 +01:00
Ruben Fiszel f193a8c889 fix(backend): compile issue 2023-01-31 19:50:22 +01:00
Ruben Fiszel 90324fb557 feat(python): add support for extra args in python 2023-01-31 09:52:29 +01:00
Ruben Fiszel 9030cb1862 allow whitespace in requirements override 2023-01-30 05:06:13 +01:00
Ruben Fiszel efaf1407b0 feat: add PIP_LOCAL_DEPENDENCIES 2023-01-28 03:09:47 +01:00
Ruben Fiszel 004593539f feat: add sql as a valid type in Python 2023-01-23 15:58:33 +01:00
Ruben Fiszel 421d87dfde add support for python import shopifyAPI 2023-01-20 19:52:18 +01:00
Ruben Fiszel 7940b04723 add selenium-wire exception 2023-01-15 22:48:49 +01:00
Ruben Fiszel 15aef1dd7c add psycopg-pool to psycopg default install 2023-01-13 12:16:48 +01:00
Ruben Fiszel c32e113ea7 add psycopg3 to the list of support imports 2023-01-13 07:54:05 +01:00
Ruben Fiszel 3ad5bfc192 feat: add relative imports for python scripts (#1075)
* all

* all
2023-01-09 13:49:30 +01:00
Ruben Fiszel b93a54c279 feat(python): add support for parsing resource type in python 2022-11-28 21:47:25 +01:00
Ruben Fiszel 27b638b611 feat(python): add Resource[resource_type] as a parsed parameter 2022-11-16 14:08:55 +01:00
Ruben Fiszel cd8d0e5ea6 feat: deprecate previous_result in favor of results per id 2022-11-10 01:58:52 +01:00
Ruben Fiszel af1d1191ee small UX and client fixes 2022-11-08 23:38:17 +01:00
Ruben Fiszel c097a3d039 fix(backend): capture up all lockfile issues 2022-11-02 14:04:44 +01:00
Ruben Fiszel 6ceac4280e feat(python): type is automatically inferred from default parameters 2022-10-29 12:48:50 +02:00
Kai Jellinghaus 6263e90735 restructure the entire backend layout using workspaces (#815) 2022-10-29 11:58:06 +02:00