Commit Graph

66 Commits

Author SHA1 Message Date
pyranota f32159f412 fix: do not split main.py on 'def main' (#5895)
doesn't work well in edge cases:

```
@schedule(
    bla=True
)
def main(x: str):
    return x
```

```
error_message = """
Please ensure you have defined main(username, password) correctly.
The function def main( should be at the end.
"""

def main(x: str):
    return x
```

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-06-20 16:40:30 +02:00
pyranota 88ab1a5136 Use regex to split input to python import parser (#6007) 2025-06-20 13:24:27 +02:00
pyranota 06e61ee958 Add fake main function to fix Python AST parsing (#6006)
The changes ensure the Python AST parser can handle code snippets where the main
function was previously removed, by appending a dummy main function.
2025-06-20 09:28:30 +02:00
pyranota 26222539e6 feat(python): inline script metadata (PEP 723) (#5712)
* make resolver

* more updates

* fix build

* fix raw_dependencies job type

* compat with http agent workers

* refactor

* rename

* more refactor

* cleanup

* more tests

* fix s3

* small fixes

* more fixing

* fix endpoint

* nit: update comment

* update ee ref

* update ee ref

* update ee ref

* implement safer `list_available_python_versions`

* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix build

* update ee ref

* ignore versions <=3.9

* fix windows build

* correct versions filter

* fix windows build (this time for real)

* inject error to debug CI

* update CI

* undo debug of CI

* fix tests

* remove outdated comment

* update ee repo ref

* Update ee-repo-ref.txt

* Update backend/parsers/windmill-parser-py-imports/src/lib.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update InstanceSetting.svelte

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
Eugene Chernyavsky 6b2ba3f261 google-genai and google.generativeai requirement from import (#5699) 2025-05-02 19:50:20 +02:00
pyranota 4c660a8869 nit(python): map azure-storage-file-datalake (#5682)
* feat(python): map azure-storage-file-datalake

* add comma
2025-04-30 10:04:10 +02:00
Maxime Thiebaut fbdaf28fb5 nit(python): add virustotal to imports mapping (#5683) 2025-04-29 22:55:58 +02:00
pyranota 75ee10ea65 fix integration tests failing (#5681)
* fix: make `#(extra_)requirements:` work better with pins

* fix(python): fix integration tests failing
2025-04-29 20:30:34 +02:00
pyranota 1ab41603f4 fix: make #(extra_)requirements: work better with pins (#5680) 2025-04-29 19:59:51 +02:00
Ruben Fiszel 193186ae75 nits 2025-04-29 15:43:30 +02:00
Ruben Fiszel f662cf5d75 fix(python): handle better relative imports with requirements or extra_requirements 2025-04-29 15:23:44 +02:00
pyranota 0b6d017fed feat(python): per import requirement pin (#5520)
* implement single line pin

* make panic-safe

* use pin even if multiple modules imported withing single statement

* add repins and make imports respect pins

* keep all pins

* Allow multiple pins

* add comments + handle stuff more safely

* fix fully qualified imports

* remove ignore

* sort nested

* apply unique to output requirements list

* fix typo

* remove mut

* update sqlx

* sort imports

* sort imports

* fix formatter and format

* refactor

* fix comptime error

* write tests

* perf: do not capture if string is empty
2025-04-11 21:31:51 +00:00
pyranota a0921abfb3 nit(python): add azure keyvault and storage to imports mapping (#5517)
* feat(python): fully qualified imports mapping

* make contributor-friendly

* use more intuitive sign

* map the most of the google and azure

* nit(python): add azure keyvault and storage to imports mapping
2025-03-27 00:11:28 +01:00
pyranota 1a5566b8c2 feat(python): fully qualified imports mapping (#5511)
* feat(python): fully qualified imports mapping

* make contributor-friendly

* use more intuitive sign

* map the most of the google and azure
2025-03-26 23:42:21 +01:00
pyranota a3faea16e7 feat: add nu-lang support (#5217)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version
2025-03-20 16:45:21 +01:00
Ruben Fiszel e71882d0a0 add zlib to import parser exception 2025-02-21 21:26:14 +01:00
Ruben Fiszel b5fa5f4dd2 add docx => python-docx python exception 2025-02-10 14:45:29 +01:00
Ruben Fiszel b74d3fd6e0 fix: pin malachite version 2025-01-31 14:39:18 +01:00
Arjun Dhawan 343ff94f3c add bigquery to the list of python depenency exceptions (#5173) 2025-01-30 13:12:34 +01:00
pyranota e47dd697f9 feat(python): Multiple runtime versions (#4579)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Remove verbose mode and enable link-mode=copy

* Granural migration of lockfiles

Before i realized we dont need it :)

* Initial draft (not-working)

* Add fallback

* Fix bug preventing uv from installing deps

'\n' - Love it

* Add verbosity indicator

* Iterate on feature
- Added instance python version
- Rework logic

* Fix EE build error
error[E0599]: no method named `iter` found for tuple `(PyVersion, std::vec::Vec<std::string::String>)` in the current scope

* Support S3

* Support NSJAIL

* Refactor `get_python`

* Make NSJAIL work [Unsafe]

config file missed /proc mount causing install phase to fail

* Trigger CI

* Clean up

* Make Actions build it

* Trigger CI #2

* Update Dockerfile and clean up

* Change fallbacks
now there is only no_uv and NOUV

* Expose INSTANCE_PYTHON_VERSION through env variable

* Change namings

* Include py-version to requirements.in

Also add comments and make code much cleaner

* Use const for python installation dir

It was hardcoded before

* Pin preinstalled version

* Update python_executor.rs

* Up to date branch

* Create PYCACHE dirs

TODO: PY_TAR_DIRS

* Fix after merge

* Make it safer

* Implement USE_SYSTEM_PYTHON

* Implement latest_stable option

* Load INSTANCE_PYTHON_VERSION on startup

* Check for multiple annotations used

* Fix Latest Stable button not pressed if selected

* Proper error handling for conflict on multiple annotations

* Fix merge conflicts

* Preinstall 3.11 and Latest Stable

* Preinstall latest stable in non-blocking manner

* Fix Warning

* Gate preinstall logic behind "python" feature

* Handle raw_deps properly

* Make it work with nsjail

* Revert docker-image.yml

* Revert Dockerfile

* Cleanup + Fixing

* Add windows support

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 11:42:48 +01:00
Patrick Dobbs e854318f29 Update lib.rs with python-taiga import alias (#4776)
* Update lib.rs with python-taiga import alias

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-21 23:18:38 +01:00
VuRsd 2eb9cfd0a9 add pysocks import handling
add PySocks dependency mapping
2024-11-08 09:56:28 +01:00
Ruben Fiszel f5b6050d5d fix: improve express oauth setup 2024-10-30 20:40:50 +01:00
wendrul 858f63344f split script argument and dependency parser packages to lighten initial load of the script editor (#4287)
* Add feature flags to split parsers into different pkgs

* Split wasm parser imports

* Use regex-lite, reorganize the parser split

* Update imports to the new wasm parser split

* Remove panic system on wasm and simplify snake case convert logic

* Adapt new imports

* Fix to_snake_case + fix tests

* Adapt wasm test dependencies

* Add publish script

* Fix publish script

* Publish script relative to script location

* pkg diff + publish

* Fix TS WASM import + add pakcage lock

* Fix lint

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-28 13:26:55 +02:00
Danny Summerlin 5549e6c30c adding msgraph-sdk to Python module aliases (#3913) 2024-06-17 14:01:52 +02:00
Ruben Fiszel b99107e8c0 add lokalize exception to import map 2024-06-10 22:14:13 +02:00
HugoCasa c9f4ae7dd8 fix: remove main decorator after split from end of file during py imports parsing (#3887)
* fix: remove main decorator after split from end of file during py imports parsing

* fix: nit
2024-06-07 22:08:22 +02:00
Ruben Fiszel 47c722b20f add opensearchpy to lib.rs mapping 2024-05-27 16:41:30 +02:00
Jeremy Worden 2f9b6aa2d2 Update lib.rs (#3795)
add ldap=>python-ldap remapping
2024-05-23 08:53:42 +02:00
Ruben Fiszel d7ef9a1062 add github=>pygithub remapping 2024-05-01 21:02:02 +02:00
Ruben Fiszel 8830a33406 support spaces in requirements regex 2024-04-24 18:56:39 +02:00
erickv de637a3dfa import replacement: haystack >> haystack-ai (#3587)
* import replacement: haystack >> haystack-ai

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-04-22 17:52:55 +02:00
Ruben Fiszel 6b98bb1f55 add pymysql 2024-04-17 22:31:07 +02:00
Ruben Fiszel 7dafa3d62a fix compile on non ee 2024-03-20 15:09:28 +01:00
wanglf 969940e7dd add pytenable to resolution exception mapping
* Update lib.rs

PYTHON_IMPORTS_REPLACEMENT: tenable => pyTenable

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-03-12 11:59:07 +01:00
Ruben Fiszel 9af7684f06 fix(python): ignore stdlib imports with _ 2024-02-21 00:10:52 +01:00
Ruben Fiszel e2f7ee33f4 add zoneinfo to stdlib 2024-02-20 20:27:49 +01:00
Ruben Fiszel 2ee069a1b8 fix(python): handle recursive python imports with loop 2024-02-20 19:42:11 +01:00
Ruben Fiszel 52b2b5048d fix: prevent postgres timeout abuse 2024-02-19 19:59:43 +01:00
Ruben Fiszel d2cb7aaa32 fix tests following parser update 2024-01-08 00:57:11 +01:00
Ruben Fiszel bb84ab654a feat: improve git sync and recompute dependents when common python script is redeployed (#2967)
* fix: allow extraConfig for agGrid to set onGridReady

* depmap

* depmap

* depmap

* depmap
2024-01-07 15:18:27 +01:00
Jacob Mastel db4776bb67 Add official Atlassian Repo Mapping (#2661)
The official Atlassian python library is named atlassian-python-api,
which requires an installation under that name. It is imported as
'atlassian' though.
2023-11-20 21:04:44 +01:00
betterthanever2 02374cd30e Update lib.rs (#2643)
Adding OpenCV to Python install/import exceptions mapping
2023-11-17 15:39:24 +01:00
Ruben Fiszel 20a6e7414e improve slow pull metrics 2023-10-30 12:20:40 +01:00
Guillaume Bouvignies 7441e407d4 fix: concurrency limit EE feature warning (#2505)
* fix: concurrency limit EE feature warning

* fix deprecated usage

* skip serializing empty optional
2023-10-24 17:59:16 +02:00
Ruben Fiszel affaf57250 update rust-python to 3.0 (#2499) 2023-10-24 09:26:36 +02:00
Ruben Fiszel 83877a3591 fix: add shapefile to python remapping 2023-10-23 23:06:08 +02:00
Bonnie White edd93b5d44 Update lib.rs (#2495) 2023-10-23 22:50:26 +02:00
Ruben Fiszel 9301cd7004 update resolver with tabular and msoffcrypto 2023-10-13 00:43:32 +02:00
Ruben Fiszel 6e6a714775 add dnspython 2023-10-03 01:08:37 +02:00