Commit Graph
68 Commits
Author SHA1 Message Date
Pyra cd06bd25e1 nit(python): add mapping for azure-identity #7922 2026-02-12 10:07:21 +00:00
Luigi 8e19f9652d Added serpapi import handling (#7871)
Added the handling for serpapi import. The name of the Python library to be installed is google-search-results.

Source: https://pypi.org/project/google-search-results/
2026-02-10 09:59:35 +00:00
Maxime Thiebaut 00fc86d099 Add grpc entry to mapping in mapping.rs (#7616)
Map `grpc` to [`grpcio`](https://pypi.org/project/grpcio/).
2026-01-20 06:00:10 +00:00
PyraandRuben Fiszel d38c96db36 feat: workspace dependencies (#7124)
* commit raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* raw requirements

Signed-off-by: pyranota <pyra@duck.com>

* implement `parse_annotation`

Signed-off-by: pyranota <pyra@duck.com>

* more progress on wdeps

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* more progress

Signed-off-by: pyranota <pyra@duck.com>

* fixes

Signed-off-by: pyranota <pyra@duck.com>

* cli improvements + raw deps

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix python versions

Signed-off-by: pyranota <pyra@duck.com>

* progress

Signed-off-by: pyranota <pyra@duck.com>

* update :)

Signed-off-by: pyranota <pyra@duck.com>

* add MaybeLock

Signed-off-by: pyranota <pyra@duck.com>

* go WIP

Signed-off-by: pyranota <pyra@duck.com>

* fix python ignoring py version from requirements

Signed-off-by: pyranota <pyra@duck.com>

* optimize php

Signed-off-by: pyranota <pyra@duck.com>

* require admin to alter

Signed-off-by: pyranota <pyra@duck.com>

* fix(cli): flow generateLocks raw deps

Signed-off-by: pyranota <pyra@duck.com>

* progress in checklist

Signed-off-by: pyranota <pyra@duck.com>

* fix agent workers

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* nit: remove default features

Signed-off-by: pyranota <pyra@duck.com>

* oh-wow

Signed-off-by: pyranota <pyra@duck.com>

* remove dbg!

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* add indexes

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* add debug flag

Signed-off-by: pyranota <pyra@duck.com>

* cli: remove noise

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* remove todos

Signed-off-by: pyranota <pyra@duck.com>

* trigger deps correctly

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix frontend again

Signed-off-by: pyranota <pyra@duck.com>

* finally fix frontend

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* fix all

Signed-off-by: pyranota <pyra@duck.com>

* more fixes...

Signed-off-by: pyranota <pyra@duck.com>

* remove test

Signed-off-by: pyranota <pyra@duck.com>

* Update backend-test.yml

* comment out legacy test

Signed-off-by: pyranota <pyra@duck.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* fix ci?

Signed-off-by: pyranota <pyra@duck.com>

* comment out thing

Signed-off-by: pyranota <pyra@duck.com>

* ignore test

Signed-off-by: pyranota <pyra@duck.com>

* ci

Signed-off-by: pyranota <pyra@duck.com>

* base fixture

Signed-off-by: pyranota <pyra@duck.com>

* fix regression

Signed-off-by: pyranota <pyra@duck.com>

* fix docs links

Signed-off-by: pyranota <pyra@duck.com>

* update min version

Signed-off-by: pyranota <pyra@duck.com>

* simplify

* implement cache for get_latest

Signed-off-by: pyranota <pyra@duck.com>

* move to workspace settings

Signed-off-by: pyranota <pyra@duck.com>

* sqlx + migration

Signed-off-by: pyranota <pyra@duck.com>

* more migrations

Signed-off-by: pyranota <pyra@duck.com>

* use box pin

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-28 16:00:11 +00:00
Ruben Fiszel 5b1ab610b7 improve stack size usage (#6936)
* oom

* final

* final
2025-10-24 22:08:54 +00:00
Ruben Fiszel 8518b7c897 check 2025-09-20 00:59:03 +00:00
Ruben Fiszel 97ed4a539b nits cleanup + faster script index #6450 2025-08-23 22:42:39 +00:00
pyranotaandRuben Fiszel f32159f412 fix: do not split main.py on 'def main' (#5895)
doesn't work well in edge cases:

```
@schedule(
    bla=True
)
def main(x: str):
    return x
```

```
error_message = """
Please ensure you have defined main(username, password) correctly.
The function def main( should be at the end.
"""

def main(x: str):
    return x
```

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-06-20 16:40:30 +02:00
pyranota 88ab1a5136 Use regex to split input to python import parser (#6007) 2025-06-20 13:24:27 +02:00
pyranota 06e61ee958 Add fake main function to fix Python AST parsing (#6006)
The changes ensure the Python AST parser can handle code snippets where the main
function was previously removed, by appending a dummy main function.
2025-06-20 09:28:30 +02:00
26222539e6 feat(python): inline script metadata (PEP 723) (#5712)
* make resolver

* more updates

* fix build

* fix raw_dependencies job type

* compat with http agent workers

* refactor

* rename

* more refactor

* cleanup

* more tests

* fix s3

* small fixes

* more fixing

* fix endpoint

* nit: update comment

* update ee ref

* update ee ref

* update ee ref

* implement safer `list_available_python_versions`

* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix build

* update ee ref

* ignore versions <=3.9

* fix windows build

* correct versions filter

* fix windows build (this time for real)

* inject error to debug CI

* update CI

* undo debug of CI

* fix tests

* remove outdated comment

* update ee repo ref

* Update ee-repo-ref.txt

* Update backend/parsers/windmill-parser-py-imports/src/lib.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update InstanceSetting.svelte

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
Eugene Chernyavsky 6b2ba3f261 google-genai and google.generativeai requirement from import (#5699) 2025-05-02 19:50:20 +02:00
pyranota 4c660a8869 nit(python): map azure-storage-file-datalake (#5682)
* feat(python): map azure-storage-file-datalake

* add comma
2025-04-30 10:04:10 +02:00
Maxime Thiebaut fbdaf28fb5 nit(python): add virustotal to imports mapping (#5683) 2025-04-29 22:55:58 +02:00
pyranota 1ab41603f4 fix: make #(extra_)requirements: work better with pins (#5680) 2025-04-29 19:59:51 +02:00
Ruben Fiszel 193186ae75 nits 2025-04-29 15:43:30 +02:00
Ruben Fiszel f662cf5d75 fix(python): handle better relative imports with requirements or extra_requirements 2025-04-29 15:23:44 +02:00
pyranota 0b6d017fed feat(python): per import requirement pin (#5520)
* implement single line pin

* make panic-safe

* use pin even if multiple modules imported withing single statement

* add repins and make imports respect pins

* keep all pins

* Allow multiple pins

* add comments + handle stuff more safely

* fix fully qualified imports

* remove ignore

* sort nested

* apply unique to output requirements list

* fix typo

* remove mut

* update sqlx

* sort imports

* sort imports

* fix formatter and format

* refactor

* fix comptime error

* write tests

* perf: do not capture if string is empty
2025-04-11 21:31:51 +00:00
pyranota a0921abfb3 nit(python): add azure keyvault and storage to imports mapping (#5517)
* feat(python): fully qualified imports mapping

* make contributor-friendly

* use more intuitive sign

* map the most of the google and azure

* nit(python): add azure keyvault and storage to imports mapping
2025-03-27 00:11:28 +01:00
pyranota 1a5566b8c2 feat(python): fully qualified imports mapping (#5511)
* feat(python): fully qualified imports mapping

* make contributor-friendly

* use more intuitive sign

* map the most of the google and azure
2025-03-26 23:42:21 +01:00
Ruben Fiszel e71882d0a0 add zlib to import parser exception 2025-02-21 21:26:14 +01:00
Ruben Fiszel b5fa5f4dd2 add docx => python-docx python exception 2025-02-10 14:45:29 +01:00
Arjun Dhawan 343ff94f3c add bigquery to the list of python depenency exceptions (#5173) 2025-01-30 13:12:34 +01:00
pyranotaandRuben Fiszel e47dd697f9 feat(python): Multiple runtime versions (#4579)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Remove verbose mode and enable link-mode=copy

* Granural migration of lockfiles

Before i realized we dont need it :)

* Initial draft (not-working)

* Add fallback

* Fix bug preventing uv from installing deps

'\n' - Love it

* Add verbosity indicator

* Iterate on feature
- Added instance python version
- Rework logic

* Fix EE build error
error[E0599]: no method named `iter` found for tuple `(PyVersion, std::vec::Vec<std::string::String>)` in the current scope

* Support S3

* Support NSJAIL

* Refactor `get_python`

* Make NSJAIL work [Unsafe]

config file missed /proc mount causing install phase to fail

* Trigger CI

* Clean up

* Make Actions build it

* Trigger CI #2

* Update Dockerfile and clean up

* Change fallbacks
now there is only no_uv and NOUV

* Expose INSTANCE_PYTHON_VERSION through env variable

* Change namings

* Include py-version to requirements.in

Also add comments and make code much cleaner

* Use const for python installation dir

It was hardcoded before

* Pin preinstalled version

* Update python_executor.rs

* Up to date branch

* Create PYCACHE dirs

TODO: PY_TAR_DIRS

* Fix after merge

* Make it safer

* Implement USE_SYSTEM_PYTHON

* Implement latest_stable option

* Load INSTANCE_PYTHON_VERSION on startup

* Check for multiple annotations used

* Fix Latest Stable button not pressed if selected

* Proper error handling for conflict on multiple annotations

* Fix merge conflicts

* Preinstall 3.11 and Latest Stable

* Preinstall latest stable in non-blocking manner

* Fix Warning

* Gate preinstall logic behind "python" feature

* Handle raw_deps properly

* Make it work with nsjail

* Revert docker-image.yml

* Revert Dockerfile

* Cleanup + Fixing

* Add windows support

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 11:42:48 +01:00
Patrick DobbsandRuben Fiszel e854318f29 Update lib.rs with python-taiga import alias (#4776)
* Update lib.rs with python-taiga import alias

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-21 23:18:38 +01:00
VuRsd 2eb9cfd0a9 add pysocks import handling
add PySocks dependency mapping
2024-11-08 09:56:28 +01:00
Ruben Fiszel f5b6050d5d fix: improve express oauth setup 2024-10-30 20:40:50 +01:00
858f63344f split script argument and dependency parser packages to lighten initial load of the script editor (#4287)
* Add feature flags to split parsers into different pkgs

* Split wasm parser imports

* Use regex-lite, reorganize the parser split

* Update imports to the new wasm parser split

* Remove panic system on wasm and simplify snake case convert logic

* Adapt new imports

* Fix to_snake_case + fix tests

* Adapt wasm test dependencies

* Add publish script

* Fix publish script

* Publish script relative to script location

* pkg diff + publish

* Fix TS WASM import + add pakcage lock

* Fix lint

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-28 13:26:55 +02:00
Danny Summerlin 5549e6c30c adding msgraph-sdk to Python module aliases (#3913) 2024-06-17 14:01:52 +02:00
Ruben Fiszel b99107e8c0 add lokalize exception to import map 2024-06-10 22:14:13 +02:00
HugoCasa c9f4ae7dd8 fix: remove main decorator after split from end of file during py imports parsing (#3887)
* fix: remove main decorator after split from end of file during py imports parsing

* fix: nit
2024-06-07 22:08:22 +02:00
Ruben Fiszel 47c722b20f add opensearchpy to lib.rs mapping 2024-05-27 16:41:30 +02:00
Jeremy Worden 2f9b6aa2d2 Update lib.rs (#3795)
add ldap=>python-ldap remapping
2024-05-23 08:53:42 +02:00
Ruben Fiszel d7ef9a1062 add github=>pygithub remapping 2024-05-01 21:02:02 +02:00
Ruben Fiszel 8830a33406 support spaces in requirements regex 2024-04-24 18:56:39 +02:00
erickvandRuben Fiszel de637a3dfa import replacement: haystack >> haystack-ai (#3587)
* import replacement: haystack >> haystack-ai

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-04-22 17:52:55 +02:00
Ruben Fiszel 6b98bb1f55 add pymysql 2024-04-17 22:31:07 +02:00
Ruben Fiszel 7dafa3d62a fix compile on non ee 2024-03-20 15:09:28 +01:00
wanglfandRuben Fiszel 969940e7dd add pytenable to resolution exception mapping
* Update lib.rs

PYTHON_IMPORTS_REPLACEMENT: tenable => pyTenable

* Update lib.rs

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-03-12 11:59:07 +01:00
Ruben Fiszel 9af7684f06 fix(python): ignore stdlib imports with _ 2024-02-21 00:10:52 +01:00
Ruben Fiszel e2f7ee33f4 add zoneinfo to stdlib 2024-02-20 20:27:49 +01:00
Ruben Fiszel 2ee069a1b8 fix(python): handle recursive python imports with loop 2024-02-20 19:42:11 +01:00
Ruben Fiszel 52b2b5048d fix: prevent postgres timeout abuse 2024-02-19 19:59:43 +01:00
Ruben Fiszel bb84ab654a feat: improve git sync and recompute dependents when common python script is redeployed (#2967)
* fix: allow extraConfig for agGrid to set onGridReady

* depmap

* depmap

* depmap

* depmap
2024-01-07 15:18:27 +01:00
Jacob Mastel db4776bb67 Add official Atlassian Repo Mapping (#2661)
The official Atlassian python library is named atlassian-python-api,
which requires an installation under that name. It is imported as
'atlassian' though.
2023-11-20 21:04:44 +01:00
betterthanever2 02374cd30e Update lib.rs (#2643)
Adding OpenCV to Python install/import exceptions mapping
2023-11-17 15:39:24 +01:00
Ruben Fiszel 20a6e7414e improve slow pull metrics 2023-10-30 12:20:40 +01:00
Guillaume Bouvignies 7441e407d4 fix: concurrency limit EE feature warning (#2505)
* fix: concurrency limit EE feature warning

* fix deprecated usage

* skip serializing empty optional
2023-10-24 17:59:16 +02:00
Ruben Fiszel affaf57250 update rust-python to 3.0 (#2499) 2023-10-24 09:26:36 +02:00
Ruben Fiszel 83877a3591 fix: add shapefile to python remapping 2023-10-23 23:06:08 +02:00