Commit Graph

76 Commits

Author SHA1 Message Date
Ruben Fiszel 44c36cdeb5 fix: fix python client for S3Object 2024-06-03 17:20:29 +02:00
Ruben Fiszel a22dc985fa feat: support multiple object storage + parquet_csv + polars -> datafusion" (#3853)
* multiple storage

* all

* rm symlinks

* all

* all
2024-05-31 11:59:51 +02:00
HugoCasa 62f318077c feat: improve permissioned audit logs (#3799)
* fix: increase token label size in db

* Revert "Revert "feat:  improve premissioned audit logs (#3793)" (#3798)"

This reverts commit 43a24d997e.

* feat: add username to email function

* fix: nit

* fix: add docstring to global func
2024-05-23 17:23:50 +02:00
Ruben Fiszel 0e022c9e4e fix(python-client): improve error message for wait_job 2024-04-29 12:10:29 +02:00
Ruben Fiszel 5d6238de11 fix python client set_user_state 2024-03-26 19:09:56 +01:00
Ruben Fiszel bf0dd29a1d fix python client set_user_state 2024-03-26 18:57:54 +01:00
Ruben Fiszel ef1ce83279 feat: add set_flow_user_states and get_flow_user_states api and sdk support 2024-03-26 17:31:17 +01:00
Ruben Fiszel d30cd3a4ba fix: improve workflow as code python client 2024-03-05 19:16:10 +01:00
Ruben Fiszel 6a5fb87491 fix: improve workflow as code python client 2024-03-05 18:42:08 +01:00
Ruben Fiszel 55c80959a0 feat: workflow as code with timelines 2024-03-05 12:45:24 +01:00
Ruben Fiszel 5407265419 fix: workflow as code python sdk improvement 2024-03-04 23:23:41 +01:00
Ruben Fiszel 0795353ec2 fix: workflow as code api improvement 2024-03-04 20:45:04 +01:00
Ruben Fiszel 619e2784f0 feat: workflow as code v0 2024-03-04 20:19:34 +01:00
Ruben Fiszel d8c020fa06 modify python signature 2024-02-24 18:51:33 +01:00
Guillaume Bouvignies e00e3f9d2d fix: Improve python writeS3File perf (#3182) 2024-02-08 17:57:42 +01:00
Ruben Fiszel 60f3a9fa6b fix: add get_root_job_id 2024-02-06 00:17:50 +01:00
Guillaume Bouvignies 86aa6d0f0d fix: Python buffered reader (#3136)
* fix: Python buffered reader

* use bytes generator only for buferred reader
2024-02-02 10:23:04 +01:00
Ruben Fiszel c1cf8b0fb1 fix(python): fix python reader 2024-02-02 01:21:20 +01:00
Ruben Fiszel 583e942174 fix(python): fix python reader 2024-02-02 01:12:01 +01:00
Guillaume Bouvignies 5dabe22935 feat: migrate s3 client to object_store (#3116)
* feat: migrate s3 client to object_store

* remove multipart from open API

* Remove multipart in favor of a single stream

* progress report

* add progress reader on server side

* small nit fix

* fix read chunk

* Fix TS and python SDK

* Fix download button

* Fix download button object viewer

* fix list

* Better errors

* export loadS3FileContent

* revert changes SDK

* fix browser

* small file list unavailable fix

* Old endpoints throws informative error messages

* Typescript SDK uses raw fetch

* update python SDK

* Error if uploaded file > 50Mb

* revert python SDL changes

* Update python SDK method docs

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-02-01 16:03:30 +01:00
Guillaume Bouvignies 6160889793 feat: Download s3 file as stream in Python and TS (#3099) 2024-01-29 14:49:08 +01:00
Guillaume Bouvignies 082aa6a61d fix: Improvement for Python write_s3_file (#3079) 2024-01-25 18:10:47 +01:00
Guillaume Bouvignies da6edee450 fix: Better UI for S3 download and S3 TS SDK endpoints (#3065)
* Better UI for S3 download and S3 TS SDK endpoints

* gitignore typescript client node_modules

* USe Windmill BE upload endpoint for TS SDK

* Use WM backend endpoint in Python upload SDK endpoint

* revert changes

* Add expiration for Python

* Add downaload endpoint

* Add toggle for public S3 resource

* revert changes to package.json

* Add link to doc page

* fix unauthorized bug
2024-01-25 15:34:40 +01:00
Guillaume Bouvignies 376038d70b feat: Download button for s3 files (#3059)
* feat: Download button for s3 files

* Add S3 load and write endpoint to Python and TS SDK

* fix
2024-01-22 21:40:30 +01:00
Ruben Fiszel 3119830062 fix: oidc token generation endpoint GET -> POST 2024-01-17 11:15:39 +01:00
Ruben Fiszel 4604ccde7d fix: expose getIdToken in python-client 2024-01-17 11:04:29 +01:00
Ruben Fiszel 640ebcb146 feat: OIDC support (#3017)
* oidc

* update

* feat: add support for multiselect in python

* d

* d

* progress

* all

* all

* all
2024-01-16 23:32:26 +01:00
Ruben Fiszel 0733dd118e fix: add ability to set secret variable from python 2024-01-13 13:02:36 +01:00
Guillaume Bouvignies 31d48533f4 S3 functions in Python SDK cannot return None (#2954) 2024-01-03 17:26:25 +01:00
Guillaume Bouvignies 3cbdd890cd fix: Update S3 TS code snippets and Python SDK for Polars 0.20.X (#2911)
* fix: update S3 TS code snippets

* use S3Object type from SDK

* update python snippets as well

* Update python SDK for Polars 0.20.X
2023-12-22 08:39:35 +01:00
Guillaume Bouvignies ebcc7ed2aa chore: add verify to Windmill python object for httpx (#2900)
* chore: add `verify` to Windmill python object for httpx

* move it before get_client
2023-12-21 02:38:16 +01:00
Guillaume Bouvignies 2155a6c133 feat: S3 objects are now typed in Python and TS SDK (#2878)
* feat: S3 objects are now typed in Python and TS SDK

* fix ts and python SDK after testing

* cleanup
2023-12-20 09:16:33 +01:00
Guillaume Bouvignies f8c7a8edf8 feat: Add download button to S3 file picker (#2857) 2023-12-14 15:45:37 +01:00
Ruben Fiszel 18e07a781d fix: improve nested flow status viewer 2023-12-13 08:14:34 +01:00
Stephan Fitzpatrick b2f705d01e Updated Windmill client constructor to accept workspace (#2822)
This commit introduces a change in the Windmill class constructor (in wmill/client.py) to include 'workspace' as a parameter. This addition is intended to make "workspace" universally configurable either via an argument or the 'WM_WORKSPACE' environment variable. If not provided, the program will now assert a missing workspace with an informative error message.
2023-12-09 00:22:41 +01:00
Ruben Fiszel 042504f905 fix: add run_flow_sync to python client + few nits 2023-12-05 19:49:18 +01:00
Guillaume Bouvignies 5cb3c34e44 feat: Various minor improvements for S3 DX (#2760)
* feat: Various minor improvements for S3 DX

* Fix openAPI yaml

* Remove unused imports
2023-12-04 13:20:04 +01:00
Ruben Fiszel 651f74b00e improve python client 2023-11-29 21:41:44 +01:00
Ruben Fiszel 772c04c9b3 update python-client 2023-11-29 18:19:47 +01:00
Stephan Fitzpatrick 3bb2a0c960 Corrently handle terminal state when invoking job through run_script method (#2703) 2023-11-25 09:38:53 +01:00
Stephan Fitzpatrick 603e7ff67f feat(python): Update return type for 'get_resource' function (#2695)
* Update return type for 'get_resource' function

Modified the 'get_resource' function in 'wmill/client.py' to additionally return a 'dict', providing support for dictionaries in addition to strings and None. This change caters towards use-cases where getting a dictionary resource is needed, enhancing flexibility of resource handling in Windmill client operations. Code updated in two locations for consistency.

* Update README.md for python-client

Expanded and updated the 'README.md' to include detailed explanations about the basic and advanced usage of the 'wmill' package. The previous version wasn't as comprehensive and lacked examples. Now it includes thorough usage guidance, additional explanations, and code examples. Change improves usability for new developers approaching the project.
2023-11-24 20:17:30 +01:00
Stephan Fitzpatrick 1c2abcda23 Feat(python): Quality of Life improvements to Python client (#2686)
* Fix references to `Windmill.start_script_async`

* Improve client usability and extend functionalities

This commit introduces several changes to improve the usability of the module and extend its functionalities. Line breaks have been added for long function calls to improve readability. A state setter has been added for the 'state' property to facilitate state updates. New functions 'cancel_running' and 'run_script' have been added to provide more control to the user over the script executions. The README file has been updated to reflect these changes and provide more comprehensive usage instructions including both basic and advanced usage of the module.

* Add line breaks and update README.md for readability

Inserted line breaks into function calls for improved readability in python-client/wmill/README.md. This enhances module usability by making code easier to follow. 'state' property setter, 'cancel_running' and 'run_script' functions have been added to enrich module's functionality. README was updated to reflect these changes and provide clearer instructions.
2023-11-23 08:39:38 +01:00
Stephan Fitzpatrick 9783abba0a Fix references to Windmill.start_script_async (#2685) 2023-11-23 07:48:48 +01:00
Ruben Fiszel 6377605821 fix client 2023-11-22 10:28:17 +01:00
Ruben Fiszel 6797d4d114 improve client 2023-11-22 10:04:44 +01:00
Stephan Fitzpatrick 37ffdaed60 feat(python): Refactor Windmill Python client (remove windmill-api) (#2665)
* "feat(python): Refactor Windmill Python client for better encapsulation and maintainability"

This PR obviates the need for the `windmill-api` library. I believe this makes the client package is easier to understand, debug, build, (and test) without it.

Additional updates were made to improve logging and add more robust error handling.

Here's what Jetbrains' AI assistant came up with to describe the changes based on the diff--I think it did a decent job:

> Simplified the Windmill Python client by refactoring out repeated code into more compact, reusable methods. Transitioned the client functions into a Windmill client class, enabling a better encapsulation of the client's state. Updated the README example to reflect this change. This improves code maintainability by making the code easier to understand and update, and improves user experience by providing a more intuitive client interface.

* "Refactor post method in Windmill Python client"

Removed the hard-coded param 'refresh_client' from the post method in wmill/client.py. since it's no longer used.

* "Update build script for Python client"

Updated the build script for the Python client for the backend to now include scaffolding code for generating the OpenAPI client, making changes to the generated client, and building the client. .

* "Add raise_for_status option in http methods"

Enhanced 'get' and 'post' methods in the client class to include a new optional parameter 'raise_for_status'. This allows for better error handling by raising exceptions for 4XX and 5XX responses, if requested. This way, non critical API calls can continue execution even if they receive a client or server error.

* "Removed refresh_client condition in post method"

* Replace `create_job` with `start_execution` in wmill client

This commit changes the method `create_job` to `start_execution` in the wmill client. The change was made to better reflect the function's purpose and make the code even more self-explanatory. Additionally, references to this changed method in README.md and various portions of client.py are also updated. The change will enhance readability and make it easier for newcomers to understand the code.

* Fix type hints and default arguments in wmill client

This commit adjusts typing hints for several methods from Dict[str, Any] to Any in wmill client since the result of a script isn't always a dictionary. Null arguments are also adjusted from {} to None, ensuring better Python standard practices and less unpredicted behaviors. These changes are aimed to enhance maintainability and make the functions more resilient.

* Update client.py

* Update pyproject.toml

* Update client.py

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2023-11-22 09:49:50 +01:00
Stephan Fitzpatrick c7a30f7c9d feat(python): Add functionality and resiliency to wmill python client (#2650)
* Refine wmill client.py `run_script_sync` and `run_script_by_path_sync` with more features

This commit enriches the functionality of the `run_script_sync` and `run_script_by_path_sync` functions. New features introduced include script cancellation upon exit, logging capabilities,
and script execution timeout. These enhancements improve script execution control and provide better debug information. The `get_result` function was also adjusted to enable the control
of 'is not None' assertion on the job result. Consequently, user flexibility is enhanced, and the method can cater to cases where a `None` result is within the expected behavior.

* remove unnecessary local import and rename cancel_atexit to cleanup
2023-11-20 02:07:20 +01:00
Ruben Fiszel 6ae1a69b75 fix: make wmill compatible with python 3.7 2023-11-18 17:15:05 +01:00
Guillaume Bouvignies 90093656b4 feat: Add Python SDK capabilities to generate Polars and DuckDB connection settings to S3 (#2625)
* feat: Add Python SDK capabilities to generate Polars and DuckDB connection settings to S3

* Add endpoint to list S3 objects

* Add FE to set the workspace S3 resource

* Fix openapi

* sqlx prepare

* Hide Windmil LFS tab
2023-11-15 19:37:02 +01:00
Ruben Fiszel 88e3648ee4 fix: optimize single step iterative forloops (#2596)
* cc

* fixed
2023-11-10 13:17:39 +01:00