* feat: Handle `pip install` by `uv`
Dirty and untested, but already something working
* Integrate with NSJAIL and prepare fallbacks
* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific
* Remove `--disable-pip-version-check`
Reason:
warning: pip's `--disable-pip-version-check` has no effect
* Fix backend compilation error
* Pip fallback overwrite UV's cache
* Initially refactor cache (No S3)
* Support S3
* Remove unused import
* Handle flags for NSJAIL
* Return deleted flag
* Update Dockerfile
* Update docker-image.yml
* Update docker-image.yml
* Add --link-mode=copy and remove -v
* Fix NSJAIL INDEX_URL
* Fix flock and windows
* Update python_executor.rs
* Remove line from Dockerfile
We dont need it and to trigger build
* fixing for windows
* Dont pin python to specific version
* Change TMP for windows
* Revert docker-image.yml
* Disable UV for ansible
Will be enabled later.
Needs proper testing and its better to split onto 2 PRs with first modifying python and second ansible
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Alexander Petric <petric.al@gmail.com>
* Refactor annotation system
* Move tests to different location
* Limit annotations per line (15)
* Write initial benchmark
* Optimize
* More optimizations
* Rewrite with 3x speed!
* Remove benchmarks
They were in wrong place and for development
* Polishing
* Remove unused leftover
* Integrate new annotation system
* Add comment explanation
* Remove `hashe-based matching`
Turned to be it is impure and non-deterministic method
* Add `tracing::error` for `// native` without `deno_core`
If you build backend without `deno_core` feature flag, and add //native
annotation to ts script, it would just silently exit.
This commit prints error to logs
* add more tests
* Update annotations.rs
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* Update `shell.nix`
- Replace pip-compile with uv packages
- Pin rust version
- Add var to trigger windmill print more info in stdout
* Replace `pip-compile` with `uv` (dirty + untested)
* Fix arguments passed to uv
Some of the flags are included by default in UV and can be safely removed:
- --resolver=backtracking
- --no-emit-index-url
Also uv does not support `--pip-args` and suggests to directly pass args to uv.
* Remove extra `dbg!`
* Replace 'pip-compile' with 'uv' in Dockerfile
* Add fallback option to `pip-compile` (Disabled)
* Add `uv` to `docker/DockerfileSlim*`
* Add `get_annotation_python` and rename `get_annotation` to `get_annotation_ts`
* Add option to fallback to pip-compile
Put `# no_uv` on top the file for specific python script
Or set `USE_PIP_COMPILE` variable to `true`
* Put back `pip-tools` into shell.nix
* Make sure lockfile resolves again if `#no_uv` used
Add #no_uv to the end of requirements (requirements.in)
That way if something breaks for customer, then they put #no_uv and new lockfile will be resolved
* Put `pip install pip-tools` in original spot
* Fix compilation error
* Fix EE compilation error
error[E0658]: attributes on expressions are experimental
--> windmill-worker/src/python_executor.rs:144:5
|
144 | #[cfg(feature = "enterprise")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information
* Add `no_cache` annotation
Will force recalculation of lockfile
And block uv from using cached values
* Target uv cache to /tmp/windmill/cache
* Prohibit uv from managing python
* Add uv to DockerfileBackendTests
* Pin uv version to 0.4.18 in Dockerfiles
* Dont put `#no_uv` in requirements.in
Instead postfix hash for requirements.in with `-no_uv`
* Push Warning to logs if fallbacked to pip-compile
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* minimal code change to get windmill worker on windows for bun and python + rustfmt
* adding support for powershell
* compiling error on unix
* rust linting comments
* comments hugo: PSModulePath
* comments ruben, refactor to simplify
* adding build workflow
* editing workflow
* editing workflow
* editing workflow
* editing workflow
* editing workflow
* skip migration env, ee fixes
* improvements powershell
* testing windows runner
* testing windows runner
* testing windows runner
* testing windows runner
* testing windows runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* install postgres on runner
* killing process tree in windows
* sqlx_offline
* install openssl for github windows runner
* used pre-installed openssl
* used pre-installed openssl
* build ee
* build ee
* build ee
* build ee
* adding commented out steps for artifact publishing
* build on tag matchinv v* pattern
* ren instead of mv on Windows
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* fix merging issue
* gate imports for windows
* fixing default cargo home path...
* fixing default cargo home path...
* comments ruben
* make pwsh default modules loading more robust on unix (#4448)
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* Improve error messages, fix dependency job issue
* Add nsjail support for ansible
* Remove invalid comment syntax
* Change nsjail config and ansible controller config to be ephemeral
* Move ansible cfg to later to overwrite possible files named ansible.cfg
* Fix ansible.cfg env var
* Fix tmp dir issue
* Allow setting progress explicitly from script body.
This feature exposes:
* `getProgress`
* `setProgress`
* `incProgress`
API in TypeScript client (python is coming soon).
NOTE: Progress cannot be out of range 0..100 and cannot decrease.
With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.
For optimization reasons, jobs start to ask for progress only after N-seconds of execution.
* feat: Add `shell.nix`
If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies
NOTE: You will still need docker
* feat: Add `dev.nu` to typescript client
Little helper function, allowing developer to work on ts client easier.
To use:
`./dev.nu watch`
Now add import of windmill in body of your script and `//nobundle` on top of the file
Edit ts client in your favourite editor and hit save. Script will do the rest.
* Cleanup files
* Fix: Failed to deserialize query string: missing field `get_progress`
* perf: Implement non-naive polling mechanism for getting job progress
* Add independant delay for getProgress
Problem in `TestJobLoader`:
There should be 2 delays:
One until we find our first progress (every 5s)
Once we found our first progress, we can do it every second
* nit: Use `query_scalar!` instead of `query_as`
* Fix: Sql error, no rows returned by a query that expected to return at least one row
* refactor: Remove global CSS for JobProgressBar
* Change UI for progress of flow subjobs
* Replace `Step 1` with `Running` in ProgressBar for individual jobs
* Remove `incProgress`
incProgress is not very usefull and error-prone
* perf: Set metric only for jobs that are actually using it
(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)
* Offload registering progress from clients to server
* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`
Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.
Could be used by SDK.
* Add `Error::MetricNotFound` for better error handling
* Fix: Make `JobProgressBar` display in red when failed
* Add persistant progress bar
Now you can reload the page after job is done and progress will be still there
* Allow succeeded individual job's progress bar stick to 100%
* Add python support
* nit: Remove usage of undefined variable in python-client
* Add `async` in ts client (for error handling)
* nit(frontend): Remove unused import
* Dont load JobProgressBar when it is not needed
* nit: npm check fix
* cargo sqlx prepare
* fix sqlx
---------
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Alexander Petric
pyranota
dieriba
Lucas Abel
HugoCasa
Guilhem
Ruben Fiszel
Ruben Fiszel
wendrul