Commit Graph

434 Commits

Author SHA1 Message Date
wendrul 4facf3ca3e feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Alexander Petric 8909bea935 feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
Lucas Abel e6c7fe7027 backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
wendrul 8f198ba68c feat: Indexer improvements: s3 backup logic reworked, settings on the frontend (#4763)
* Add settings page + common code for indexer

* Prepare sqlx
2024-11-20 22:40:36 +01:00
Ruben Fiszel 45ce5ff8c6 add FORCE_HOSTNAME env var 2024-11-20 16:07:24 +01:00
HugoCasa 6924370f11 refactor sql result formatting (#4760) 2024-11-20 14:53:39 +01:00
HugoCasa b3a7cb0583 feat: pdf file preview (#4753)
also fixes s3 missing content-length header
2024-11-19 19:09:20 +01:00
Lucas Abel 97457a5679 feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00
wendrul 4d9ec909d1 fix: playbook files permission mode incompatible with Windows (#4740) 2024-11-18 22:14:19 +01:00
wendrul 5e10782779 feat: Add mode (permissions) option to files in ansible (#4724)
* Add chmod option to files in ansible

* Add 0 prefix if string

* Rename chmod to mode
2024-11-18 17:02:38 +01:00
Alexander Petric c069969732 cloud_hosted non ee scope (#4723) 2024-11-15 18:14:11 +01:00
Alexander Petric ff5fcc59d4 if CLOUD_HOSTED and workspace_id set, autoacknowledge for superadmin (#4719) 2024-11-15 16:31:42 +01:00
Alexander Petric c32038a76d feat(monitoring): workspace critical alerts (#4684)
* critical alert ui

* updating ui, backend logic

* revert

* type check fix npm

* checking out cli files from main

* moving alert icon

* adding sqlx mock data

* more sqlx changes

* feat(frontend): nodes from flow can be connected directly in expr input through a plug icon (#4652)

* Add flow prop picker

# Conflicts:
#	frontend/src/lib/components/propertyPicker/PropPicker.svelte

* fix unwanted copy

* cleaning

* Fix unset context

* move button and always display input

* fix unwanted proppicker display

* update

* update

* clean all

* clean all

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>

* replace hide/show with toggle

* adding mutable setting and navigation to settings to configure channels

* merge fix

* ee non ee changees

* auto-acknowledge when muted

* pr comments

* fix bad log

* user inner modal component

* update unaknowledge alerts after acknowledging from modal

* aknowledge -> acknowledge

* format

* adding backend support for workspace critical alerts

* immediately check for alerts

* immediately check for alerts

* adding openapi changes

* simplify loading of superadmin/ee

* update modal logic

* frontend logic update

* sqlx prepare

* adding mute functionality for workspace critical alerts + show alerts button for instance and workspace settings

* npm / rust warnings & errors

* reverting non-mac cargo toml

* reverting non-mac cargo toml

* mute toggles in critical alert ui

* ui polish

* adding acknowledged_workspace column

* sqlx prepare

* make sure we wait for all stores to be loaded

* adding workspace mute logic to report_critical alert

* auto ack recovered alerts

* toggle workspace as superadmin and critical alerts in logs menu

* adding critical alert button if width < 786px

* refresh on change

* Move notification to logs (#4698)

* Move notification to logs

* removing unused button

---------

Co-authored-by: Alexander Petric <petric.al@gmail.com>

* tailwind typo

* auto ack on mute

* don't change deref

* sqlx prep

* avoid renaming db column, keep acknowledged instead of acknowledged_global

* hide critical alert menu when not ee

* if workspace muted, also acknowledge global when workspace set

* removing save button for mute setting and improve ux/responsiveness

* fix: deployment callbacks have a concurrency limit of 1 on same path

* sqlx prep

* ee-repo ref

* z-[9999] for modal

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-15 09:48:48 +01:00
pyranota f240d1322a feat: Handle pip install by uv (#4517)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Update Dockerfile

* Update docker-image.yml

* Update docker-image.yml

* Add --link-mode=copy and remove -v

* Fix NSJAIL INDEX_URL

* Fix flock and windows

* Update python_executor.rs

* Remove line from Dockerfile

We dont need it and to trigger build

* fixing for windows

* Dont pin python to specific version

* Change TMP for windows

* Revert docker-image.yml

* Disable UV for ansible

Will be enabled later.
Needs proper testing and its better to split onto 2 PRs with first modifying python and second ansible

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Alexander Petric <petric.al@gmail.com>
2024-11-15 00:12:56 +01:00
dieriba 556b4a41a1 feat: Support mistral anthropic for ai (#4692)
* wip: openai proxy and other ai proxy integration

* fixing migration script

* wip: support different ai provider in front, fix proxy openai

* wip: adding frontend ai provider

* updated copilot types

* wip: working on anthropic integration

* done AI proxy front

* adding new type and support for anthropic

* updating gitignore

* adding streaming response

* added streaming prompt

* push lib/gen

* wip: fixing anthropic

* anthropic fully supported

* fix backend missing var error and fully support stream event for anthropic

* remove gen directory

* fixing openapi file

* add support for mistral, and update create workspace components

* remove deref.json

* remove package-json

* openapi

* fix ui enable code

* added utility function for init workspace ai provider

* fix workspace switch bug

* update anthropic property and fixed frontend error

* fix workspace settings

* update error message and fix typo migration file

* chore: update openapi file

* fix dev file

* add .sqlx

* all

* update sqlx

---------

Co-authored-by: dieriba <t.dieriba@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-14 14:24:35 +01:00
Lucas Abel 50ff183bae feat(backend): monitor minimal version of living workers (#4704) 2024-11-14 12:33:50 +01:00
Lucas Abel 44ed4045f7 api: cleanup job data structure (#4705) 2024-11-14 12:25:30 +01:00
Lucas Abel 2d572695ef nit: generalize usage of has_failure_module (#4706) 2024-11-14 08:49:12 +01:00
Alexander Petric d9148eaa78 feat(frontend): critical alerts UI (#4653) 2024-11-09 00:42:10 +01:00
HugoCasa dc8bd6d2b5 feat: http custom routes for static assets (#4666)
* feat: http custom routes for static assets

* update ee ref + fix build

* fix build

* nit
2024-11-07 20:16:14 +01:00
HugoCasa ad50d2fd69 allow no tls for instance settings smtp (#4656)
* allow no tls for instance settings smtp

* update ee ref
2024-11-06 16:58:24 +01:00
Guilhem 22ab51e991 feat(frontend): improve and simplify scheduled poll flows (#4560) 2024-11-04 23:18:09 +01:00
Ruben Fiszel fd4fecf945 split better oss and ee 2024-11-02 16:31:03 +01:00
HugoCasa c803f9cdb2 feat: private hub user accessible url setting (#4617)
* feat: private hub user accessible url setting

* improve UI
2024-11-01 11:20:31 +01:00
Ruben Fiszel 557b949efe fix(s3): align s3 handler additional creds providers 2024-11-01 00:51:28 +01:00
HugoCasa 7fdb524fc6 improve key renewal (#4519)
* improve key renewal

* sqlx and fix

* fix build

* improve everything

* nits

* nits

* update ee ref

* nit
2024-10-29 09:26:45 +01:00
Ruben Fiszel 71f84dee08 handle better empty tag overrides 2024-10-29 09:01:02 +01:00
Ruben Fiszel ab38215c86 fix(backend): in flows, workspace scripts should use their set tags instead of the default one 2024-10-29 07:29:39 +01:00
Ruben Fiszel 85de44cb93 feat: autoscaling v0 (#4593)
* all

* all

all

all

all

all

all

all

all

all

* all

* all

* nits

* nits
2024-10-28 00:47:19 +01:00
wendrul e645e6fe5e Remove debug line (#4548) 2024-10-18 20:31:53 +02:00
pyranota c3ba832444 Annotations refactor (#4512)
* Refactor annotation system

* Move tests to different location

* Limit annotations per line (15)

* Write initial benchmark

* Optimize

* More optimizations

* Rewrite with 3x speed!

* Remove benchmarks

They were in wrong place and for development

* Polishing

* Remove unused leftover

* Integrate new annotation system

* Add comment explanation

* Remove `hashe-based matching`

Turned to be it is impure and non-deterministic method

* Add `tracing::error` for `// native` without `deno_core`

If you build backend without `deno_core` feature flag, and add //native
annotation to ts script, it would just silently exit.

This commit prints error to logs

* add more tests

* Update annotations.rs

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-17 09:14:31 +02:00
Ruben Fiszel 5bf0ade82d fix tests 2024-10-09 22:37:07 +02:00
Ruben Fiszel b397580d23 fix s3 bucket test 2024-10-09 21:30:28 +02:00
Alexander Petric 585f55edd2 feat(monitoring): Critical Alerts for Jobs Waiting in Queue [enterprise] (#4491)
* adding the job queue monitor

* first draft of queue metrics ui

* fix svelte check complaints

* harmonizing naming

* adding concurrency_lock table

* updating ee repo ref

* Rename 20241008155800_alert_lock_table.up.sql to 20241008155800_concurrency_lock_table.up.sql

* Rename 20241008155800_alert_lock_table.down.sql to 20241008155800_concurrency_lock_table.down.sql

* updating ee.rs

* adding owner as optional column

* handling of ee/non-ee

* Update ee.rs with newline
2024-10-09 20:48:18 +02:00
pyranota 1e7a37ac8b feat: Replace pip-compile with uv (#4460)
* Update `shell.nix`

- Replace pip-compile with uv packages
- Pin rust version
- Add var to trigger windmill print more info in stdout

* Replace `pip-compile` with `uv` (dirty + untested)

* Fix arguments passed to uv

Some of the flags are included by default in UV and can be safely removed:
- --resolver=backtracking
- --no-emit-index-url

Also uv does not support `--pip-args` and suggests to directly pass args to uv.

* Remove extra `dbg!`

* Replace 'pip-compile' with 'uv' in Dockerfile

* Add fallback option to `pip-compile` (Disabled)

* Add `uv` to `docker/DockerfileSlim*`

* Add `get_annotation_python` and rename `get_annotation` to `get_annotation_ts`

* Add option to fallback to pip-compile

Put `# no_uv` on top the file for specific python script

Or set `USE_PIP_COMPILE` variable to `true`

* Put back `pip-tools` into shell.nix

* Make sure lockfile resolves again if `#no_uv` used

Add #no_uv to the end of requirements (requirements.in)
That way if something breaks for customer, then they put #no_uv and new lockfile will be resolved

* Put `pip install pip-tools` in original spot

* Fix compilation error

* Fix EE compilation error

error[E0658]: attributes on expressions are experimental
   --> windmill-worker/src/python_executor.rs:144:5
    |
144 |     #[cfg(feature = "enterprise")]
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information

* Add `no_cache` annotation

Will force recalculation of lockfile
And block uv from using cached values

* Target uv cache to /tmp/windmill/cache

* Prohibit uv from managing python

* Add uv to DockerfileBackendTests

* Pin uv version to 0.4.18 in Dockerfiles

* Dont put `#no_uv` in requirements.in

Instead postfix hash for requirements.in with `-no_uv`

* Push Warning to logs if fallbacked to pip-compile

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 16:00:35 +02:00
Alexander Petric 6eb7f8ce81 feat(worker): support workers to run natively on windows (#4446)
* minimal code change to get windmill worker on windows for bun and python + rustfmt

* adding support for powershell

* compiling error on unix

* rust linting comments

* comments hugo: PSModulePath

* comments ruben, refactor to simplify

* adding build workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* skip migration env, ee fixes

* improvements powershell

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* killing process tree in windows

* sqlx_offline

* install openssl for github windows runner

* used pre-installed openssl

* used pre-installed openssl

* build ee

* build ee

* build ee

* build ee

* adding commented out steps for artifact publishing

* build on tag matchinv v* pattern

* ren instead of mv on Windows

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix merging issue

* gate imports for windows

* fixing default cargo home path...

* fixing default cargo home path...

* comments ruben

* make pwsh default modules loading more robust on unix (#4448)

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 15:59:12 +02:00
HugoCasa 04ef675b1d skip step better UI (#4465) 2024-10-01 12:28:46 +02:00
HugoCasa 62d88436b2 feat: flow step skipping (#4461) 2024-09-30 16:30:59 +02:00
HugoCasa 41ff40f777 send stats when renewing key if last >24h (#4430)
* feat: send stats when renewing key if last >24h

* nits

* fix: sqlx

* nit

* renewal reason

* stats reason

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-09-30 12:23:07 +02:00
Ruben Fiszel 41e8c0def2 optimize skip failure query 2024-09-29 00:38:43 +02:00
Ruben Fiszel 887fd9043e optimize more queries 2024-09-28 17:34:37 +02:00
Ruben Fiszel f189aaad5a fix: improve suspend_first behavior and frequency 2024-09-28 15:46:50 +02:00
Ruben Fiszel 048625404d add suspended flows to cli queues 2024-09-26 16:52:09 +02:00
Ruben Fiszel 5a191d83ca improve list queue api 2024-09-26 14:31:17 +02:00
Ruben Fiszel bd6ddc166b feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
HugoCasa 97cdafac86 feat: add return_last_result annotation to sql (#4443) 2024-09-25 18:11:11 +02:00
HugoCasa 1feccd493e feat: add tag filtering to external JWT authentication (#4425)
* feat: tag filtering jwt ext auth

* move tags to scopes

* fix symlink

* update ee ref
2024-09-25 11:21:19 +02:00
Ruben Fiszel 0fc74340b0 improve init script error handling 2024-09-24 19:27:00 +02:00
wendrul a7ef3ed293 fix: various improvements for ansible (#4419)
* Improve error messages, fix dependency job issue

* Add nsjail support for ansible

* Remove invalid comment syntax

* Change nsjail config and ansible controller config to be ephemeral

* Move ansible cfg to later to overwrite possible files named ansible.cfg

* Fix ansible.cfg env var

* Fix tmp dir issue
2024-09-23 16:44:37 +02:00
Ruben Fiszel b1862ab2bd critical channels build on non ee 2024-09-21 22:54:15 +02:00