Commit Graph

3734 Commits

Author SHA1 Message Date
Alexander Petric 1a1ea68216 feat(frontend): operator visibility setting per workspace (#5124)
* feat(frontend): operator visibility setting per workspace

* section + reactivity

* sqlx prep

* npm check

* more sqlx

* fix table bg dark mode

* improve db query / reactivity

* sqlx prepare

* more merge fixing

* npm check

* npm check
2025-01-24 18:17:00 +01:00
dieriba 1eaef8542b fix pg trigger display (#5127) 2025-01-24 17:07:48 +01:00
pyranota 74bb660b97 fix(bun, windows): Fix "A required privilege is not held by the client." (#5126) 2025-01-24 15:32:29 +01:00
dieriba 316a2167e7 feat: postgres triggers (#4860)
* feat: init database triggers

* feat:  wip: database_triggers

* feat:  add database triggers front view

* feat: 🚧 database_triggers

* feat: 🚧 add definition in yaml, updated backend code and added migration

* feat: 🚧 updated migration file, update openapi.yml, updated database_triggers page and backend function

* fix: struct rust

* feat: 🚧 update migrate, database trigger backend function fixed

* feat: 🚧 add resource picker front, update backend function

* feat: 🚧 edit inner database inner, update triggers

* feat: 🚧 database_triggers

* feat: 🚧 update openapi yaml, prettied websocker trigger

* feat: 🚧 database_triggers

* feat: 🚧 add resource module, update variable file, working on main loop for database_trigger

* feat: 🐛 working sqlx query

* feat: 🚧 fix query with sqlx, added main loop

* feat: 🚧 add new column database_trigg

* feat: 🛂 run jobs works

* feat: 🚧 handling slot name and replication

* feat: restructring triggers, decoding trigger message on work

* feat: 🚧 database_trigger

* feat: 🚧

* feat: 🚧 converter done, work on custom script

* feat: 🚧 multiple trigger

* feat: 🚧 adding new argument function

* feat: 🚧 database_trigger

* feat: 🚧 add generate template for front, update script picker

* feat: 🚧 template script fix bug, work on restructing backend logic

* feat: 🚧 update autogenerated script, add persistence state for template script

* feat: 🚧 update structure client

* feat: 🚧 rewrited crud function

* feat: 🚧 added publication handler

* feat: 🚧 new ui finished

* feat: 🚧 added slot function hanlder finish front ux

* feat:  ux improvement done, backend logic done

* feat: 🐛 fix where clause

* feat:

* feat:

* chore: update .sqlx and remove empty package

* fix: update publication and remove unneccessary print

* feat: finish converter to json, remove save button, remove unused crate

* feat: update migration for database trigger, fixed query, fixed frontend error with missing feature on back

* chore: update .sqlx

* chore: update sqlx

* chore: .sqlx

* chore: add unused

* fix: use right database resource in back and front

* fix build

* refactor:

* fix sqlx

* nits: retrieve all script from database trigger editor

* fix: template script object, fix route name

* merge main

* feat: rework ux

* feat/finish ux and update backend logic

* feat: database_trigger

* feat: database_trigger

* chore: update sqlx

* chore: fix ci

* nits: fix worker.rs

* feat: database_trigger

* Update frontend/src/lib/components/triggers/database/DatabaseTriggersPanel.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat:

* fix ci

* nits: database to postgres

* nits: fix .sqlx

* feat: add link to original code and open tab functionallity

* nits: update .sqlx

* nits: fix request type and yaml file

* nits: add abel suggestion

* fix: update .sqlx

* fix: update .sqlx

* nits: add comment to hex.rs

* nits: remove features, rename database to postgres name

* nits: replace database name by postgres

* nits: database name to postgres

* nits: update .sqlx

* nits: update .sqlx

* nit

* nit

* nits

* nit

* nits

* nits

* nits

* nits

* fix sqlx

* nits: update .vscode

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 15:12:41 +01:00
pyranota e47dd697f9 feat(python): Multiple runtime versions (#4579)
* feat: Handle `pip install` by `uv`

Dirty and untested, but already something working

* Integrate with NSJAIL and prepare fallbacks

* Refactor fallback
no_uv disable compile and install
where no_uv_install and no_uv_compile are a bit more specific

* Remove `--disable-pip-version-check`
Reason:
   warning: pip's `--disable-pip-version-check` has no effect

* Fix backend compilation error

* Pip fallback overwrite UV's cache

* Initially refactor cache (No S3)

* Support S3

* Remove unused import

* Handle flags for NSJAIL

* Return deleted flag

* Remove verbose mode and enable link-mode=copy

* Granural migration of lockfiles

Before i realized we dont need it :)

* Initial draft (not-working)

* Add fallback

* Fix bug preventing uv from installing deps

'\n' - Love it

* Add verbosity indicator

* Iterate on feature
- Added instance python version
- Rework logic

* Fix EE build error
error[E0599]: no method named `iter` found for tuple `(PyVersion, std::vec::Vec<std::string::String>)` in the current scope

* Support S3

* Support NSJAIL

* Refactor `get_python`

* Make NSJAIL work [Unsafe]

config file missed /proc mount causing install phase to fail

* Trigger CI

* Clean up

* Make Actions build it

* Trigger CI #2

* Update Dockerfile and clean up

* Change fallbacks
now there is only no_uv and NOUV

* Expose INSTANCE_PYTHON_VERSION through env variable

* Change namings

* Include py-version to requirements.in

Also add comments and make code much cleaner

* Use const for python installation dir

It was hardcoded before

* Pin preinstalled version

* Update python_executor.rs

* Up to date branch

* Create PYCACHE dirs

TODO: PY_TAR_DIRS

* Fix after merge

* Make it safer

* Implement USE_SYSTEM_PYTHON

* Implement latest_stable option

* Load INSTANCE_PYTHON_VERSION on startup

* Check for multiple annotations used

* Fix Latest Stable button not pressed if selected

* Proper error handling for conflict on multiple annotations

* Fix merge conflicts

* Preinstall 3.11 and Latest Stable

* Preinstall latest stable in non-blocking manner

* Fix Warning

* Gate preinstall logic behind "python" feature

* Handle raw_deps properly

* Make it work with nsjail

* Revert docker-image.yml

* Revert Dockerfile

* Cleanup + Fixing

* Add windows support

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-24 11:42:48 +01:00
Alexander Petric a88fbb238a feat: msft teams support for critical alerts (#5113)
* feat: msft teams support for critical alerts

* ee changes

* ee

* sqlx prep

* multiple teams channels

* commit file, not symlink

* improve reactivity

* docs link

* Update ee-repo-ref.txt
2025-01-23 19:57:13 +01:00
Ruben Fiszel ff72f3a9a1 chore(main): release 1.448.1 (#5118)
* chore(main): release 1.448.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-22 18:35:51 +01:00
Ruben Fiszel 737e664792 fix: improve environment variables config 2025-01-22 18:19:37 +01:00
Ruben Fiszel 047c16acd4 chore(main): release 1.448.0 (#5111)
* chore(main): release 1.448.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-22 14:15:32 +01:00
HugoCasa a8fa75a5c3 fix memory limits reading on fly.io/firecracker (#5114) 2025-01-22 12:53:38 +01:00
pyranota fcfad69195 feat: Migrate to bun.lock (#5112)
* bun: Migrate to bun.lock

(In backwards compatible way)

Read more: https://bun.sh/blog/bun-lock-text-lockfile

* Clean up

* More clean up

* Mount bun.lock to jailed process
2025-01-22 12:32:17 +01:00
Ruben Fiszel 669a1ff8e6 fix: improve workspace specific tags behavior 2025-01-21 18:15:34 +01:00
Ruben Fiszel e81c2c983d fix: improve workspace specific tags behavior 2025-01-21 16:24:50 +01:00
Lucas Abel ba711c48df backend: always prioritize flow steps (#5108)
When a flow start, his flow steps have lower priority than already scheduled jobs, hence steps are only executed when the queue is drained which considerably increase flow execution time when queue size is significant.
This add a very low priority to flow steps in order to prioritize finishing a started flow over new jobs.
2025-01-21 09:27:40 +01:00
Ruben Fiszel 2b0c55b32c chore(main): release 1.447.6 (#5098)
* chore(main): release 1.447.6

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-21 09:06:41 +01:00
pyranota 8eb5e72ccd fix(python): Replace Inf with null (#5099) 2025-01-20 19:38:35 +01:00
pyranota 5831822afe fix: improve uv dependency install reliability with an atomic success file
* Revert solution with Mutex

* Implement valid.windmill logic

* Remove unused import

* valid.windmill -> .valid.windmill

Just like .lock

* Dont delete wheels if cancelled/failed

Now we dont clean up requirement folder if it was failed.

This way we can fully utilize uv's flock system.

Also if we left wheel dir, but it was partially filled (resulting to invalid wheel)
we use --reinstall flag in order to overwrite any content of wheels

* Add comment

* Add --reinstall to nsjail
2025-01-20 16:21:10 +01:00
Ruben Fiszel 4301a1758f chore(main): release 1.447.5 (#5082)
* chore(main): release 1.447.5

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-17 20:45:05 +01:00
wendrul 298aaaefa3 fix: c# on windows, oracledb test connection, cli for C# and Oracle DB (#5090)
* Fix c# on windows

* Fix test connection on oracledb

* Add csharp and oracledb to CLI

* Fix missing extensions

* Build wasm for cli
2025-01-17 20:16:22 +01:00
Ruben Fiszel 0d153a0a2d fix: fix workspace specific custom tags 2025-01-17 19:48:38 +01:00
HugoCasa 742d04df38 fix: preprocessor script kind option (#5081)
* fix: preprocessor script kind option

* oups

* oups
2025-01-17 14:10:39 +01:00
HugoCasa 1f53ded622 team plans billing threshold alert (#5078)
* team plans billing threshold alert

* nits

* tmp ee ref

* update ee ref
2025-01-17 14:06:47 +01:00
Ruben Fiszel 12bc86fe2c use tsx loader for relative imports 2025-01-16 18:29:33 +01:00
Ruben Fiszel 797f46c680 chore: update bun to 1.43.0 and loaded from tsx to ts 2025-01-16 18:26:03 +01:00
Ruben Fiszel e33968ad57 chore(main): release 1.447.4 (#5076)
* chore(main): release 1.447.4

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-16 13:45:15 +01:00
HugoCasa b7eeed0729 triggers improvements (#5074)
* triggers improvements

* nits

* nits

* update ee ref
2025-01-16 13:01:45 +01:00
Ruben Fiszel ccf24f1cab nits 2025-01-16 11:51:46 +01:00
Ruben Fiszel 70b6e23f0a chore(main): release 1.447.3 (#5073)
* chore(main): release 1.447.3

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-16 09:35:52 +01:00
Ruben Fiszel 65f42511d9 chore(main): release 1.447.2 (#5069)
* chore(main): release 1.447.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-15 19:37:35 +01:00
Ruben Fiszel 37d0466d9e chore(main): release 1.447.1 (#5065)
* chore(main): release 1.447.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-15 18:31:08 +01:00
HugoCasa 4a68ce9ac3 fix: improve runnable permissions (#5063) 2025-01-15 18:25:12 +01:00
HugoCasa 2b0878ec2b fix(backend): bun cache hash consistency (#5067) 2025-01-15 18:24:10 +01:00
HugoCasa bc3d93b935 fix(backend): better deleted user error (#5060)
* fix(backend): better deleted user error

* nit
2025-01-15 17:02:10 +01:00
Ruben Fiszel 3f4f0772e3 chore(main): release 1.447.0 (#5061)
* chore(main): release 1.447.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-15 11:21:10 +01:00
wendrul 77d825540f feat: add oracle db support on ee (#5062) 2025-01-14 21:21:24 +01:00
HugoCasa 04fbda28e8 feat: on behalf of (#5058)
* feat: on behalf of

* fix

* revert mistakke

* temp ee ref

* renaming + CLI

* sqlx

* nits + ee ref
2025-01-14 18:32:25 +01:00
HugoCasa 8c2f2ebb1e fix: update ms sql template (#5059) 2025-01-14 17:48:12 +01:00
Ruben Fiszel 2509affc65 chore(main): release 1.446.0 (#5054)
* chore(main): release 1.446.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-14 13:49:36 +01:00
Ruben Fiszel 427aae6164 chore(main): release 1.445.1 (#5050)
* chore(main): release 1.445.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-01-13 15:28:27 +01:00
Guilhem ebfde197fc fix(ui): capture v2 UX (#4954)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* Add history tab

# Conflicts:
#	frontend/src/lib/components/EditableSchemaForm.svelte

* add workflow_dispatch to build

* add workflow_dispatch to build

* Move input copy from in a panel

* better capture UI

* Add smoothing animation on open input editor

* fix bad wrapping in input editor menu

* clean code

* restore captureTable

* clean

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* add capture pagination

* Add capture table

* Add table for history inputs

* nit

* simplify capture table

* stop capturing on closing capture section

* show only 10 items per page for captures

* set focus to logs on script test

* Fix collapsable section

* nit

* open captures when applying args from triggers

* update edit input drawer

* fix all EditableSchemaForm

* fix panel init animation

* Add capture popover

* change open tab button

* clean code

* fix bad table display

* Fix JSON input bad sync

* make capture icon bigger

* nit

* nit

* revert unwanted change

* Change capture name and nits

* infinite scroll and cleaning

* Add toast when failing to retreive first step input

* add icons to dropdown edit input list

* remove unused log

* Add click outside to all schema pickers

* fix click oustide

* Fix migrations

* solve run button not refreshing properly

* Add a capture drawer

* change apply args and schema logic

* change apply args and schema logic

* fix toggle display

* fix schema display

* improve preview mode

* Add first step panel

* nit

* clean

* Add hint for captures

* clean

* nit

* nit

* fix script node page height

* improve animation on redirecting to capture

* improve saved input picker

* nit

* add Json placeholder

* nit

* add list animations

* delete useless logs

* add animated list to captures history and saved inputs

* Delete unused component

* clean

* nit

* clean code

* improve tab scroll experience

* nit

* nits

* nits

* fix script args update

* nit

* nit: include preview in history inputs in edit mode

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-13 14:45:38 +01:00
HugoCasa 96d31f9577 prevent cancelling schedule next tick + remove need of flow_status inside add_completed_job (#5022) 2025-01-13 14:26:28 +01:00
HugoCasa a3aa091d1e fea(backend): return 500 on sync webhook/route error (#5049)
* fea(backend): return 500 on sync webhook/route error

* fix sqlx
2025-01-13 12:25:06 +01:00
Ruben Fiszel 7255ad8154 chore(main): release 1.445.0 (#5035)
* chore(main): release 1.445.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2025-01-10 20:01:53 +01:00
HugoCasa e66fcf9270 feat: nats triggers (#5039)
* feat: nats triggers

* missing files

* fix build

* update feature flags

* event stream fixes

* update ee ref
2025-01-10 19:47:17 +01:00
HugoCasa 7b808c3977 fix(backend): multiple routes with same path but different methods (#5040) 2025-01-09 19:08:56 +01:00
Alexander Petric 0c391372cc feat(frontend): allow workspace admin to set workspace color (#5032)
* feat(frontend): allow workspace admin to set workspace color

* sqlx prep

* sqlx prep

* make border wider, add removed comment
2025-01-09 06:42:36 +01:00
Alexander Petric e47a3aa7c7 nit: always show flow meta data, append user provided msg (#5026) 2025-01-06 22:59:16 +01:00
Ruben Fiszel ec2ff527d0 chore(main): release 1.444.0 (#5017)
* chore(main): release 1.444.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <rubenfiszel@users.noreply.github.com>
2025-01-06 13:59:13 +01:00
HugoCasa 01c6f84520 feat: captures (#4807)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* add workflow_dispatch to build

* add workflow_dispatch to build

* better capture UI

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* improve migration + http capture fixes

* fix sqlx

* update ee ref

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-06 12:07:09 +01:00
Lucas Abel 8586476a13 backend: do not force get_schedule_opt to be within a transaction (#5019) 2025-01-06 09:36:57 +01:00