Commit Graph

374 Commits

Author SHA1 Message Date
HugoCasa 1f53ded622 team plans billing threshold alert (#5078)
* team plans billing threshold alert

* nits

* tmp ee ref

* update ee ref
2025-01-17 14:06:47 +01:00
HugoCasa b7eeed0729 triggers improvements (#5074)
* triggers improvements

* nits

* nits

* update ee ref
2025-01-16 13:01:45 +01:00
wendrul 77d825540f feat: add oracle db support on ee (#5062) 2025-01-14 21:21:24 +01:00
HugoCasa 04fbda28e8 feat: on behalf of (#5058)
* feat: on behalf of

* fix

* revert mistakke

* temp ee ref

* renaming + CLI

* sqlx

* nits + ee ref
2025-01-14 18:32:25 +01:00
Guilhem ebfde197fc fix(ui): capture v2 UX (#4954)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* Add history tab

# Conflicts:
#	frontend/src/lib/components/EditableSchemaForm.svelte

* add workflow_dispatch to build

* add workflow_dispatch to build

* Move input copy from in a panel

* better capture UI

* Add smoothing animation on open input editor

* fix bad wrapping in input editor menu

* clean code

* restore captureTable

* clean

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* add capture pagination

* Add capture table

* Add table for history inputs

* nit

* simplify capture table

* stop capturing on closing capture section

* show only 10 items per page for captures

* set focus to logs on script test

* Fix collapsable section

* nit

* open captures when applying args from triggers

* update edit input drawer

* fix all EditableSchemaForm

* fix panel init animation

* Add capture popover

* change open tab button

* clean code

* fix bad table display

* Fix JSON input bad sync

* make capture icon bigger

* nit

* nit

* revert unwanted change

* Change capture name and nits

* infinite scroll and cleaning

* Add toast when failing to retreive first step input

* add icons to dropdown edit input list

* remove unused log

* Add click outside to all schema pickers

* fix click oustide

* Fix migrations

* solve run button not refreshing properly

* Add a capture drawer

* change apply args and schema logic

* change apply args and schema logic

* fix toggle display

* fix schema display

* improve preview mode

* Add first step panel

* nit

* clean

* Add hint for captures

* clean

* nit

* nit

* fix script node page height

* improve animation on redirecting to capture

* improve saved input picker

* nit

* add Json placeholder

* nit

* add list animations

* delete useless logs

* add animated list to captures history and saved inputs

* Delete unused component

* clean

* nit

* clean code

* improve tab scroll experience

* nit

* nits

* nits

* fix script args update

* nit

* nit: include preview in history inputs in edit mode

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-13 14:45:38 +01:00
HugoCasa 96d31f9577 prevent cancelling schedule next tick + remove need of flow_status inside add_completed_job (#5022) 2025-01-13 14:26:28 +01:00
HugoCasa a3aa091d1e fea(backend): return 500 on sync webhook/route error (#5049)
* fea(backend): return 500 on sync webhook/route error

* fix sqlx
2025-01-13 12:25:06 +01:00
HugoCasa e66fcf9270 feat: nats triggers (#5039)
* feat: nats triggers

* missing files

* fix build

* update feature flags

* event stream fixes

* update ee ref
2025-01-10 19:47:17 +01:00
HugoCasa 7b808c3977 fix(backend): multiple routes with same path but different methods (#5040) 2025-01-09 19:08:56 +01:00
Alexander Petric 0c391372cc feat(frontend): allow workspace admin to set workspace color (#5032)
* feat(frontend): allow workspace admin to set workspace color

* sqlx prep

* sqlx prep

* make border wider, add removed comment
2025-01-09 06:42:36 +01:00
HugoCasa 01c6f84520 feat: captures (#4807)
* feat: captures

* flow UI and improvements

* fix: build

* fix sqlx

* Move Capture WIP

* Add capture to webhook and websocket

* Move connection status viewer in the head

* change trigger section label

* Add popover capture picker using melt ui

* Add shortcut to triggers capture from input form

* remove capture tab in input

* Allways show capture

* remove useless logs

* Add email capture

* Add kafka capture into triggers

* Add edit option in capture table

* use light header for arg input

* Add prefilled group id

* Change button label

* fix logic

* Open resource drawer if prototype has fields

* fix default completion

* Change name Prototype

* Dissociate Editor Mode for triggers

* Fix bug for script

* Fix apply args

* fix apply schema

* Add capture table to script

* fix apply args

* Add capture button for script

* Delete capture tab

* Set capture on when opening triggers capture

* fix connection indicator

* fix minor issues

* Add preprocessor logic

* Use slot in log Panel for captures

* handle capture refresh in script

* Delete capture tab from script editor

* reset kafka resource on toggle static

* fix minor issue

* Allow resource in kafka capture

* use simple capture button in flow

* Remove capture panel

* Polish route trigger editor

* Fix resource saving

* Remove excessive padding

* merge nits

* add workflow_dispatch to build

* add workflow_dispatch to build

* better capture UI

* fix sqlx

* fix build

* fix build

* build

* make initial_messages optional in line with db

* fix npm check

* better handle args for capture webhook and http

* improve migration + http capture fixes

* fix sqlx

* update ee ref

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-01-06 12:07:09 +01:00
HugoCasa 0c19171f57 feat: allow s3 file download/preview from inside apps (#5004)
* feat: allow s3 file download/preview from inside apps

* improve security + handle image preview

* fix build

* fix build
2025-01-04 01:47:57 +01:00
Lucas Abel f5c85d7db9 fix(apps): check for auth when executing as publisher (#4979) 2025-01-03 16:47:15 +01:00
Alexander Petric 2832b887a5 interactive slack improvements (#5001)
* refactor / rust style

* add detailed description to default message

* default args, dynamic enums

* sqlx prep

* cancel / close behavior
2025-01-03 16:27:03 +01:00
Lucas Abel eeece84a4b fix: handle 'flownode' job kind where missing (#4990) 2024-12-30 19:10:47 +09:00
Alexander Petric 6308bf0dcb feat: interactive slack approvals (#4942)
* feat: interactive slack approvals

* move form creation logic to backend

* adding python client

* polish messages

* date time picker and default values

* Initial commit

* Initial commit

* refactor

* cleanup

* cleanup

* cleanup

* cleanup, treating numbers/integers/booleans

* adding slack to ts dependencies

* sqlx prepare

* gitignore, adding package.lock

* ellipsis comments

* Update typescript-client/client.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* refactor to move everything serverside + modal

* sqlx prep

* reverting slack dependency

* python client update

* fix build errors

* Update pyproject.toml

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2024-12-20 21:47:45 +01:00
Lucas Abel 83d24cb9ab cache: better cache system (#4951)
* cache: consider source as untrusted

* cache: better scoped cache

* cache: non-lazy flow value parse error

* fixup! cache: consider source as untrusted

* fixup! cache: consider source as untrusted

* fixup! cache: better scoped cache
2024-12-20 11:19:57 +01:00
Lucas Abel fb536df066 fix: fix redeploying flows with attached schedules 2024-12-18 18:56:30 +01:00
Lucas Abel 51b74e713c cache: improve error reporting (#4940) 2024-12-17 19:44:51 +00:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
Ruben Fiszel 9cdf90c50f update sqlx 2024-12-14 13:27:42 +01:00
pyranota 7398c1c09c Display memory peak while installing wheels with uv (#4889)
* fix no_uv not affecting deploy

Before this fix no_uv, no_uv_compile and no_uv_install were not affecting Dependency jobs

These jobs are only affected if used USE_PIP_COMPILE or USE_PIP_INSTALL env variables

To make it more consistant, no_uv should also affect dep jobs.

Also make ansible use uv by default

* Make it build

* Add no_uv_usage stats

* Provide build-env for Samael through shell.nix

* Run update_sqlx.sh

* Update ee-repo-ref

* Display memory peak while installing wheels with uv

* Invert DISABLE_NSJAIL

* Make it safer

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-12-12 11:26:02 +01:00
pyranota 3618e26326 Add no_uv_usage stats (#4883)
* fix no_uv not affecting deploy

Before this fix no_uv, no_uv_compile and no_uv_install were not affecting Dependency jobs

These jobs are only affected if used USE_PIP_COMPILE or USE_PIP_INSTALL env variables

To make it more consistant, no_uv should also affect dep jobs.

Also make ansible use uv by default

* Make it build

* Add no_uv_usage stats

* Provide build-env for Samael through shell.nix

* Run update_sqlx.sh

* Update ee-repo-ref

* Update sqlx cache

* Update ee-repo-ref

* Update ee repo ref

* Update sqlx cache

* Update ee-repo-ref.txt
2024-12-12 00:32:03 +01:00
Lucas Abel af5cca1b00 feat(cache): re-work job results cache (#4898) 2024-12-11 12:01:48 +01:00
Ruben Fiszel 96714b8b9b update sqlx 2024-12-10 00:42:37 +01:00
HugoCasa 8e9cb9c4d3 ee compute units telemetry (#4865)
* ee compute units telemetry

* update ee ref
2024-12-06 16:52:03 +01:00
Lucas Abel 2bc4934c4f feat: add db storage for app inline scripts (#4837) 2024-12-06 13:37:45 +01:00
Lucas Abel f175158b9f fix: handle flow & workspace renames for flow_node (#4861) 2024-12-06 09:40:32 +01:00
Lucas Abel 667167a022 fix: fix flow_node uniqueness (#4850)
`jsonb` comparison wasn't working as expected, and duplicated entries
were inserted within `flow_node`. To resolve this add a second hash
column, `hash_v2` with a unique default for uniqueness, and use this new
column to ensure unique entries. The previous hash column is left for
backward compatibility. Duplicated entries already insterted will remain
as is without breaking, and only new ones will preserve uniqueness.
2024-12-05 17:42:33 +01:00
Guilhem e888c84a36 polish critical alert modal (#4802)
* polish critical alert modal

* Use Table component

* use table component pagination

* Add acknowledge to the table

* Polishing table

* fix nit

* Filter alerts

* Add notification count on modal

* Adjust table height

* Change mute description

* Fix layout

* minor fix

* Fix small screen issue

* fix toast on refresh

* Revert "fix toast on refresh"

This reverts commit ae3593e1af.

* filtering to backend, superadmin also acknowledges workspace (unless CLOUD_HOSTED), simplifications

* sqlx prep

* improve reactivity

* improvements

---------

Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <petric.al@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-12-05 11:07:11 +01:00
HugoCasa 1ec6c6f765 feat: app custom paths (#4828)
* feat: app custom paths

* nit

* make ee only + fix sqlx

* fix: custom http routes auth

* nits

* fix auth + nits

* apps_ee

* move custom path to ee

* fix app jwt

* update ee ref
2024-12-04 16:50:17 +01:00
Ruben Fiszel 71fc31c471 improve draft only delete security 2024-12-03 23:13:35 +01:00
HugoCasa 8b20530584 apply jwt ext tag scopes to single job endpoints (#4824)
* apply jwt ext tag scopes to single job endpoints

* fix sqlx
2024-11-29 19:23:20 +01:00
wendrul 088c666a7f feat: indexer extra settings + parallel downloads + many improvements (#4822)
* Nits on the service logs page

* Show all the hosts returned by query + sumOtherDocCount warning

* Remove from index endpoint

* Fix tests

* Prepare sqlx
2024-11-29 19:14:05 +01:00
Lucas Abel 3fbb2bfc8a feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Lucas Abel 390ee3113b fix: invalid null comparison while inserting flow node (#4815) 2024-11-28 17:29:16 +01:00
Lucas Abel 77937a494a fix: invalid jsonb comparison while inserting flow node (#4814) 2024-11-28 14:43:54 +01:00
wendrul 09fe535bdc fix: service logs: aggregation query + killpill on index pull + retention period for logs on s3 (#4795)
* Search across all hosts in index. killpill on index pull

* Add retention period for service logs

* Remove console log

* monitor log_file when older than biggest retention period (job / log)

* Retention period deletes service and job logs

* Add setting to monitor logs out of s3

* Prepare sqlx

* Update ee repo ref

* Prepare sqlx

* Unused variables
2024-11-27 18:57:23 +01:00
Lucas Abel 2911775d0a feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Lucas Abel 150ca33f0d fix: disable flow_version_lite in dedicated workers (#4800) 2024-11-27 10:02:44 +01:00
Lucas Abel e4784e89da feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Lucas Abel 29faefe856 fix: rename job workspace as well (#4785) 2024-11-25 08:09:30 +01:00
Ruben Fiszel 6768e5bbbd fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
wendrul 4facf3ca3e feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Alexander Petric 8909bea935 feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
Lucas Abel e6c7fe7027 backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
Lucas Abel c99d360c3c fix: garbage collect job table + delete leaked ones (#4767) 2024-11-21 09:02:24 +01:00
wendrul 8f198ba68c feat: Indexer improvements: s3 backup logic reworked, settings on the frontend (#4763)
* Add settings page + common code for indexer

* Prepare sqlx
2024-11-20 22:40:36 +01:00
Ruben Fiszel f59c0c0076 fix: prevent groups to be ill-defined with non writer owners 2024-11-19 20:13:54 +01:00
Lucas Abel 97457a5679 feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00