Commit Graph

349 Commits

Author SHA1 Message Date
HugoCasa 46194d5944 ee compute units telemetry (#4865)
* ee compute units telemetry

* update ee ref
2024-12-06 16:52:03 +01:00
Lucas Abel ef08fc8e29 feat: add db storage for app inline scripts (#4837) 2024-12-06 13:37:45 +01:00
Lucas Abel c0b93f9c97 fix: handle flow & workspace renames for flow_node (#4861) 2024-12-06 09:40:32 +01:00
Lucas Abel 5417ac5f07 fix: fix flow_node uniqueness (#4850)
`jsonb` comparison wasn't working as expected, and duplicated entries
were inserted within `flow_node`. To resolve this add a second hash
column, `hash_v2` with a unique default for uniqueness, and use this new
column to ensure unique entries. The previous hash column is left for
backward compatibility. Duplicated entries already insterted will remain
as is without breaking, and only new ones will preserve uniqueness.
2024-12-05 17:42:33 +01:00
Guilhem e2d1565749 polish critical alert modal (#4802)
* polish critical alert modal

* Use Table component

* use table component pagination

* Add acknowledge to the table

* Polishing table

* fix nit

* Filter alerts

* Add notification count on modal

* Adjust table height

* Change mute description

* Fix layout

* minor fix

* Fix small screen issue

* fix toast on refresh

* Revert "fix toast on refresh"

This reverts commit ae3593e1af.

* filtering to backend, superadmin also acknowledges workspace (unless CLOUD_HOSTED), simplifications

* sqlx prep

* improve reactivity

* improvements

---------

Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <petric.al@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-12-05 11:07:11 +01:00
HugoCasa af4123f17f feat: app custom paths (#4828)
* feat: app custom paths

* nit

* make ee only + fix sqlx

* fix: custom http routes auth

* nits

* fix auth + nits

* apps_ee

* move custom path to ee

* fix app jwt

* update ee ref
2024-12-04 16:50:17 +01:00
Ruben Fiszel 61d31cc9a7 improve draft only delete security 2024-12-03 23:13:35 +01:00
HugoCasa e89d5f7b9f apply jwt ext tag scopes to single job endpoints (#4824)
* apply jwt ext tag scopes to single job endpoints

* fix sqlx
2024-11-29 19:23:20 +01:00
wendrul 6987a36846 feat: indexer extra settings + parallel downloads + many improvements (#4822)
* Nits on the service logs page

* Show all the hosts returned by query + sumOtherDocCount warning

* Remove from index endpoint

* Fix tests

* Prepare sqlx
2024-11-29 19:14:05 +01:00
Lucas Abel 0a65b90049 feat(cache): implement flow node caching (#4808)
* feat(cache): implement flow node caching

* feat(cache): implement script caching

* feat(cache): improve cache
2024-11-29 09:35:00 +01:00
Lucas Abel 995edad334 fix: invalid null comparison while inserting flow node (#4815) 2024-11-28 17:29:16 +01:00
Lucas Abel 7079b030eb fix: invalid jsonb comparison while inserting flow node (#4814) 2024-11-28 14:43:54 +01:00
wendrul db4172439c fix: service logs: aggregation query + killpill on index pull + retention period for logs on s3 (#4795)
* Search across all hosts in index. killpill on index pull

* Add retention period for service logs

* Remove console log

* monitor log_file when older than biggest retention period (job / log)

* Retention period deletes service and job logs

* Add setting to monitor logs out of s3

* Prepare sqlx

* Update ee repo ref

* Prepare sqlx

* Unused variables
2024-11-27 18:57:23 +01:00
Lucas Abel 4e52e54314 feat(backend): store flow inner modules into flow_node table (#4778) 2024-11-27 16:08:31 +01:00
Lucas Abel 41a7fbb80f fix: disable flow_version_lite in dedicated workers (#4800) 2024-11-27 10:02:44 +01:00
Lucas Abel ce121fcf37 feat(backend): implement flow scripts (#4748)
* feat(backend): implement flow scripts

* conditionally resolve reduced flow value

* conditionally use flow scripts based on the min worker version

* grant all to `flow_{version_lite,node}`

* do not silently discard errors

* env: add flow script feature flag

* flow script on cloud automatically

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-25 23:20:51 +01:00
Lucas Abel d84d578181 fix: rename job workspace as well (#4785) 2024-11-25 08:09:30 +01:00
Ruben Fiszel e54e8ab405 fix: retry on inserting completed job (#4784)
* fix: retry on pushing next scheduled job of schedule

* all

* improve error handling

* update sqlx
2024-11-25 01:33:44 +01:00
wendrul 9f34712bd7 feat: Add a devops role to act as a "readonly admin" (#4775)
* Prepare sqlx

* Add devops role as bool

* Add devops reloe and use it for certain endpoints

* Critical alerts: complete devops role visiblity

* Update ee repo ref

* Prepare sqlx
2024-11-22 14:55:48 +01:00
Alexander Petric 54b419d8ab feat(schedule): support for extended cron syntax (#4754)
* init

* croner with fall back on scheduling failure

* adding packages

* fallback mechanism v1

* removing debug console log

* improving cargo organization

* queue cargo.toml

* use workspace cargo-tz

* test next occurrence when creating new croner schedule

* toggle label

* treat legacy edit requests where no cron_version is provided as v1

* fix edit schedule
2024-11-21 19:24:38 +01:00
Lucas Abel 002f0b92ea backend: rework raw values fetching logic (#4770) 2024-11-21 13:59:42 +01:00
Lucas Abel 40cc9cc007 fix: garbage collect job table + delete leaked ones (#4767) 2024-11-21 09:02:24 +01:00
wendrul 6e70df1db8 feat: Indexer improvements: s3 backup logic reworked, settings on the frontend (#4763)
* Add settings page + common code for indexer

* Prepare sqlx
2024-11-20 22:40:36 +01:00
Ruben Fiszel ae7835ce9c fix: prevent groups to be ill-defined with non writer owners 2024-11-19 20:13:54 +01:00
Lucas Abel b5f411a8ce feat(backend): move some static fields out of job tables (#4689)
* feat(backend): move some static fields out of job tables

* backend: re-use `flow_value` if any and avoid clone

* update minimal version to 1.427.0
2024-11-19 11:27:55 +01:00
HugoCasa 2a3c0d55fd feat: kafka triggers (#4713)
* feat: kafka triggers

* sqlx

* fix build

* improve error messages on windows

* nit

* fix build

* missing action

* nit

* maybe fix ssl on windows

* nit

* update ee ref

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-18 17:03:50 +01:00
Ruben Fiszel 663aad02dd fix error logs on empty get_logs 2024-11-16 09:30:36 +01:00
Alexander Petric 5b39f8f0ba feat(monitoring): workspace critical alerts (#4684)
* critical alert ui

* updating ui, backend logic

* revert

* type check fix npm

* checking out cli files from main

* moving alert icon

* adding sqlx mock data

* more sqlx changes

* feat(frontend): nodes from flow can be connected directly in expr input through a plug icon (#4652)

* Add flow prop picker

# Conflicts:
#	frontend/src/lib/components/propertyPicker/PropPicker.svelte

* fix unwanted copy

* cleaning

* Fix unset context

* move button and always display input

* fix unwanted proppicker display

* update

* update

* clean all

* clean all

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>

* replace hide/show with toggle

* adding mutable setting and navigation to settings to configure channels

* merge fix

* ee non ee changees

* auto-acknowledge when muted

* pr comments

* fix bad log

* user inner modal component

* update unaknowledge alerts after acknowledging from modal

* aknowledge -> acknowledge

* format

* adding backend support for workspace critical alerts

* immediately check for alerts

* immediately check for alerts

* adding openapi changes

* simplify loading of superadmin/ee

* update modal logic

* frontend logic update

* sqlx prepare

* adding mute functionality for workspace critical alerts + show alerts button for instance and workspace settings

* npm / rust warnings & errors

* reverting non-mac cargo toml

* reverting non-mac cargo toml

* mute toggles in critical alert ui

* ui polish

* adding acknowledged_workspace column

* sqlx prepare

* make sure we wait for all stores to be loaded

* adding workspace mute logic to report_critical alert

* auto ack recovered alerts

* toggle workspace as superadmin and critical alerts in logs menu

* adding critical alert button if width < 786px

* refresh on change

* Move notification to logs (#4698)

* Move notification to logs

* removing unused button

---------

Co-authored-by: Alexander Petric <petric.al@gmail.com>

* tailwind typo

* auto ack on mute

* don't change deref

* sqlx prep

* avoid renaming db column, keep acknowledged instead of acknowledged_global

* hide critical alert menu when not ee

* if workspace muted, also acknowledge global when workspace set

* removing save button for mute setting and improve ux/responsiveness

* fix: deployment callbacks have a concurrency limit of 1 on same path

* sqlx prep

* ee-repo ref

* z-[9999] for modal

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-15 09:48:48 +01:00
dieriba bcc96d7e6a feat: Support mistral anthropic for ai (#4692)
* wip: openai proxy and other ai proxy integration

* fixing migration script

* wip: support different ai provider in front, fix proxy openai

* wip: adding frontend ai provider

* updated copilot types

* wip: working on anthropic integration

* done AI proxy front

* adding new type and support for anthropic

* updating gitignore

* adding streaming response

* added streaming prompt

* push lib/gen

* wip: fixing anthropic

* anthropic fully supported

* fix backend missing var error and fully support stream event for anthropic

* remove gen directory

* fixing openapi file

* add support for mistral, and update create workspace components

* remove deref.json

* remove package-json

* openapi

* fix ui enable code

* added utility function for init workspace ai provider

* fix workspace switch bug

* update anthropic property and fixed frontend error

* fix workspace settings

* update error message and fix typo migration file

* chore: update openapi file

* fix dev file

* add .sqlx

* all

* update sqlx

---------

Co-authored-by: dieriba <t.dieriba@gmail.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-11-14 14:24:35 +01:00
Ruben Fiszel 0be1a3a09d feat: allow setting password and login type from superadmin UI 2024-11-14 13:22:48 +01:00
Lucas Abel 47424b1446 feat(backend): monitor minimal version of living workers (#4704) 2024-11-14 12:33:50 +01:00
Lucas Abel 247eb5dd90 nit: cleanup raw_flow usage (#4707)
* nit: cleanup `raw_flow` usage

* nit: refactor two queries into one
2024-11-14 12:23:27 +01:00
Lucas Abel 483283894e nix: nit fixes + bench script (#4703) 2024-11-14 09:08:11 +01:00
Lucas Abel 2529161f3c more benchmark methods (#4695)
* feat(backend): add `rawscript` to `add_batch_jobs` API

* bonus: improve `add_batch_jobs` API performances for `flow` kind

* add `bigrawscript` and `bigscriptinflow` to `benchmark_oneoff.ts`

* add `bigrawscript` and `bigscriptinflow` to the bench suite
2024-11-13 15:53:01 +01:00
Alexander Petric f870518d94 feat(frontend): critical alerts UI (#4653) 2024-11-09 00:42:10 +01:00
HugoCasa c686cc437d feat: http custom routes for static assets (#4666)
* feat: http custom routes for static assets

* update ee ref + fix build

* fix build

* nit
2024-11-07 20:16:14 +01:00
HugoCasa 5f1f16f1a0 feat: websocket authentication (#4635)
* feat: websocket authentication

* npm check, sqlx and preprocessor template

* better run job

* nit
2024-11-05 11:54:08 +01:00
wendrul 36ea8a21d6 feat: Add full-text search on windmill service logs (#4576)
* Add service_log indexer file (ee)

* Rename to completed_run

* rename to service_logs

* Update lib.rs

* Make indexing of logs and simple frontend

* Add common ee file

* Update lib.rs

* Make log search global to fix openapi unused path param

* Prepare sqlx

* Show last indexed also when no jobs are found

* Adapt feature flags to service logs indexer

* Update api and main

* Build service log search page

* scope query to the min and max time

* Migrate modal, bug fixes

* Remove unused import

* Make the service logs page take the full screen

* improve quick access menu to service_logs, fix date stuff, hide 0 matches

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-11-05 09:52:07 +01:00
HugoCasa 7fdb524fc6 improve key renewal (#4519)
* improve key renewal

* sqlx and fix

* fix build

* improve everything

* nits

* nits

* update ee ref

* nit
2024-10-29 09:26:45 +01:00
Ruben Fiszel 85de44cb93 feat: autoscaling v0 (#4593)
* all

* all

all

all

all

all

all

all

all

all

* all

* all

* nits

* nits
2024-10-28 00:47:19 +01:00
Ruben Fiszel 2cbf027653 fix: do not update created_at of scripts on lockfile generation 2024-10-25 17:44:41 +02:00
HugoCasa 1741ce7823 fix ws killpill (#4551)
* websocket killpill

* oups

* nit

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-19 01:17:15 +02:00
HugoCasa b0fbcd8305 feat: websocket triggers (#4505)
* feat: websocket triggers

* fix

* frontend nits

* nits

* new trigger UI

* sqlx
2024-10-18 00:21:52 +02:00
wendrul bfc1f53cea Win 465 index observability 1 (#4538)
* Factor isJobCancellable function in a utils file

* Add message for last indexed job and warning of deployement

* Prepare sqlx

* Update ee repo ref
2024-10-16 19:25:50 +02:00
Faton Ramadani 258445e523 feat(frontend): unify all triggers UX and simplify flow settings (#4259)
* feat(frontend): added list of triggers in the flow graph

* feat(frontend): added list of triggers in the flow graph

* feat(frontend): clean up

* feat(frontend): improve UX

* feat(frontend): triggers

* feat(frontend): triggers

* feat(frontend): done

* feat(frontend): fix trigger when position when a preprocessor is presetn

* Glm/rework flow settings v2 (#4497)

* fat(frontend): simplify flow settings menu

* improve scroll

* changing mute toggle

* Add advanced settings badge

* Add nord theme colors

* Add bage for advanced options

* fix minor issue

* fix minor issue

* Add triggers menu to flow settings

* Add quick trigger access

* remove triggers in flow settings

* fix minor issue

* Move triggers settings to flow right panel

* polishing

* fix unset store

* remove save up to for triggers

* fix padding

* reset default tag color

* remove custom select component

* revert path change

* revert section modif

* Revert unused feature

---------

Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>

* Connect top bar cron to schedules settings

* Turn copilot into node

* fix copilot placement

* remove useless import

* fix center copilot

* fix binding

* remove copilot on top of preprocessor

* render copilot node on condition

* quickfix

* remove copilot node

* fix minor issues

* fix route count update

* fix schedule sync

* harmonize colors

* fix alignment and add edges

* recenter node summary

* fix schedules sync

* Add id title

* all

* all

* all

* iteration

* all

* all

* done

* fix

* more fixes

---------

Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
Co-authored-by: Guilhem <guilhem@mbp-de-windmill.home>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-10-16 17:58:15 +02:00
wendrul 98a64188f5 fix: Add indexer backup lock to fit the deployment model (#4531)
* Pass db to init_index

* Prepare sqlx

* Update ee repo ref
2024-10-16 14:36:12 +02:00
Ruben Fiszel ba9c51a1b0 sqlx 2024-10-09 21:17:01 +02:00
HugoCasa 8841180b5f BYPASSRLS admin policy falllback (#4493)
* BYPASSRLS admin policy falllback

* use fixed list instead

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-08 20:08:29 +02:00
HugoCasa f61c8f45a4 fix sqlx skip step (#4466) 2024-10-01 13:34:31 +02:00
HugoCasa 41ff40f777 send stats when renewing key if last >24h (#4430)
* feat: send stats when renewing key if last >24h

* nits

* fix: sqlx

* nit

* renewal reason

* stats reason

* update ee ref

* Update ee-repo-ref.txt

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-09-30 12:23:07 +02:00