Commit Graph
437 Commits
Author SHA1 Message Date
wendrul 299c1430eb feat: backend arg schema validation (#5455)
* Make schema validation struct

Schema Validation rules that are constructed from the schema or from the
MainArgSig(TODO).

* Make other validator builder

* Fail dependency job like with lockfile failing for schema validator

* Add last types + tests

* Remove unused dependency

* fix typos

* Migration ID was colliding with another, changed it manually

* Add Oneof + other fixes

* fix: cache for querying scripts correclty handles ScriptMetadata

* Add cache for schema validation from main arg sig

* Prepare sqlx

* Remove default features

* Feature flags

* Fix down migration table name

* cleanup: put validation logic inside a function

* Refactor to cache the should_validate boolean

Changed the schemavalidators cache to take in an
Option<SchemaValidator>, effectively storing the `should_validate_schema` information.

Also pass the schema when avaialble to construct the schema validator

* Add other job kinds to u8 cache key just in case

* Only cache if not preview
2025-03-19 19:41:10 +01:00
HugoCasaandRuben Fiszel 884aba4811 perf: improve perf of get completed flow node (#5418)
* perf: improve perf of get completed flow node

* better

* nit

* nits

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-03-19 10:51:30 +01:00
Ruben Fiszel eefb77f445 nit move away from using views 2025-03-19 10:48:43 +01:00
Ruben Fiszel b8d7092478 fix: fix delete completed job 2025-03-19 10:10:16 +01:00
Ruben Fiszel aaa4cde349 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel 5134deefad improve concurrency limits II (#5485) 2025-03-17 17:24:50 +01:00
Ruben Fiszel fcaee6fef5 fix: improve concurrency limit check performances 2025-03-17 00:53:14 +01:00
Ruben Fiszel fe6c2abd59 fix: improve row lock contention on concurrency counter 2025-03-16 17:20:39 +01:00
HugoCasa 1e1bc56f33 feat: keep captures across drafts and deploys (#5482) 2025-03-14 15:17:48 +01:00
dieribaandHugoCasa df4b6db0d4 feat: add raw string option and wrap option for http trigger (#5467)
* add new column

* handled worskpaced route

* update .sqlx

* update: rename migration file, rewrite query for checking existing route path key, add copy button url for each route trigger

* update .sqlx

* update sql query

* update .sqlx

* fix: prevent precedence on query

* nits

* update .sqlx

* update query

* Update http_triggers.rs

* nits

* handling two new option in http route 'wrap_body' and 'raw_string'

* update sqlx and add wrap_body and raw_string to http route

* fix: undefined route path

* nits: remove unnecessay ref as type implement copy

* nits

* fix: undefined protection

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-03-13 20:39:54 +00:00
dieribaandHugoCasa 04ea7b851b feat: option to prefix http route with workspace id (#5461)
* add new column

* handled worskpaced route

* update .sqlx

* update: rename migration file, rewrite query for checking existing route path key, add copy button url for each route trigger

* update .sqlx

* update sql query

* update .sqlx

* fix: prevent precedence on query

* nits

* update .sqlx

* update query

* Update http_triggers.rs

* nits

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-03-11 15:24:22 +00:00
Ruben Fiszel 5765d4cf75 nit zombie job monitoring 2025-03-11 07:08:28 +01:00
Ruben Fiszel 00d5c39347 improve slightly index creation 2025-03-11 06:35:57 +01:00
dieriba ba0cfb199c use-exists-function-for-folder-and-improve-error-handling (#5464)
* update: removing try and catch for checkExistalready fn and use exist function folder instead of get folder for folder kind

* update .sqlx
2025-03-10 12:38:11 +01:00
Alexander Petricandellipsis-dev[bot] cc599a6727 feat(frontend): pick image from workspace storage bucket (#5382)
* feat(frontend): pick image from workspace storage bucket

* also upload

* update policy for unauthed s3 download

* sqlx prep

* sqlx prep

* force policy

* no need for ee

* image picker

* Update frontend/src/lib/components/apps/editor/settingsPanel/InputsSpecEditor.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* field not needed

* feature flag

* filter for image files

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-06 11:53:53 +01:00
0955af1aea feat: MQTT triggers (#5277)
* first commit

* update migration mqtt

* chore: update dependencies

* add cli support and update schema for mqtt

* update: casing style

* update: rename get_resource function and moving to resource.rs

* feat:

* feat: refactored

* feat: add ready endpoints for workers to enterprise

* feat: handle connection to server

* fix: main.rs

* feat: added support v5

* feat: 🚧 updated .github, handle more case for v5 and handle v3

* feat: handled client, update script helper

* update .sqlx

* refactor: added generic implement and trait to remove redundant code, misses client impleme

* feat: remove optinal property update test function

* feat: done

* handle client certificate

* minor fix

* fix: tls and mtls

* fix: min topic len in frontend

* Update frontend/src/lib/components/triggers/mqtt/MqttEditorConfigSection.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-api/openapi.yaml

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-api/src/mqtt_triggers.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* update openapi, and script_helper.ts

* fix: toggle button and update mqtt migration

* nits: remove postgres type and only relies on serde

* fix: timeout connection

* fix typo and captures

* fix: capture table reactivity

* handle more mqtt v5 option

* nits: remove clone method

* update: better naming

* nits: display tooltip qos

* fix: hanle tls connection with natively

* update: add documentation links for qos and topics

* fix tooltip and add default value

* update: shoz preprocessor only if selected in capture table

* nits: show mqtt icon in resource related componenets

* update hub link and repo ref

* fix: unused var

* update ee ref

* fix: give good reference id for script hub

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Clement Zhang <clement.zhang2@gmail.com>
2025-03-04 23:52:47 +01:00
Alexander Petricandellipsis-dev[bot] 8ac8ed4a98 feat(backend): option to invalidate all sessions on logout (#5419)
* feat(backend): option to invalidate all sessions on logout

* lazy static

* Update backend/.sqlx/query-8cccb31aa56af16af675c692bf4e14b297d2caeafd1c4e4b1584f3bc9ff4c563.json

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* sqlx

* sqlx

* all -> invalidate-all

* add audit log

* rename env var

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-03-04 23:47:33 +01:00
Alexander PetricandRuben Fiszel 13253b2d6e feat: more controls on setting token duration (#5421)
* allow setting max session length

* more options for expiration

* sqlx

* option to invalidate all old sessions on new session

* sqlx update script on mac

* order

* add audit log

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-03-04 20:08:42 +01:00
Ruben Fiszel d7879e237b fix: improve cancel performance 2025-03-03 23:50:13 +01:00
Ruben Fiszel c5b8fc0e6a fix sqlx 2025-03-03 14:51:51 +01:00
HugoCasa fd3459568f feat: track workspace runnables used in flows (#5369)
* feat: track workspace runnables used in flows

* track script hash

* weird

* do it with lock

* Revert "feat: add support for | None and Optional in python (#5361)"

This reverts commit 69a2d85f25.

* Revert "Revert "feat: add support for | None and Optional in python (#5361)""

This reverts commit bb8f709894.

* update openapi

* delete old in lock_modules + don't track hub scripts
2025-02-27 10:01:12 +01:00
Ruben Fiszel f7865229ef feat: limit the number of times a job can be restarted (3) after loss of pings 2025-02-25 16:35:29 +01:00
Ruben Fiszel afa3beb953 bench.sh 2025-02-24 01:20:20 +01:00
Ruben Fiszel 3d6015e44a feat: make flow lock deployment error visible in UI 2025-02-22 15:02:16 +01:00
Alexander Petric 70ab21dd85 teams cleanup cleanup (#5330)
* sqlx

* repo ref

* adding teams channel to criticalerrorchannel in non-ee
2025-02-19 19:01:18 +01:00
c1351f8155 feat: SQS triggers (#5182)
* feat: first commit

* fix: npm check

* fix: openapi file

* feat: update openapi and migration

* feat: basic implementation done

* fix: fix: no used function when no feature

* feat: capture done

* Update capture.rs

* nits: change sqs trigger

* fix: make migration great again

* feat: add message attributes

* feat: nits: fix error messages, remove console.log and add try catch

* update sqs icon and ee feature for sqs_trigger

* update: change sqs name casing and added test connection button

* nits: update Icon and add create from template button

* fix: ci build and error compilation

* update migration type sqs

* update link on create from template button for sqs, add archive in workspace export and update sqlx

* fix: ci

* Update SqsTriggerEditorInner.svelte

* add link to docs, use generic function for resource and fix import error

* chore: update .github ci

* nits: remove empty

* update to match ee repo changement

* Update backend/windmill-api/src/resources.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* add sqs handling for the cli and refacoring sqsEditorInner

* Update cli/sync.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix: add break to switch statement for sqs case

* fix: display aws_resource_path when retrieve or create a new trigger

* rework sqs ui, fix postgres optional port

* fix: ci

* update ui for trigger

* update repo ref and specific

* feat: add ready endpoints for workers to enterprise

* update ref

* Update frontend/src/lib/script_helpers.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-02-18 23:49:45 +01:00
Ruben Fiszel d8d33a9151 fix: fix teams cleanup preventing start 2025-02-18 10:25:42 +01:00
Ruben Fiszel 60a3cc2e65 make benchmarks even more stable 2025-02-18 08:43:03 +01:00
Ruben Fiszel 283fd376cd nit benchmarks 2025-02-18 01:54:14 +01:00
Ruben Fiszelandrubenfiszel 2af6128d6e chore(main): release 1.463.4 (#5309)
* chore(main): release 1.463.4

* Update CHANGELOG.md

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-02-18 00:55:01 +01:00
Alexander Petric fbe96b68fe fix: improve teams settings in workspace settings (#5316) 2025-02-17 22:44:26 +01:00
Ruben Fiszel a034c3ff28 fix: improve que job indices for faster performances 2025-02-17 21:26:46 +01:00
Ruben Fiszel 3b75187cf5 fix: improve que job indices for faster performances 2025-02-17 19:17:30 +01:00
Ruben Fiszel 84ff1981da fix: improve que job indices for faster performances 2025-02-17 17:56:28 +01:00
Ruben Fiszel d3a4e39bc0 fix: improve runs page load time 2025-02-14 13:30:07 +01:00
Ruben Fiszelandrubenfiszel 147b6e9d9d chore(main): release 1.462.2 (#5286)
* chore(main): release 1.462.2

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
2025-02-14 04:38:09 +01:00
Ruben Fiszel 3926cc2c43 fix: ai_models in workspace_settings is now optional 2025-02-14 02:45:13 +01:00
Ruben Fiszel c42815c387 fix extra_perms for schedule light 2025-02-13 21:34:53 +01:00
Ruben Fiszel 81c2ca9ad6 workspace colors fix 2025-02-13 20:23:12 +01:00
Alexander PetricandRuben Fiszel 66702cd97f feat: teams workspace scripts (#5238)
* backend and clients

* adding teams_team_name to workspace_settings

* openapi.yaml adding teams workspace settings endpoints

* teams workspace settings frontend

* workspaces router

* build ce

* ee gate

* ee oauth

* update client

* workspace error handler

* remove log

* ce

* point to new hub scripts

* updating hubPaths

* updating hubPaths

* cleanup

* merge

* schedule teams error

* polish, reactivity

* sqlx compilewarning

* sqlx migrate

* make it build

* router

* fix

* remove sqlx workaround

* Update ee-repo-ref.txt

* simplify some logic

* sqlx

* latest ee ref

* latest ee ref

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-02-13 16:53:57 +01:00
Ruben Fiszel 271ccca710 fix: better handling of null pre-processor return values 2025-02-12 22:10:53 +01:00
Lucas AbelandRuben Fiszel ecbec1fe50 fix(backend): improve schedule queries plan to leverage indices better for performance (#5273)
* backend: fix schedule queries plan

* rework

* fix skipped as non success

* update sqlx

* update

* all

* fix npm run check

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-02-12 16:34:03 +01:00
f3e91d714d feat: add postgres trigger captures (#5165)
* feat: 🚧 add postgres trigger config in back

* feat: add capture section for postgres

* feat: update postgres trigger panel, update capture migration

* fix: ci and casing style

* feat: capture done

* fix: update sqlx

* fix: remove unused

* feat: handle persisting data

* fix: add feature flag

* feat: capture done

* fix: feature flag

* feat: capture done

* nits: add plus into postgres capture label

* feat: trigger done

* Update PostgresTriggerEditorInner.svelte

* Update script_helpers.ts

* nit

* chore: update to meet pr comment requirement

* refactor:

* fix:

* fix: miss closing curly braces

* fix:

* fix: ci

* Update mod.rs

* polishing postgres trigger

* update minor changes on UI

* nits: remove log and update function parameters

* fix: remove wrong name

* nits: change icon

* fix: nit migration

* nits: refacto and remove alert

* nits: change variable name

* update workspaces_export

* fix: ci

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Guilhem <guilhemlemouel@gmail.com>
2025-02-11 18:16:12 +01:00
HugoCasaandRuben Fiszel 9617add8ac feat: triggers cli sync (#5243)
* feat: triggers cli sync

* fix build

* update ee ref

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-02-10 15:14:28 +01:00
Ruben Fiszel 5f46d79f26 fix: fix concurrent limit jobs non restarting 2025-02-10 11:33:19 +01:00
Ruben Fiszel 7bf5e381ac fix: worker name in job + better timeout handling for same_worker jobs (#5248) 2025-02-08 22:20:06 +01:00
HugoCasa 95e9491185 fix: workflow as code status (#5246)
* fix: get update endpoint

* [WIP] fix: workflow as code

* fix fix

* Revert "fix: get update endpoint"

This reverts commit 7af9abf868.

* fix test
2025-02-08 07:09:14 +01:00
HugoCasa 6162ea81f1 fix: get update endpoint (#5245) 2025-02-07 19:23:38 +01:00
Lucas Abel 17a86e6907 backend: fix workflow as code (#5239)
* backend: improve `/get_job_update` after v2

* backend: insert missing `workflow_as_code_status` on completion

also insert `flow_status` from so we can remove the query when `_duration` is above 500

* backend: fix workflow_as_code after v2

* backend: add `workflow_as_code` worker test
2025-02-07 12:27:43 +01:00
HugoCasa e2186e704b feat: serve static websites (#5218)
* feat: serve static websites

* nit

* sqlx

* nit

* nits

* nits

* nits

* nits
2025-02-06 17:38:12 +01:00