Commit Graph

357 Commits

Author SHA1 Message Date
Ruben Fiszel 3805398d1c nits supabase 2023-07-27 17:54:43 +02:00
Ruben Fiszel af7c92ac16 feat: worker group for flows 2023-07-25 12:41:51 +02:00
Jakub Kołodziejczak 223b80768a chore(backend): upgrade sqlx to ^0.7 (#1865)
* DRAFT chore(backend): upgrade sqlx to ^0.7

related to:
* https://github.com/windmill-labs/windmill/pull/1858
* https://github.com/launchbadge/sqlx/issues/1163#issuecomment-1627685514

* (vol. 2) in 0.7, `Transaction` can no longer implement `Executor` directly

ref: https://github.com/launchbadge/sqlx/blob/afb6b1066e61f8e3875f530d96cfb5a299f13fda/examples/postgres/transaction/src/main.rs#L14-L17

notice that I'm temporarly using my custom patch
https://github.com/mrl5/sqlx/commit/16e4c9a8f380214ad61b1744d3e1f5fa866ffbcc
it's related to https://github.com/launchbadge/sqlx/issues/2611

* post git rebase chores

* use upstream fix from 0.7.1

* fix compile

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2023-07-15 14:06:20 +02:00
Ruben Fiszel f539dbe298 feat: add jumpcloud support for sso 2023-07-12 19:29:40 +02:00
Ruben Fiszel f55198766e fix tests 2023-07-12 17:31:12 +02:00
Ruben Fiszel d36a9d19aa fix: make workers bind their http servers on any available port for OCI compliance 2023-07-12 11:19:01 +02:00
Guillaume Bouvignies 0327b85f89 feat: Per script concurrency limit with time window (#1816)
* Add concurrency a per-script concurrency limit

* Add a per-script per time window concurrency limit

* Improve feature with a better estimation of the next schedule timestamp

* Add concurrency limits for inline scripts

* Fix tests

* run sqlx prepare

* Add requeue event to job logs
2023-07-11 23:35:26 +02:00
Guillaume Bouvignies 7f3c7ea9d1 Specific NOOP jobs benchmark (#1810)
* Refactor and fix NOOP benchmark

* Add endpoint to toggle workers on/off when compiled in benchmark mode

* Improve noop benchmark
2023-07-10 14:16:34 +02:00
Ruben Fiszel 734f75d493 feat: step mocking for flows 2023-06-22 12:01:08 +02:00
Ruben Fiszel 027fa6ff09 feat: add cache as a primitive for flows (#1671)
* feat: add cache as a primitive for flows

* fix failure module
2023-06-04 14:39:10 +02:00
Ruben Fiszel 9ea18b4593 feat(backend): add job_id as a query arg to force set the new job_id 2023-05-15 10:50:43 +02:00
Ruben Fiszel 080e5e962c fix tests 2023-05-03 14:31:54 +02:00
Ruben Fiszel d998a0c9bf fix tests 2023-05-03 00:38:43 +02:00
Ruben Fiszel 73c5a7473d feat: add drafts for apps 2023-05-01 18:13:57 +02:00
Ruben Fiszel 02ba82064f feat(backend): worker groups (#1452)
* tagged scripts

* tagged scripts

* fix

* fix

* fix

* all

* update sqlx
2023-04-23 16:57:23 +02:00
Ruben Fiszel 48b2e7d1d2 fix tests 2023-04-20 20:17:03 +02:00
Ruben Fiszel f0b939cd3a add barrier when num workers > 1 2023-04-19 17:00:03 +02:00
Ruben Fiszel 26c7e9e8c3 add barrier when num workers > 1 2023-04-19 12:46:25 +02:00
Ruben Fiszel 7c58c3cd0c fix refreshFlow loop 2023-04-16 00:40:54 +02:00
Ruben Fiszel be41e45130 split worker.rs into more files 2023-04-13 17:44:47 +02:00
Kai Jellinghaus 63102b036c feat(backend): Redis based queue (#1324)
* Merge?

* Fix V8 breaking change

* WIP

* WIP

* Cleanup

* Move to git reference

* Fix Merge conflict

* update

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2023-04-11 12:24:13 +02:00
Ruben Fiszel 66d2ae813a fix tests + scheduled_for uses now instead of system time 2023-04-11 11:48:14 +02:00
Ruben Fiszel a23a14f9f2 make create_token_for_owner not a transaction anymore 2023-04-09 21:47:05 +02:00
Ruben Fiszel adcfab6194 fix backend tests 2023-04-09 11:20:15 +02:00
Ruben Fiszel 41fea4b07e fix input_transform -> input_transforms tests 2023-03-31 11:27:37 +02:00
Ruben Fiszel 77079b4b68 consistent sort + minor nits 2023-03-30 17:27:21 +02:00
Ruben Fiszel 784bc8139f fix tests 2023-03-28 21:40:36 +02:00
Ruben Fiszel 2dfbf5ffcf fix autogen script names 2023-03-28 21:07:21 +02:00
Ruben Fiszel 0263c20a5b introduce root_job and leaf_jobs for efficient result_by_id 2023-03-03 12:44:44 +01:00
Ruben Fiszel 69d4a23b4f send a webhook on new user + lazy static refactor (#1203)
* supercharge

* supercharge

* progress

* progress

* display config

* display config

* display config

* display config

* fix extensions

* fix build

* disable nsjail = false for test
2023-02-15 10:59:50 +01:00
Ruben Fiszel ae23830f2a implement MAX_FREE_CONCURRENT_RUNS 2023-01-25 12:58:29 +01:00
Ruben Fiszel 7276244d68 improve cancel approval error message 2023-01-24 12:11:39 +01:00
Ruben Fiszel a71e6f27ec various fixes 2023-01-17 11:30:22 +01:00
Ruben Fiszel 41ff90d41d fix tests wrt to new error handling 2023-01-14 18:47:37 +01:00
Ruben Fiszel a14cc80dfe improve error handling 2023-01-14 15:08:47 +01:00
Ruben Fiszel 8d4b085c54 fix backend tests 2023-01-11 14:56:05 +01:00
Ruben Fiszel 61b3aff0c9 fix backend tests 2023-01-09 14:51:27 +01:00
Ruben Fiszel a5be91c812 fix backend tests 2023-01-09 14:00:17 +01:00
Ruben Fiszel 59a3ba0a25 add visible_to_owner 2022-12-15 16:01:17 +01:00
Ruben Fiszel a8c098736b fix fixtures 2022-12-12 22:33:24 +01:00
Ruben Fiszel 039e1fa1e6 feat: implement usage tracker + quotas 2022-12-11 18:47:39 +01:00
Kai Jellinghaus b87fc7796a feat(backend): Input transform errors (#1006)
* Add pre run error & propagate input transforms there

* Handle pre-run errors

* Update SQLX data

* Update SQLX data
2022-12-08 23:56:49 +01:00
Ruben Fiszel 018f8896bc feat: add include_header to pass request headers to script 2022-12-05 23:36:31 +01:00
Ruben Fiszel 3153ec481f fix deno_flow_same_worker test 2022-11-27 00:50:09 +01:00
Kai Jellinghaus d33d465d3a fix(backend): bash flow lock & add flow lock tests (#933)
* Fix Bash flow lock

* Tests w/ fixes

* Add Sequence privileges
2022-11-23 19:17:19 +01:00
Kai Jellinghaus 7efdb3af6b feat(backend): sync cache features on all workers [enterprise] (#907)
* Periodic Job

* Add write lock & bash execution

* Use env for bash script

* Enterprise Feature

* Only time based

* Cleanup

* Move code

* Use rclone sync

* WIP

* Use buildarg

* Remove unecessary deps

* 10 Minute Intervals

* Revert testing change

* Manually pull rclone

* fix rclone

* Initial sync

* Update worker.rs

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-11-18 17:20:39 +01:00
Kai Jellinghaus 937e16c2af feat(backend): Flow lock (#868)
* Add lockfile information

* Allow Lockfiles to be pushed to queue

* Handle raw_lock on worker

* Flow Lock Job

* Allow RawScript path

* Simplify handle_code_execution_job

* Regenerate SQLX

* Fix tests

* fix test

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2022-11-13 18:31:08 +01:00
Ruben Fiszel 791c0635d5 tweak identity to extract previous_result 2022-11-10 08:48:06 +01:00
Ruben Fiszel cd8d0e5ea6 feat: deprecate previous_result in favor of results per id 2022-11-10 01:58:52 +01:00
Ruben Fiszel 7c97fac746 feat: support bash as 4th language (#865)
* bash support

* bash backend working

* frontend part

* bash backend working
2022-11-06 23:48:18 +01:00