Commit Graph
240 Commits
Author SHA1 Message Date
Ruben Fiszel 7537f1a1d7 feat(backend): cache hub scripts in the worker cache 2023-05-09 12:45:54 +02:00
Ruben Fiszel d814a25b0f add cairo to openbb build 2023-05-09 11:11:09 +02:00
Ruben Fiszel 6b5ceed652 feat(backend): in python, if a value is bytes, it is encoded to base64 automaticaly 2023-05-09 09:36:07 +02:00
Ruben Fiszel 13242abff1 feat: add support for full fleged apps (react, svelte, vue) (#1536)
* add react support

* use whoami

* update

* update

* update

* create react app

* all

* sqlx
2023-05-08 19:10:45 +02:00
Ruben Fiszel 2a958a9d1d fix default value in python 2023-05-05 10:03:28 +02:00
Ruben Fiszel 8791a86a93 feat(backend): default parameters are used in python if missing from args 2023-05-05 09:44:53 +02:00
Ruben Fiszel a07e3e8438 fix(backend): fix python transformers 2023-05-03 23:27:14 +02:00
Ruben Fiszel de9abd129d feat(backend): add EXIT_AFTER_NO_JOB_FOR_SECS for ephemeral workers 2023-04-28 00:09:48 +02:00
Ruben Fiszel 324d4f5e9e feat(backend): expose tag in the job (#1486)
* fix

* update tag

* update tag

* update tag

* fix
2023-04-27 19:09:53 +02:00
Ruben Fiszel 0b7bad3816 feat(backend): add JOB_RETENTION_SECS to delete completed jobs completed after expiration period 2023-04-26 19:34:54 +02:00
Ruben Fiszel 722783f7f6 feat(backend): worker groups (#1452)
* tagged scripts

* tagged scripts

* fix

* fix

* fix

* all

* update sqlx
2023-04-23 16:57:23 +02:00
Ruben Fiszel d76e90757e fix(backend): avoid potential conflict between pull from tar and background sync 2023-04-21 23:49:14 +02:00
Ruben Fiszel 5f8a730fdf fix(backend): global cache synco only start if all piptars have been downloaded 2023-04-21 22:21:04 +02:00
Ruben Fiszel 3d031c7017 feat(backend): only run fully deployed scripts 2023-04-21 12:49:47 +02:00
Ruben Fiszel 670ba51d9b feat(backend): workers are instantly ready and sync with global cache in background 2023-04-21 09:45:25 +02:00
Ruben Fiszel 68ab75169a improve log line for cache syncing 2023-04-21 01:54:58 +02:00
Ruben Fiszel 455e2e9c25 fix untar_all_piptars 2023-04-21 01:02:51 +02:00
Ruben Fiszel ddc04f93c3 better rclone filters 2023-04-21 00:31:39 +02:00
Ruben Fiszel fac32f3ebf better rclone filters 2023-04-20 23:45:31 +02:00
Ruben Fiszel ed87020a83 set relative folder for archive creation 2023-04-20 23:33:53 +02:00
Ruben Fiszel 1f304a7c84 remove deno/gen/windmill from tmp cache 2023-04-20 21:04:17 +02:00
Ruben Fiszel 96f79bc6cc --s3-no-cache-buckets 2023-04-20 20:50:02 +02:00
Ruben Fiszel 369dd0dac6 feat(backend): global cache refactor for pip using tar for each dependency (#1443)
* cache refactor

* exclude tar from being synced to bucket

* run

* update

* update
2023-04-20 20:05:12 +02:00
Ruben Fiszel 7f886a673b fix python install on save with nsjail 2023-04-20 10:33:11 +02:00
Ruben Fiszel 62d196ecec add barrier when num workers > 1 2023-04-19 17:00:03 +02:00
Ruben Fiszel 29848ae466 add barrier when num workers > 1 2023-04-19 12:46:25 +02:00
Ruben Fiszel c919827cf8 feat(ee): sync cache in background 2023-04-19 01:15:41 +02:00
Ruben Fiszel 0e77e37fbd feat(ee): sync cache in background 2023-04-19 00:55:51 +02:00
Ruben Fiszel fa2e38a2f2 only cache on first worker 2023-04-18 21:23:29 +02:00
Ruben Fiszel 6954d0882e log how much time took each command child 2023-04-18 08:55:26 +02:00
Ruben Fiszel 26a86ba0d8 improve bash output buffering 2023-04-18 01:42:28 +02:00
Ruben Fiszel 33437bb847 performance: process completed job in background (#1396)
* compile

* improve
2023-04-15 11:06:51 +02:00
Ruben Fiszel cb7e686dd9 feat(backend): install python scripts on save 2023-04-13 22:57:53 +02:00
Ruben Fiszel 11b9c8fadb do a test run on first iteration 2023-04-13 21:54:42 +02:00
Ruben Fiszel 34f48aa1e3 split worker.rs into more files 2023-04-13 17:44:47 +02:00
Ruben Fiszel 8928cfa3d7 fix refreshOn 2023-04-13 09:22:52 +02:00
Ruben Fiszel a5499c26f3 fix(backend): deno uses --no-check 2023-04-13 00:22:46 +02:00
Ruben Fiszel a5f6d73f7d fix(backend): do not fail on schedule not existing anymore 2023-04-12 20:06:50 +02:00
Ruben Fiszel 670c84b901 exclude deno/gen/file from global cache 2023-04-12 19:18:48 +02:00
Ruben Fiszel 3932e5dfb9 add GOPRIVATE and NETRC for private modules 2023-04-11 19:55:38 +02:00
Ruben Fiszel 8552c92837 fix relative url deno loading2 2023-04-11 15:57:59 +02:00
Ruben Fiszel 955a213a50 fix(backend): nested deno relative imports 2023-04-11 15:16:55 +02:00
Kai JellinghausandRuben Fiszel d45e6c94ab feat(backend): Redis based queue (#1324)
* Merge?

* Fix V8 breaking change

* WIP

* WIP

* Cleanup

* Move to git reference

* Fix Merge conflict

* update

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2023-04-11 12:24:13 +02:00
Ruben Fiszel 0718931616 fix tests + scheduled_for uses now instead of system time 2023-04-11 11:48:14 +02:00
Ruben Fiszel df4c6289ac improve metrics 2023-04-10 11:58:44 +02:00
Ruben Fiszel 72c8d3921d improve metrics 2023-04-10 11:53:32 +02:00
Ruben Fiszel e911869d99 perf: parallelize more operations for deno jobs 2023-04-10 10:55:56 +02:00
Ruben Fiszel e1712e63a6 improve token lock 2023-04-10 08:00:41 +02:00
Ruben Fiszel de87d7ac27 fetch token in the background 2023-04-10 01:16:15 +02:00
Ruben Fiszel 2b003c684f make create_token_for_owner not a transaction anymore 2023-04-09 21:47:05 +02:00