Commit Graph

940 Commits

Author SHA1 Message Date
pyranota d3e0b1c636 feat: Allow setProgress and getProgress from within the script (#4400)
* Allow setting progress explicitly from script body.

This feature exposes:
 * `getProgress`
 * `setProgress`
 * `incProgress`

API in TypeScript client (python is coming soon).

NOTE: Progress cannot be out of range 0..100 and cannot decrease.

With exposed APIs there is also UI changes, so progress can be shown for individual jobs as well.

For optimization reasons, jobs start to ask for progress only after N-seconds of execution.

* feat: Add `shell.nix`

If you dont have anything but nix, dont worry, run nix-shell in root, or activate with direnv and get all needed dependencies

NOTE: You will still need docker

* feat: Add `dev.nu` to typescript client

Little helper function, allowing developer to work on ts client easier.

To use:
  `./dev.nu watch`

Now add import of windmill in body of your script and `//nobundle` on top of the file

Edit ts client in your favourite editor and hit save. Script will do the rest.

* Cleanup files

* Fix: Failed to deserialize query string: missing field `get_progress`

* perf: Implement non-naive polling mechanism for getting job progress

* Add independant delay for getProgress

Problem in `TestJobLoader`:

There should be 2 delays:

    One until we find our first progress (every 5s)
    Once we found our first progress, we can do it every second

* nit: Use `query_scalar!` instead of `query_as`

* Fix: Sql error, no rows returned by a query that expected to return at least one row

* refactor: Remove global CSS for JobProgressBar

* Change UI for progress of flow subjobs

* Replace `Step 1` with `Running` in ProgressBar for individual jobs

* Remove `incProgress`

incProgress is not very usefull and error-prone

* perf: Set metric only for jobs that are actually using it

(https://github.com/windmill-labs/windmill/pull/4373#discussion_r1759843773)

* Offload registering progress from clients to server

* Add `jobId?` argument to typescript-client's `setProgress` and `getProgress`

Allows to set progress of other jobs and flows,
if jobId specified, than flow id will be inferred automatically.

Could be used by SDK.

* Add `Error::MetricNotFound` for better error handling

* Fix: Make `JobProgressBar` display in red when failed

* Add persistant progress bar

Now you can reload the page after job is done and progress will be still there

* Allow succeeded individual job's progress bar stick to 100%

* Add python support 

* nit: Remove usage of undefined variable in python-client

* Add `async` in ts client (for error handling)

* nit(frontend): Remove unused import

* Dont load JobProgressBar when it is not needed

* nit: npm check fix

* cargo sqlx prepare

* fix sqlx

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-09-18 01:16:19 +02:00
HugoCasa d3e8731be2 feat: failed key renewal alert + renew on start if no recent renewal (#4387)
* feat: failed key renewal alert + renew on start if no recent renewal

* Update ee-repo-ref.txt
2024-09-16 16:54:35 +02:00
Ruben Fiszel d533a017a7 fix: improve first time setup experience 2024-09-15 21:02:36 +02:00
HugoCasa 6ca628c811 fix: queue metrics graph (#4388) 2024-09-13 23:34:48 +02:00
Ruben Fiszel 458dcfe1e8 fix: handle better same_worker flow monitor 2024-09-11 01:29:40 +02:00
HugoCasa a009c8df23 feat: schedule success handler (#4346) 2024-09-06 19:03:09 +02:00
Ruben Fiszel 84e18c73c0 fix: improve wm_labels indices + UX nits 2024-09-04 11:40:23 +02:00
HugoCasa c487934e3b feat: add slack as a critical alert channel (#4319)
* feat: add slack as a critical alert channel

* update ee ref
2024-09-04 09:48:13 +02:00
Ruben Fiszel c429e09211 nits zombie job log improvement 2024-09-01 10:36:53 +02:00
Ruben Fiszel c93ccb6f55 improve service logs with json_fmt 2024-08-31 23:04:12 +02:00
Ruben Fiszel c62e152d27 feat: service logs (#4244) 2024-08-31 14:57:35 +02:00
HugoCasa 6915ae6bfe fix: prevent duplicate worker alerts (#4309) 2024-08-31 13:27:59 +02:00
HugoCasa c3ce68066c feat: min workers in worker group alert + zombie job critical alert (#4307)
* feat: min workers in worker group alert + zombie job critical alert

* updatee ee ref
2024-08-30 17:25:53 +02:00
wendrul 91d328b132 feat: add rust (#4253)
* Add rust on frontend

* Add parser for rust

* Handle rust job execution

* Main has to return a result that is serializable

* Update parser: parse many rust types

* Rust parser dependencies

* Frontend wasm parser

* Add windmill parser rust to windmill parser wasm (temporarily)

* Add rust as a tag on worker__default

* Change init code for rust

* Cleanup rust_executor.rs

* Remove tree-sitter dep and fix unused imports

* Add lockfile and build logic

* Add cargo and rustup to windmill image

* Fix env var to work on docker image and dev env

* modify package.json

* Deps after parser publish

* Add stashed migration

* Unify rust versions

* Add error message when php or cargo are not present to use another image

* Error message conditionally on feature flag

* all ci/cd changes

* all ci/cd changes

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-30 17:01:04 +02:00
HugoCasa 5a300fb3e2 feat: show last job instead of current job on workers page (#4293)
* feat: show last job instead of current job on workers page

* fix: keep old sql column name
2024-08-27 22:04:02 +02:00
HugoCasa 9efad477da optional starred info on get item + update name of user instance (#4295) 2024-08-27 21:58:51 +02:00
Ruben Fiszel 7235aa5d65 fix: improve password field lifetime incorrectly recycled too early 2024-08-16 17:28:10 +02:00
HugoCasa 069dfe9601 sqlx for refresh cgroup readings (#4251) 2024-08-16 14:11:56 +02:00
Ruben Fiszel 8e9aeefbc2 fix compile 2024-08-13 17:06:51 +02:00
Ruben Fiszel 8950aae44e feat(frontend): improve display of waiting jobs + schedule filter + suspended jobs on runs page 2024-08-13 10:41:36 +02:00
HugoCasa be48eaaeff fix: handle >1 num workers stats (#4177)
* fix: handle >1 num workers stats

* fix: sqlx

* fix: ee ref test

* fix: sqlx + ee ref

* update ee ref
2024-08-07 17:52:39 +02:00
HugoCasa fba0817dd5 fix: email triggers sqlx CE and improve email parsing (#4203)
* fix: CE sqlx

* update ee ref
2024-08-06 23:05:36 +02:00
wendrul af0e901954 feat: deployment UI filter deployable items (#4183)
* Add column to workspace settings table

* Add endpoint to change the deploy UI settings

* Make frontend page to edit UI settings

* Prepare sqlx

* Add deployment restrictions to frontend

* Fix function name

* Change dependency to minimatch, CE compatibility

* Remove default include_path

* Remove picomatch types (old dep)

* Use empty list instead of globstar as default

* All deployable if config is null
2024-08-05 13:27:13 +02:00
Ruben Fiszel 9dc47d7bd9 update sqlx 2024-08-01 18:19:21 +02:00
Ruben Fiszel 69e400563c feat: variables created by password fields expire after 7 days 2024-08-01 16:05:03 +02:00
Ruben Fiszel 78c115c68c fix: improve index usage and runs page performance 2024-07-31 13:08:28 +02:00
Ruben Fiszel 340868d282 feat: remove nativets in favor of bun with native pragma 2024-07-29 19:26:26 +02:00
Ruben Fiszel d00cb20375 improve slightly concurrency run min started_at query 2024-07-27 12:43:24 +02:00
Ruben Fiszel 6df0b31d20 improve slightly concurrency run min started_at query 2024-07-27 12:28:55 +02:00
Ruben Fiszel 68c13e5702 chore: update sqlx to 0.8.0 2024-07-24 11:51:54 +02:00
wendrul bad678be43 Prepare sqlx (#4122) 2024-07-23 20:03:30 +02:00
wendrul f98518e241 Prepare sqlx (#4121) 2024-07-23 19:54:31 +02:00
HugoCasa b4bfc404f0 fix: sqlx build (#4120) 2024-07-23 17:14:36 +02:00
HugoCasa ecf4706784 feat: rehydrate instance settings/configs/users from CLI (#4035)
* feat: rehydrate instance settings/configs/users from CLI

* fix: nits

* feat: encryption key, saved instances, include all workspaces

* preshot client version

* Update deps.ts
2024-07-15 19:38:44 +02:00
Ruben Fiszel 7f2cbb40f6 feat: improve flow status viewer (show branch chosen + all iterations in for loop) (#4074)
* all

* all

* all

* all

* all
2024-07-13 18:53:24 +02:00
Ruben Fiszel 45c895d72d fix sqlx 2024-07-11 22:40:41 +02:00
Ruben Fiszel 2cb137c067 feat: improve cancel all for non started jobs (#4065)
* update version

* improve cancelAll

* improve error

* update sqlx
2024-07-11 17:11:29 +02:00
wendrul af444c8b36 feat: Full-text search on runs using tantivy and command palette for quick actions (#4046)
* Add indexer crate and files

* POC searcher

incomplete schema
only indexes at startup

* POC search component frontend

* Demo of the frontend element

* add Results and Args as text

* minimal functionality

* Make jump to scripts by name

also flows and apps

* Add button on sidebar to open search

* Update lock on indexer after merge

* Make arrow key navigation compatible with scrol

* Show empty result screen and log as a coming feat

* Add summary to script searchable items

* Catch `parts is undefined` error (uFuzzy)

* Index refreshing using tokio interval

* Fix JobLoader workspace being wrongly defined

* Fix click outside

* Add debouncing for completed run search

* Binary mode working + job index tracker

* Warning for no license + fix height scrollbars on content search

* Make it compile without EE files

* remove panic to use errors

* Move global search

* Cleanup UI, no more tab switcher but clear placeholders and actions

* Add tantivy feature flag for windmill-api

* Rework indexer mode

* Mac compatibility for shortcut

* Update test for new run_server

* Prepare sqlx

* Mac compatibility

* Fix openapi yaml

* Fix frontend

* Frontend api fix

* Update docker-compose.yml and caddyfile

With the (by default deactivated) container and reverse proxy to use the
windmill indexer

* fix feature flag for tests

* fix feature falg for running tests

* fix feature flag for running tests

* Make content search use search modal instead

* Add tantivy feature to ee build steps

* Remove old Content search

* change volume location for indexer

* Update dependencies

* Prepare sqlx

* Uncomment line on docker compose

* Add line between input and results

* Update ee repo ref
2024-07-11 11:13:27 +02:00
Ruben Fiszel be7e7ba05c fix: fix migration for instances starting from scratch 2024-07-11 02:12:51 +02:00
Ruben Fiszel a443368d41 specialize soft cancel filter 2024-07-11 01:27:56 +02:00
Ruben Fiszel 9e27ee8b57 improve try migration lock 2024-07-11 01:15:12 +02:00
Ruben Fiszel 7fe93fb506 fix: improve performance of cancel_all 2024-07-11 00:41:03 +02:00
Ruben Fiszel 2836eaf765 sqlx 2024-07-11 00:12:11 +02:00
HugoCasa bbcaec9769 fix: improve flow version fix migration (#4050) 2024-07-09 11:02:53 +02:00
HugoCasa ff8d51a8db fix: always run flow versioning edge case fix migration (#4047) 2024-07-08 22:52:04 +02:00
HugoCasa af3fd09d80 fix(backend): make value of flow_version not null (#4039)
* fix: flow versioning nit: make value of flow version not null

* fix: sqlx

* fix: missing code change

* fix: improve nit migration
2024-07-08 11:56:53 +02:00
Ruben Fiszel a3f40bf0e4 fix sqlx 2024-07-08 11:55:59 +02:00
HugoCasa 0a613b76c8 fix: flow versioning edge case (#4037) 2024-07-07 20:04:42 +02:00
Ruben Fiszel 0ada83a80a fix: fix wrong interaction between suspended steps and forloop parallelism 2024-07-04 14:53:43 +02:00
HugoCasa 7f2289d4d5 feat: flow versioning (#4009)
* feat: flow versioning

* fix: sqlx

* fix: update schedule test for flow versioning

* fix: with_deployment_msg + UI nits

* fix: nit

* fix: improve down migration

* patch: keep latest flow version in flow table for backward compat

* fix: app deployments in list view

* chore: update ee ref

* fix: merge

* fix: tests
2024-07-03 15:53:45 +02:00