Commit Graph

612 Commits

Author SHA1 Message Date
dieriba c4178c05be feat: periodic worker group script (#6159)
* feaat periodic script

* feat periodic script

* fix unused

* check minimun time

* fix unused import

* update repo ref

* update ref

* nits and update repo ref

* add missing checks

* fix

* improve UX

* improve UX

* improve UX

* fix

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-21 22:57:56 +00:00
dieriba dd1f520d83 fix: improve ssh agent worker naming (#6211)
* remove ssh suffix

* use hostname tags

* feat

* feat

* update sqlx

* use hostname

* use worker prefix

* nits
2025-07-21 13:43:25 +00:00
Ruben Fiszel dec72e201b feat: use sse for flow status updates 2025-07-19 14:42:26 +00:00
Ruben Fiszel ce442a7493 nit 2025-07-13 23:02:42 +00:00
Ruben Fiszel b4a8d7212b nit unused import 2025-07-13 22:25:05 +00:00
Ruben Fiszel 372d56ae40 apply iptables for cloud hosted only 2025-07-13 20:08:09 +00:00
wendrul 835645643e feat: Better tracing for audit logs, including a graph to visualize them (#6078)
* Migrate audit log page to svelte 5

* Add email and span cols to audit table

* Add token_prefixs to audit logs (into AuditAuthorable trait)

* Add audit logs graph (wip)

* Add audit span on push and jwt

* Unify same job audit into the same audit span

* Improve the graph visually

* Fix typo

* functioning graph with svelte issue

* Fix leak

* feat: migrate AuditLogsTable from DataTable to VirtualList for performance

- Replace DataTable component with VirtualList for handling thousands of rows
- Migrate to Svelte 5 runes ($props, $bindable, $derived, $state)
- Implement flattenLogs() for virtual scrolling with grouped date headers
- Add sticky indices and dynamic height calculation
- Update parent component to use callback prop pattern instead of events
- Preserve all existing functionality: filtering, selection, pagination
- Follows RunsTable.svelte implementation pattern

Resolves performance issues when displaying large audit log datasets.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* Fix remaining virtual list issues

* WIP graph

* Fix chart styling

* Fix npm check

* Fix missing audit_span arguments

* Update sqlx

* use varchar 255 for email as in other tables

* Remove syntax inconsistency

* Match struct with ee crate

* Update ee-repo-ref.txt

* Update worker_flow.rs

* Remove redefinition of trait to prevent shadowing

* Re add trait on oss but only when no `private` flag

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-07-03 21:02:45 +00:00
Ruben Fiszel 86eb9074cc feat: blacklist remote agent worker token (#5985) 2025-06-20 12:07:33 +02:00
Ruben Fiszel 126b610561 introduce QUIET_LOGS 2025-06-13 11:25:27 +02:00
pyranota 22b2f4988d fix: never consider minor version for global site packages (#5893) 2025-06-07 03:35:28 +02:00
Alexander Petric 6d3a2d279a invalidate auth cache when logging out (#5869)
* invalidate auth cache when logging out

* use proper sqlx migrate add

* formatting

* pg notify on token, not email
2025-06-05 09:30:23 +02:00
dieriba f2526571a3 feat: ssh repl like direct to workers hosts machine (#5809)
* feat(backend): add repl for worker

* feat(frontend): add repl to interact with worker and missing packages

* nits

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/result_processor.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/WorkerRepl.svelte

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* fix typo

* fix cd

* nits

* refactoring

* feat: handle other agent worker to also access repl feature, fix bugs

* update repo ref

* nits and match function args

* nits

* typo

* remove struct AgentWorkerData

* update repo ref

* nits

* impl new for authed client and revert to async

* update ref

* updage ee repo ref

* fix missing method/imports small bugs

* update ee repo ref

* update .sqlx

* test

* clean wait for interactive_shell future

* free call stack

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-06-03 18:41:14 +02:00
Diego Imbert 0e316239dd EE Refactor (#5844)
* app compiles with every ee substituted

* Replace all oss files content

* Revert "Replace all oss files content"

This reverts commit ea4017d59f.

* delete all ee

* hide all _ee files under private flag

* hide every oss stuff when private flag set

* pub use *

* gitignore and substitute script

* pub mod for ee needed for ee repo

* small mistakes

* remove oidc_oss impl

* ee ref (temp)

* ee ref

* fix --all-features selecting private in OSS CI

* ee repo ref

* allow unused
2025-06-02 22:12:33 +02:00
Ruben Fiszel 5b96bccedd feat: add aws oidc support for instance s3 storage (#5810)
* backend

* iterate

* all

* all

* all

* iterate

* revert

* all

* add tracing to get of authed client

* all

* all

* lal

* all

* update

* fix

* push

* all

* all

* revert

* frontend

* fix checks

* avoid deadlock

* safer

* fix

* fix
2025-05-25 14:03:38 +02:00
HugoCasa 30edcdfe0e fix: workspace preprocessor improvements (#5784)
* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix: workspace preprocessor fixes

* tmp ee ref

* fix build

* update ee ref

* fix: hub script preprocessor handling

* fix build

* good ref

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
2025-05-21 17:49:21 +02:00
Ruben Fiszel 7a43893616 nit 2025-05-20 13:23:02 +02:00
Ruben Fiszel ba4c89e7db nit 2025-05-20 12:55:13 +02:00
Ruben Fiszel d223b0b12e add log on s3 loading 2025-05-20 10:24:55 +02:00
Ruben Fiszel 62b6540a78 nit delete performance 2025-05-15 17:52:25 +02:00
HugoCasa ddd18d22a6 perf: cache http trigger routers and auth (#5748)
* perf: cache http trigger routers and auth

* fix build

* fix

* fix build

* fix build

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-15 17:38:45 +02:00
Ruben Fiszel 0efba945ba fix: improve perf of job deletion 2025-05-15 15:44:55 +02:00
dieriba 54b4815df2 nits: use generic deserializer for empty data (#5727)
* use generic deserializer for empty data

* update repo ref

* fix import
2025-05-12 11:00:55 +02:00
Ruben Fiszel 7c69959853 fix: improve agents workers handling of WHITELIST_ENVS 2025-05-11 08:38:46 +02:00
HugoCasa 254c3cf8ef feat: preprocessor refactor (#5629)
* feat: preprocessor refactor

* nit

* oneOf kind support + new capture format + fix build

* fix build

* fix build

* update preprocessor templates

* cache runnable version info

* update ee ref

* update hub scripts && ee ref
2025-05-09 09:50:01 +02:00
pyranota 7b48c10f12 nit: fix error if db oversize checker is disabled (#5718) 2025-05-08 17:05:24 +02:00
Ruben Fiszel afc9cf5940 update deps versions (#5703) 2025-05-05 16:19:39 +02:00
pyranota 4fd0561239 feat: critical alert if disk near full (#5549)
* feat: critical alert if disk near full

* update logic to cover edge-case

* update logic

* windows support

* Update README.md

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* make it periodically

* add extra safety

* respect killpills

* do not check every 3 secs

* move to monitor.rs

* rework

* alter readme

* extending functionality

* fix worker_mode

* fix compilation

* fix typo

* make use of AI suggestion

* update ee-repo-ref

* update ee ref

* logs are CE

* update ee repo ref

* remove systemstat from worker crate

* fix comp error + sync cargo.lock

* more comptime fixing

* Update ee-repo-ref.txt

* fix compilation error

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-05-05 08:54:48 +02:00
Ruben Fiszel 7eb9d7d46c fix: do not track relative deps for scripts with raw defined deps from CLI (#5696) 2025-05-02 00:37:28 +02:00
Ruben Fiszel cf77ff088b fix: improve mcp mode api 2025-04-28 19:17:24 +02:00
Alexander Petric dc5c8d8c5f fix: check for valid teams_channel config when saving critical alerts settings (#5660)
* fix: check for valid teams_channel config when saving critical alerts settings

* Update frontend/src/lib/components/InstanceSettings.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* sqlx prepare

* Update InstanceSettings.svelte

* Update InstanceSettings.svelte

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-04-25 14:27:33 +02:00
centdix 1625524311 feat: Add MCP only mode (#5661)
* add mcp mode

* fix flag usage

* change mcp path

* do not migrate db in mcp mode

* handle null values
2025-04-25 14:22:45 +02:00
Ruben Fiszel 12ba15c928 improve benchmarks 2025-04-12 11:09:21 +00:00
Ruben Fiszel d5186da271 allow multiple workers on agent mode (#5607) 2025-04-11 21:19:29 +00:00
Ruben Fiszel 5e054d3385 load base url and critical channels in initial_load 2025-04-11 08:22:21 +00:00
dieriba 6339775404 feat: add gcp trigger (#5501)
* update: add migration for gcp_trigger table, add cli method gcp and add gcp_trigger args for github ci

* update: add gcp module and routing it in lib.rs

* update: added gcp type to TRIGGER KIND type in db, add new feature condition to try_get_fn and added gcp_trigger feature in cargo file

* update: add ui for gcp trigger, added missing triggers type for job kin, add gcp trigger in deploy to functionallity

* adding google cloud crate

* update: handle pull and push delivery

* update: handle pull and push delivery

* update: handle push event, changed front ui

* update: ui

* fix: capture for gcp

* update: automatically manage pub sub subscription and refactoring

* capture done

* update cli types

* fix: wrong func argument and type openapi

* fix: missing import

* update .sqlx

* update: svg color and remove pulse button for capture push gcp

* update: handle deployto

* update script helper and link to hub gcp script/flow

* update gcp representation

* update: add confirmation modal

* add unique index to mitigate same subscriber id, fix `zombie worker`, update test connection logic

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update cli/gen/services.gen.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* update repo ref

* fix feature function

* update dependencies

* update .sqlx

* fix missing import

* update: handle existing subscription and creating new one or update it

* fix ee build

* handle existing config properly

* update .sqlx

* update .sqlx

* remove unused

* update documentation link fix ci

* use on instance of empty_string_as_none fn

* update refo ref

* updat script_helper

* update ui

* update migration

* add missing arguments

* fix and nits

* update repo ref

* remove unused

* nits

* add doc links

* nits

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerPanel.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorInner.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* use trigger path as default route path

* update .sqlx

* update repo ref

* update ref

* update route path

* update types

* fix deploy to

* nits

* update repo ref

* update .sqlx

* Update frontend/src/lib/components/triggers/gcp/GcpTriggerEditorConfigSection.svelte

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* nits

* nits

* nits

* remove dupliacte fn and update repo ref

* remove unused import

* fix missing import

* fix: wong name var

* update script helper and template script

* Update frontend/src/lib/script_helpers.ts

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update ee-repo-ref.txt

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-04-10 20:39:01 +00:00
Ruben Fiszel 63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
pyranota f3ed108e43 nit: replace KJQXZ with more meaningful notation (#5547)
* nit: replace `KJQXZ` with more meaningful notation

Originally this string is located in all places, where modification is needed in order to add new language support

* relative -> related

* revert shebang in substitue.sh

* remove '}'
2025-04-02 21:03:04 +00:00
Ruben Fiszel 7c96ef15d1 add disable_jobs_monitoring_env_variable 2025-03-30 00:17:16 +00:00
Ruben Fiszel 45a6d5934e add disable_jobs_monitoring_env_variable 2025-03-30 00:12:27 +00:00
pyranota 59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
Ruben Fiszel 0638ed0365 set alerts = true when setting change 2025-03-27 01:51:29 +01:00
Ruben Fiszel f58ac70ab8 check latest sent log file localy 2025-03-27 00:47:04 +01:00
Ruben Fiszel 247e8f10c0 small nits 2025-03-26 22:23:22 +01:00
Ruben Fiszel 5573d88695 perf: cache workspace premium check 2025-03-25 02:00:51 +01:00
Ruben Fiszel 3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00
Ruben Fiszel a3f6db7dca perf: cache workspace env variables to avoid one query (#5499) 2025-03-22 18:37:48 +01:00
pyranota a3faea16e7 feat: add nu-lang support (#5217)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version
2025-03-20 16:45:21 +01:00
Ruben Fiszel 880db319e8 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
Ruben Fiszel 303a81f1bf only reload license key periodically on ee 2025-03-18 21:57:17 +01:00
Ruben Fiszel 8a7b119d4d improve concurrency limits II (#5485) 2025-03-17 17:24:50 +01:00