835645643e
feat: Better tracing for audit logs, including a graph to visualize them ( #6078 )
...
* Migrate audit log page to svelte 5
* Add email and span cols to audit table
* Add token_prefixs to audit logs (into AuditAuthorable trait)
* Add audit logs graph (wip)
* Add audit span on push and jwt
* Unify same job audit into the same audit span
* Improve the graph visually
* Fix typo
* functioning graph with svelte issue
* Fix leak
* feat: migrate AuditLogsTable from DataTable to VirtualList for performance
- Replace DataTable component with VirtualList for handling thousands of rows
- Migrate to Svelte 5 runes ($props, $bindable, $derived, $state)
- Implement flattenLogs() for virtual scrolling with grouped date headers
- Add sticky indices and dynamic height calculation
- Update parent component to use callback prop pattern instead of events
- Preserve all existing functionality: filtering, selection, pagination
- Follows RunsTable.svelte implementation pattern
Resolves performance issues when displaying large audit log datasets.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Fix remaining virtual list issues
* WIP graph
* Fix chart styling
* Fix npm check
* Fix missing audit_span arguments
* Update sqlx
* use varchar 255 for email as in other tables
* Remove syntax inconsistency
* Match struct with ee crate
* Update ee-repo-ref.txt
* Update worker_flow.rs
* Remove redefinition of trait to prevent shadowing
* Re add trait on oss but only when no `private` flag
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: GitHub Action <action@github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-07-03 21:02:45 +00:00
dd91089436
improve graphql error reporting ( #6092 )
2025-07-01 15:45:04 +02:00
c51e128920
fix: add support for GCS object storage ( #6083 )
...
* AI: Updates to files (run 15923369101)
* all
* ee ref
* unneeded
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch >
2025-06-30 20:11:57 +00:00
bb110985e2
make OUTSTANDING_WAITING_TIME_TS configurable
2025-06-28 19:06:55 +02:00
835f1d2ec9
fix: add windows paths to uv install to find git/ssh ( #6063 )
2025-06-26 08:26:59 +02:00
900c8edd7b
fix: ignore type only imports when computing ts lockfiles
2025-06-25 14:29:52 +02:00
29f6fab60c
fix: pwsh skip already installed modules outside of cache ( #6037 )
2025-06-24 12:41:54 +02:00
b3b6c53430
remove WM_OIDC_JWT from context vars
2025-06-20 12:12:38 +02:00
86eb9074cc
feat: blacklist remote agent worker token ( #5985 )
2025-06-20 12:07:33 +02:00
c4adaeeabd
fix: improve piptar upload - sequential uploads via background task queue ( #5994 )
...
* feat: improve piptar upload with sequential background task queue
Replace simultaneous piptar uploads with sequential processing via background job queue while keeping dependency installation parallelized.
Key changes:
- Add PiptarUpload job kind and database migration
- Queue piptar uploads instead of using tokio::spawn
- Implement job handler for sequential S3 uploads
- Maintain parallel dependency installation as requested
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Remove PiptarUpload job kind migration files
Refactoring piptar uploads to use tokio channel instead of Windmill jobs.
Migration files are no longer needed.
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Refactor piptar uploads from Windmill jobs to tokio channel
Replace the complex job queue infrastructure for piptar uploads with a simple
tokio channel approach as requested. This maintains sequential upload behavior
while keeping dependency installation parallel.
Key changes:
- Add PIPTAR_UPLOAD_CHANNEL global channel for sequential processing
- Replace JobPayload::PiptarUpload with simple channel send
- Remove PiptarUpload from JobKind enum and all job handling code
- Remove job dispatcher case from worker.rs
- Simplify upload logic while maintaining same functionality
Benefits:
- Reduced complexity by removing unnecessary job infrastructure
- Sequential uploads without blocking dependency installation
- Better separation of concerns
🤖 Generated with [Claude Code](https://claude.ai/code )
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
* Update worker.rs
* Update python_executor.rs
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-06-19 15:32:21 +02:00
47d1ef0f1c
fix additional python paths for windows
2025-06-18 18:24:54 +02:00
0f63d03093
dind compile
2025-06-17 08:14:11 +02:00
38eb71bdf5
fix: always rm containers in docker mode
2025-06-17 02:32:49 +02:00
26bec054a3
fix: flow steps use their tags if any specific when used as subflow
2025-06-17 02:27:24 +02:00
0b89260540
feat(go): local go.mod ( #5929 )
...
* feat(go): local go.mod
* readability refactor
* remove dbg!
* ignore module
* remove space
Signed-off-by: pyranota <pyra@duck.com >
* Update backend/windmill-worker/src/go_executor.rs
---------
Signed-off-by: pyranota <pyra@duck.com >
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
2025-06-13 10:17:31 +02:00
e8836a393a
fix: improve concurrent job parallelism performance
2025-06-09 16:59:07 +02:00
a2eac4ae37
nit: fix windows build ( #5896 )
2025-06-07 03:35:54 +02:00
22b2f4988d
fix: never consider minor version for global site packages ( #5893 )
2025-06-07 03:35:28 +02:00
ec552d5ef6
fix(python): account instance version when cli deploy and local lockfile ( #5894 )
...
* fix: do not ignore instance python version with cli deploy and local lockfile
* ellipsis
2025-06-06 17:49:59 +00:00
a0b302d2c5
fix: cannot parse INSTANCE_PYTHON_VERSION ( #5874 )
2025-06-05 16:56:30 +02:00
50a5c1f56a
fix(python): assign PATCH version to python runtime only when needed ( #5866 )
...
* fix
* clean up
* nit
* fix integration tests
* nit
* another nit
* remove dublicate test
2025-06-04 17:15:46 +02:00
f2526571a3
feat: ssh repl like direct to workers hosts machine ( #5809 )
...
* feat(backend): add repl for worker
* feat(frontend): add repl to interact with worker and missing packages
* nits
* Update backend/windmill-worker/src/result_processor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* Update backend/windmill-worker/src/result_processor.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update frontend/src/lib/components/WorkerRepl.svelte
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* fix typo
* fix cd
* nits
* refactoring
* feat: handle other agent worker to also access repl feature, fix bugs
* update repo ref
* nits and match function args
* nits
* typo
* remove struct AgentWorkerData
* update repo ref
* nits
* impl new for authed client and revert to async
* update ref
* updage ee repo ref
* fix missing method/imports small bugs
* update ee repo ref
* update .sqlx
* test
* clean wait for interactive_shell future
* free call stack
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-06-03 18:41:14 +02:00
0e316239dd
EE Refactor ( #5844 )
...
* app compiles with every ee substituted
* Replace all oss files content
* Revert "Replace all oss files content"
This reverts commit ea4017d59f
2025-06-02 22:12:33 +02:00
ed61d97700
feat(rust): shared build directory ( #5610 )
...
* feat: rust incremental compilation
* update dockerfiles
* fix compilation error
* add windows flags
* init
* polishing
* update
* return mount-point
* Update backend/windmill-worker/src/rust_executor.rs
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
* share worker dir if nsjail off
* final refactor
---------
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-29 22:11:35 +02:00
0bdcad80ce
duckdb optional dep
2025-05-29 14:54:56 +02:00
fdefd4be93
feat: duckdb sql lang support ( #5761 )
2025-05-27 15:52:57 +02:00
5b96bccedd
feat: add aws oidc support for instance s3 storage ( #5810 )
...
* backend
* iterate
* all
* all
* all
* iterate
* revert
* all
* add tracing to get of authed client
* all
* all
* lal
* all
* update
* fix
* push
* all
* all
* revert
* frontend
* fix checks
* avoid deadlock
* safer
* fix
* fix
2025-05-25 14:03:38 +02:00
26222539e6
feat(python): inline script metadata (PEP 723) ( #5712 )
...
* make resolver
* more updates
* fix build
* fix raw_dependencies job type
* compat with http agent workers
* refactor
* rename
* more refactor
* cleanup
* more tests
* fix s3
* small fixes
* more fixing
* fix endpoint
* nit: update comment
* update ee ref
* update ee ref
* update ee ref
* implement safer `list_available_python_versions`
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783 )
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix build
* update ee ref
* ignore versions <=3.9
* fix windows build
* correct versions filter
* fix windows build (this time for real)
* inject error to debug CI
* update CI
* undo debug of CI
* fix tests
* remove outdated comment
* update ee repo ref
* Update ee-repo-ref.txt
* Update backend/parsers/windmill-parser-py-imports/src/lib.rs
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
* Update InstanceSetting.svelte
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
af74653b7f
improve err message
2025-05-22 19:48:26 +02:00
7c24fbcef2
fix: improve docker mode unexpected exit handling
2025-05-22 19:37:47 +02:00
3f825ec77f
reduce the pip resolution cache duration
2025-05-22 18:47:02 +02:00
e3e25daee7
fix
2025-05-22 14:12:10 +02:00
d662e18f97
add more labels to traces
2025-05-22 12:07:21 +02:00
dfd8c4cd2a
more verbose docker wait errors
2025-05-22 01:07:29 +02:00
30edcdfe0e
fix: workspace preprocessor improvements ( #5784 )
...
* add tracing to get of authed client
* internal: Trigger claude when commenting with /aider (#5783 )
* add claude instructions files
* call claude too when using aider
* fix
* add draft for linear claude integration
* fix: workspace preprocessor fixes
* tmp ee ref
* fix build
* update ee ref
* fix: hub script preprocessor handling
* fix build
* good ref
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev >
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com >
2025-05-21 17:49:21 +02:00
07c2ff5668
feat(python): add annotation to skip result post-processing ( #5769 )
...
* feat(python): add annotation to skip result post-processing
Typically windmill will replace all NaN, Infinity and -Infinity from resulting string.
We do it because JSON specification does not support these types as well as DB.
However it will substitute also the cases when any of those words are used within the string.
E.g. script returning "To Infinity and Beyond" will be postprocessed to "To null and Beyond".
Current behaviour is done for the performance sake and now can be disabled with `#skip_result_postprocessing` annotation.
* add comments
* remove extra comments
2025-05-19 22:31:46 +02:00
beb658757f
nit error
2025-05-17 10:56:16 +02:00
59afa493fa
fix: improve handling of custom concurrency key/tag with preprocessors ( #5762 )
...
* fix
* rm spurrious diff
* handle failing preprocessors
* nits
* update
* update
2025-05-17 10:52:28 +02:00
b29c6e7636
fix: S3 sql mode returns S3Object ( #5764 )
...
* fix upload_s3_file not checking response
* Return S3Object instead of path string
2025-05-16 19:28:57 +02:00
e87d4f3c1a
fix: avoid deadlocks in sending completed job to result processors ( #5742 )
...
* buffer
* unbounded and bounded
* fix
* buffer
* impl clone
* flume it all
* update
* update
* update
2025-05-15 09:13:09 +02:00
669a95c3be
oracle support for s3 streaming + misc ( #5738 )
...
* changed SQL templates to include S3
* oracle support for s3 streaming
2025-05-14 15:45:00 +02:00
5b26498e09
add more duration warnings to logs
2025-05-14 10:13:14 +02:00
d35a7d22f9
fix: add support for log compaction on docker jobs ( #5732 )
...
* improve docker compact
* improve docker compact
* update ref
* update
* update agent workers
2025-05-13 21:42:15 +02:00
f73c90c751
fix: Ansible lockfile back compatibility issue ( #5731 )
...
* fix: Ansible lockfile back compatibility issue
* Replace condition with more readable alternative
2025-05-13 16:19:18 +02:00
c7886ea07a
feat: sql jobs outputting to s3 + streaming for high-number of rows ( #5704 )
...
* stream to s3 boilerplate
* S3 works with new syntax
* snowflake s3 streaming support
* postgres s3 support
* fix postgres stream format
* mysql s3 streaming
* mssql s3 streaming
* new s3 mode syntax
* optional folder param
* rename folder to prefix
* json_stream_arr_values
* cargo toml rollback
* convert_ndjson with datafusion
* format conversion kinda works
* Fixed not finishing the datafusion writer
* support for pg and mssql
* fix file ext
* bigquery conversion and works with s3 streaming
* fix s3 flag parser
* snowflake s3 streaming support
* factor out duplicate code
* remove anyhow
* Err case for parse s3 mode
* Send error to mpsc
* bigquery s3 streaming fix for huge queries
* remove extra stuff
* snowflake s3 streaming support
* small regex mistake
* cfg(not(feature = "parquet"))
* fix CI (unused import)
* error handling fix (graphite)
2025-05-13 10:19:44 +02:00
b51568c166
fix: make ansible more resilient to invalid lockfiles
2025-05-12 11:40:13 +02:00
54b4815df2
nits: use generic deserializer for empty data ( #5727 )
...
* use generic deserializer for empty data
* update repo ref
* fix import
2025-05-12 11:00:55 +02:00
a23a4c0faf
nit compile
2025-05-11 11:52:42 +02:00
a3c76fb10c
fix: improve error display of nativets exceptions
2025-05-11 11:35:31 +02:00
7c69959853
fix: improve agents workers handling of WHITELIST_ENVS
2025-05-11 08:38:46 +02:00
wendrul
Alexander Petric
windmill-internal-app[bot]
Ruben Fiszel
HugoCasa
claude[bot]
pyranota
dieriba
Diego Imbert