Commit Graph

41 Commits

Author SHA1 Message Date
Ruben Fiszel 9c85565221 fix: bypass OTEL MITM tracing proxy for git sync jobs (#8796)
Git sync runs as a DeploymentCallback job. When the OTEL MITM tracing
proxy is enabled, all HTTP/HTTPS traffic from the script is rerouted
through a local intercepting proxy that chains to the corporate upstream
proxy. Git's HTTPS to GitHub fails in this setup (TLS interception with
chained CONNECT tunneling is fragile, and git's CA env handling diverges
from what the proxy injects), so customers see "GitHub.com URL couldn't
be reached" until they disable OTEL.

Detect DeploymentCallback jobs in get_proxy_envs_for_lang and fall back
to the stock PROXY_ENVS so git talks to the corporate proxy directly,
unmodified. The git sync script is system code; we don't need HTTP spans
for it.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 16:32:19 +00:00
Ruben Fiszel 73fe45b6cb feat: workspace-specific registry overrides (#8406)
* feat: add workspace-specific registry overrides

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move workspace registries to end of registries tab

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: workspace overrides use field selector instead of showing all fields

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: polish workspace registries UI to match design guidelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show field selector directly and fix addField initialization logic

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: namespace pip_resolution_cache by workspace when registry overrides exist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: namespace binary/bundle caches by workspace when registry overrides exist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* perf: zero-cost cache suffix when no workspace overrides exist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: reload workspace_registries via notify events on setting change

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review findings

- Fix discardCategory not reverting workspace_registries changes
- Fix get_no_default: convert to async fn with owned Uuid param
- Fix append_logs: use windmill_queue import already available
- Fix ruby URL parsing: support both comma and whitespace delimiters
- Add WorkspaceRegistryMap type alias to reduce inline type noise

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 12:49:25 +00:00
hugocasa 65a92d9899 fix: set nsjail time_limit from job timeout so configured defaults are respected (#8389)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 12:06:31 +00:00
Ruben Fiszel 8b1fe8f9de fix: gracefully handle uninitialized OTEL tracing proxy port (#8274)
* fix: gracefully handle uninitialized OTEL tracing proxy port

When OTEL tracing proxy is enabled but the MITM proxy port hasn't been
assigned yet (race condition at startup, or NUM_WORKERS > 1), fall back
to standard proxy envs instead of failing the job with
"OTEL tracing proxy port not initialized".

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: log to job logs when OTEL tracing proxy is unavailable

When the OTEL tracing proxy is enabled but the port isn't initialized
(race at startup or NUM_WORKERS > 1), append a warning to the job logs
explaining why HTTP request tracing is unavailable for that job.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 09:34:21 +00:00
Ruben Fiszel 077779ec52 fix: improve windows compatibility
* ci: add Windows backend integration test workflow

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: temporarily add push trigger for testing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* ci: add --no-fail-fast to run all test binaries

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: Windows path handling for backend integration tests

- WINDMILL_DIR: use std::env::temp_dir() on Windows instead of /tmp/windmill
- HOME_ENV: fall back to USERPROFILE on Windows when HOME is not set
- loader.bun.js: normalize paths to forward slashes for consistent
  comparison with Bun's resolver output on Windows
- bun_executor.rs: convert job_dir to forward slashes in JS template
  strings to avoid backslash escape issues (\t -> tab, etc.)
- go_executor.rs: fix windows_gopath() double backslash bug (r"\\" -> "\\")
- bash_executor.rs: default to "bash" (in PATH) on Windows instead of /bin/bash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: improve Windows diagnostics and fix onLoad handler

- Include path in create_directory_async/sync panic messages
- Add WINDMILL_DIR initialization debug output
- Fix loader.bun.js onLoad: use properly escaped regex instead of
  returning undefined (Bun requires onLoad to return an object)
- Add env var debug output to CI workflow

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: sanitize Windows-invalid characters in test worker names and fix cargo path

- Replace :: with __ in worker names (colons illegal in Windows dir names)
- Fix HOME_DIR to fall back to USERPROFILE on Windows
- Add PATH fallback for cargo discovery on Windows
- Add debug logging to bun loader for fetch errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle single colons in worker names, pass MSVC linker env vars, revert bun debug

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .exe binary name on Windows and normalize bun import URL paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use absolute path for rust binary, normalize bun resolve paths

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use .wurl extension instead of .url for bun import resolution on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use custom namespace for bun plugin to bypass default file resolution

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use virtual namespace for bun import resolution to avoid Windows path issues

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: handle Windows 8.3 paths and namespace-prefixed importers in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: strip namespace prefix from args.path and handle absolute imports without leading slash in bun loader

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: simplify bun loader and remove redundant cargo path lookups

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: use platform-specific cargo binary path with .exe on Windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: replace HOME_DIR with HOME_ENV in rust_executor to remove duplication

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* refactor: keep original bun loader on linux, use virtual namespace loader only on windows

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-03-04 20:20:18 +00:00
Ruben Fiszel 424ca59dfe feat: make WINDMILL_DIR configurable via environment variable (#8215)
* fix: auto-heal corrupted python runtime cache on remote workers

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "fix: auto-heal corrupted python runtime cache on remote workers"

This reverts commit 0ea013a554.

* feat: make WINDMILL_DIR configurable via environment variable

Allow users to configure the base directory for Windmill's tmp/cache files
via the WINDMILL_DIR env var (default: /tmp/windmill). This fixes Python
runtime cache corruption on RHEL systems where systemd-tmpfiles-clean
removes files from /tmp.

Converts TMP_DIR (renamed to WINDMILL_DIR) and all derived cache directory
constants from compile-time const &str (concatcp!) to runtime lazy_static
String values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref ERROR_DIR lazy_static for AsRef<Path> and Display traits

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref to branch name for CI compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref lazy_static constants in all executor files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee ref

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: panic if WINDMILL_DIR has trailing slash

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: also reject trailing backslash in WINDMILL_DIR for Windows

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: deref GO_BIN_CACHE_DIR in test utils

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: replace remaining hardcoded /tmp/windmill paths and validate empty WINDMILL_DIR

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: nsjail powershell mount dst, Windows path assumptions, pwsh deref consistency

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore Windows /tmp path translation in go and bun executors

The Windows path translation replaces /tmp with the Windows temp dir
(e.g. C:\tmp) before normalizing slashes. Without this, the default
WINDMILL_DIR=/tmp/windmill produces paths without a drive letter on
Windows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

This commit updates the EE repository reference after PR #436 was merged in windmill-ee-private.

Previous ee-repo-ref: e8c03e16720833230ebd1878b4c63642ecc6c80f

New ee-repo-ref: 6fd5a2ce908235a17975ad4dbdf0051cd89334f3

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-04 08:53:25 +00:00
Ruben Fiszel 6bf544f507 refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend

Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 12:24:45 +00:00
Ruben Fiszel 423e07376b feat: gate private registry configuration behind enterprise feature (#7953)
Add read_ee_registry<T> helper that returns None and emits a job log
in CE builds when a private registry is configured. Applied to all
language executors: pip, npm/bun, deno, cargo, go, maven, nuget,
powershell, and ruby. Maven settings.xml file write in monitor.rs
is also gated.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 14:02:14 +00:00
Ruben Fiszel 581dde8d0b feat: add maven settings.xml support for java private registries (#7940)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-13 16:57:30 +00:00
Ruben Fiszel a9e4a5c8e7 windows build nit 2026-02-13 15:16:13 +00:00
Ruben Fiszel 7d88676b15 rust nits 2026-02-13 14:13:11 +00:00
Ruben Fiszel 50f04fe8d4 rust nits 2026-02-13 14:08:42 +00:00
Ruben Fiszel e144432a16 fix: rust + java works with just /tmp mounted 2026-02-13 14:03:45 +00:00
Ruben Fiszel 791cb3e225 fix: java + rust only relies on /tmp, + https proxy improvement for java 2026-02-13 10:47:59 +00:00
Ruben Fiszel 2646629194 feat: add force_sandboxing global setting and #sandbox bash annotation (#7816)
* feat: add force_sandboxing global setting and #sandbox bash annotation

Add a new global setting `force_sandboxing` to enable nsjail sandboxing
for all jobs, overriding the DISABLE_NSJAIL environment variable.
Also add a #sandbox annotation for bash scripts to enable sandboxing
on a per-script basis.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: decouple nsjail probing from DISABLE_NSJAIL and apply force_sandboxing to all executors

NSJAIL_AVAILABLE now always probes for the nsjail binary at startup
regardless of the DISABLE_NSJAIL policy flag, fixing #sandbox annotation
and force_sandboxing being unusable in default (DISABLE_NSJAIL=true)
configuration. All language executors now use is_sandboxing_enabled()
instead of checking DISABLE_NSJAIL directly, so force_sandboxing
applies to all job types, not just bash.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: clarify force_sandboxing and DISABLE_NSJAIL relationship

Neither setting overrides the other — sandboxing is enabled when either
force_sandboxing=true OR DISABLE_NSJAIL=false. Updated frontend
description and backend comments to reflect this.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* revert typo

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-12 18:20:22 +00:00
Ruben Fiszel e2c28e42db fix: add private registries support for RUST + java home nit 2026-02-12 14:23:52 +00:00
Ruben Fiszel e1a815f6a0 refactor: extract windmill-dep-map crate for parallel api/worker compilation (#7846)
* refactor: extract windmill-dep-map crate for parallel api/worker compilation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve WebhookShared type mismatch and missing enterprise propagation

- Make windmill-api webhook_util re-export from windmill-common instead of
  duplicating types, fixing Extension<WebhookShared> mismatch between
  windmill-store and windmill-api
- Add windmill-api-jobs/enterprise to windmill-trigger enterprise feature
  so check_license_key_valid is available when trigger subcrates enable
  enterprise on windmill-trigger

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stop trigger features from unconditionally enabling enterprise

Move enterprise propagation for all trigger subcrates from individual
trigger feature definitions to the enterprise feature itself, so
enterprise is only enabled when explicitly requested.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove unused pub use re-exports and disable CI cargo cache

- Remove unused re-exports from windmill-worker/src/lib.rs:
  trigger_dependents_to_recompute_dependencies, handle_job_error,
  and unused bun/otel items
- Fix callers to use direct module paths instead
- Add windmill-dep-map as dev-dependency for tests
- Disable cargo cache in backend-check CI (faster from-scratch builds)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore bun re-exports used by tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: re-enable cargo cache for check_ee_full CI job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 00:39:56 +00:00
Pyra 95df7b9a6a feat: otel REST tracing (#7571) 2026-01-20 13:38:02 +00:00
Alexander Petric 5aa251a2d2 feat(security): unshare pid of worker job process (#7106)
* feat(security): unbind pid for worker jobs to create extra process isolation

* review

* simplify

* cleanup + compose files

* re-add removed comments from nu executor

* simplify

* fail immediately

* updates

* update ping backend

* nsjail / unshare in workers page

* migrations

* frontend + sqlx

* frontend

* frontend

* fix error message

* undo example changes
2025-11-18 23:04:31 +00:00
hugocasa 993baf46bd feat(backend): flow streaming (#6520)
* feat(backend): flow streaming

* all streaming languages + sync api

* sqlx

* fix build

* UI and nits

* nit

* feat: stream last flow step

* sqlx

* nit

* use get for stream endpoints + add snippet in UI

* refactor

* nits

* Update backend/windmill-worker/src/common.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* nits

---------

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-09-18 15:49:35 +00:00
claude[bot] fcd58191d4 fix: force stdin to Stdio::null for all user code execution (#6575)
Set stdin to Stdio::null for all Commands that execute user code across all supported languages to prevent unwanted input consumption. This affects Python, Deno, Bash, PowerShell, Go, Rust, PHP, Ruby, Java, C#, Ansible, Nu, and Bun executors.

The dedicated worker handler was intentionally left unchanged as it requires stdin for inter-process communication.

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-09-10 16:14:25 +00:00
Ruben Fiszel 1892895cd9 disable process group for dotnets 2025-08-13 16:46:14 +00:00
claude[bot] 161dbbc6d6 feat: add native result streaming (#6242)
* feat: add stream output feature to SSE job updates

Adds stream_output field to JobUpdate struct that extracts log lines
starting with '[wm_stream]:' from job logs. Regular logs now exclude
stream lines, which are captured separately for specialized handling.

- Added stream_output: Option<String> field to JobUpdate struct
- Created extract_stream_output_from_logs() function to filter stream lines
- Modified get_job_update_data() to use stream extraction logic
- SSE clients now receive both new_logs and stream_output in job updates

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* feat: rename stream_output to stream and handle newlines as \n

- Renamed `stream_output` field to `stream` in JobUpdate struct
- Updated extract_stream_output_from_logs to extract_stream_from_logs
- Changed stream output to join with literal \n instead of actual newlines
- Stream lines are properly excluded from regular new_logs

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* decision tree nits

* push ee ref

* push ee ref

* fix: fix id renaming in apps

* remove duplicate caching (#6285)

* feat: migrate audit log ids to bigints (blocking migration for EE)

* fix(mcp): add proper check for mcp routes (#6282)

* add proper check for mcp routes

* cleaner

* apply to flow

* fix add checks scopes

---------

Co-authored-by: dieriba <dieriba.pro@gmail.com>

* chore(main): release 1.514.0 (#6283)

* chore(main): release 1.514.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: pin tokio to 1.46.1 and aws-sdks-ts

* pin rustls to 0.23.29 + pin aws-sdk

* chore(main): release 1.514.1 (#6288)

* chore(main): release 1.514.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improve docker logs collection in docker mode

* support $res: string in form inputs of arrays

* fix import nit

* fix: fix DynSelect

* nits

* fix: resource-type-ts-parser (#6289)

* fix: resource types as arg in typescript handle imported defined types

* Update nix flake (#6291)

* merge

* Small UI fixes (#6294)

* fix step history not refreshing with staticInputs

* fix array of obj not showing up in json editor in test this step

* datatable scales correctly in DisplayResult and scrolling is much more usable

* avoid next button disapearing and changing layout / hurting ux

* nits

* fix bug when renaming module A to B then module C to A, C takes the schema of A

* fix bug with comments in sql repl

* fix aggrid theme randomly not loading

* bindable script

* better delete button in db manager

* property select doesnt exist

* fix all warnings

* delete $flowStateStore[id] on delete

* feat(cli): generate cursor rules on init (#6270)

* create cursor rules on init

* change gen

* add missing resource-type command

* add resource type command in guidance

* add schema option

* revert

* nit

* nit

* add flow guidance

* nit

* chore(main): release 1.515.0 (#6292)

* chore(main): release 1.515.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix: improved logs for script

* nits logs

* chore(main): release 1.515.1 (#6295)

* chore(main): release 1.515.1

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* even more indexer tracings

* add more tracing logs

* feat: prevent too large results (>500Mb) from OOMing database

* nit naming

* feat: add CA certificate update at startup via environment variable (#6280)

* feat: add CA certificate update at startup via environment variable

Add support for running 'update-ca-certificates' at binary startup
when RUN_UPDATE_CA_CERTIFICATE_AT_START environment variable is set to "true".

- Check for RUN_UPDATE_CA_CERTIFICATE_AT_START env var on startup
- Execute update-ca-certificates command if env var is set to "true"
- Log success/failure appropriately with tracing
- Continue startup even if CA certificate update fails
- Non-blocking implementation with proper error handling

Fixes #6279

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* refactor: extract CA certificate update logic into separate function

Extract the CA certificate update logic from windmill_main() into a
dedicated update_ca_certificates_if_requested() function for better
code organization and maintainability.

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* improvements

---------

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>

* fix: indexer collection of job logs before indexing (#6300)

* Add flume as dependecy for indexer

* Update ee-repo-ref

* Remove flags from cargo.toml

* Update ee-repo-ref

* Update ee-repo-ref

* fix rust sdk build error (#6305)

Signed-off-by: pyranota <pyra@duck.com>

* fix broken audit logs filter (#6304)

* rename to from to

* goto fix

* default to false if field not present operator settings (#6301)

* git sync UI improvements (#6303)

* ui improvements round 1

* modal cleanup

* init

* UI refactor

* UI cleanup + refactor

* legacy cleanup

* success model -> github actions, non-ee warnings

* sqlx

* npm check

* ee warning everywhere

* last comments

* formatting

* no hardcoded theme

* claude review improvemenets

* fix: no process relative imports for scripts with codebase

* fix: sqs oidc authentication disconnect #6307

* handle metadata for new scripts happen after commit

* handle_deployment_metadata in a task

* nits

* chore: add windmill-utils-internal package (#6299)

* add utils package

* naming

* cleaning

* add docs

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* add docs

* chore(main): release 1.516.0 (#6298)

* chore(main): release 1.516.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* merge

* indexer improvements

* upgrade tantivy to 0.24.2

* use tantivy fork

* nit warnings

* fix oss build

* improve indexer

* chore: use windmill-utils-internal for cli (#6297)

* add utils package

* naming

* cleaning

* simplify assignPath

* rename old files

* same for locks

* create on confirm

* default true

* use replaceinlinescripts from utils

* use extractscriptfromflows

* make it compile

* cleaning

* use argsigtojson

* fix

* fix missing await

* cleaner

* cleaning

* cleaning

* use in frontend

* add docs

* testing

* remove log

* use autogenerated types

* remove old

* fix

* cleaning

* adapt usage

* draft

* better build script

* fix build

* revert to default creation

* add docs

* remove and rename

* make everything work

* add await

* only if not installed

* add vs code setting

* add to publish action

* fix bc

* safer use of sep

* fix

* do not rename on push

* no publish on release

* use published package on frontend

* nit

* Add dependencies to run sqlx prepare to nix flake (#6309)

* feat(cli): wmill-lock.yaml v2 for easier git merge diffs

* merge

* merge

* all

* all

* rm warnings

* fix styling on aichatinput (#6312)

* fix: use with_capacity back presusre for tantivy directory multipart writes (#6313)

* use with capacity for tantivy directory multi part uploads

* Update ee repo ref

* Update ee-repo-ref

* Update ee-repo-ref

* chore(main): release 1.517.0 (#6310)

* chore(main): release 1.517.0

* Apply automatic changes

---------

Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>

* fix typo on cli build (#6314)

* cleanup

* feat(utils): add flow.yaml validation function (#6316)

* add validateflow function

* cleaner code

* preprocess json

* cleaning

* create specific package

* cleaning

* add tests

* fix: cleanup concurrency_counter automatically + remove orphans keys automatically

* fix: add disabled support to resource picker in schema forms

* fix: add wm_labels to tracing spans

* all

* merge

* all

* fix: delete empty git connection (#6318)

* fix checks

* bun handling

* all

* all?

* all

* all

* update

* all

* update

* check

* fix history

* all

* all

* all

* Remove leftover debug tracing statements

- Remove commented debug trace in jobs.rs for stream output
- Remove commented debug trace in result_stream.rs for stream processing

Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>

* fix test

* all

* handle iter

* fix

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: dieriba <dieriba.pro@gmail.com>
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
Co-authored-by: wendrul <53628737+wendrul@users.noreply.github.com>
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
Co-authored-by: Alexander Petric <alpetric@users.noreply.github.com>
Co-authored-by: Alexander Petric <alex@windmill.dev>
Co-authored-by: pyranota <92104930+pyranota@users.noreply.github.com>
2025-08-06 22:40:42 +00:00
pyranota a2eac4ae37 nit: fix windows build (#5896) 2025-06-07 03:35:54 +02:00
pyranota ed61d97700 feat(rust): shared build directory (#5610)
* feat: rust incremental compilation

* update dockerfiles

* fix compilation error

* add windows flags

* init

* polishing

* update

* return mount-point

* Update backend/windmill-worker/src/rust_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* share worker dir if nsjail off

* final refactor

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-05-29 22:11:35 +02:00
Ruben Fiszel 5b96bccedd feat: add aws oidc support for instance s3 storage (#5810)
* backend

* iterate

* all

* all

* all

* iterate

* revert

* all

* add tracing to get of authed client

* all

* all

* lal

* all

* update

* fix

* push

* all

* all

* revert

* frontend

* fix checks

* avoid deadlock

* safer

* fix

* fix
2025-05-25 14:03:38 +02:00
Ruben Fiszel 63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
pyranota 59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
Ruben Fiszel 3edca4bc91 perf: optimize number of queries needed for job run (#5504)
* optPerf

* update sqlx

* update sqlx
2025-03-24 20:12:26 +01:00
Ruben Fiszel 880db319e8 feat: filter by worker + backend perf opt (#5489)
* all

* all

* all

* refactor

* sqlx

* ref

* impr

* fix

* fix

* fix

* ee private
2025-03-19 09:55:53 +01:00
wendrul 6357ed3d5e fix: Remove cache dir mount and mount only the cache executable (Rust, C#) (#5270) 2025-02-11 16:08:19 +01:00
Lucas Abel 71d527c29e nix: rust support (#4957) 2024-12-20 11:12:44 +01:00
Lucas Abel 977ac5c3f3 feat(cache): remove persistent raw values from queue (#4866)
* feat(cache): remove persistent raw values from queue

* test: add tests and fix discovered regressions

* fix csharp executor after rebase

* invalidate script cache after locking

* fix lite flow version race

* fix flow dependencies lock race

* update version flag to `v.1.440`
2024-12-16 17:10:16 +01:00
wendrul c85d2a4957 feat: add C# support (#4908)
* Add csharp stuff and merge after the rewrite

    * Add csharp, boilerplate + minimal execution
    * Add initial parser
    * Frontend + wasm export of the tree sitter parser
    * Arg spread and use cache

* More types and req parsing

* Add logo

* Parse async and void to fit the wrapper to the main sig

* Csharp editor resource and var buttons

* Lockfile generation

* Add default parsing using serde_json::from_str

* Update init code for c#

* Nsjail for c#

* Remove c_variadic feature

* Remove unused imports

* Add c_variadic only for wasm32 assembly

* Fix c# test

* Add dotnet to the images

* Update dockerfiles

* Update dockerfile again

* Touch Dockerfile

* npm install c# parser

* Add opt mount in nsjail

* Update ee repo ref

* Add csharp to backend image for tests

* Add setting to set nuget package

* Windows compatibility

* Disable csharp test

* ADd feature flag
2024-12-13 00:29:01 +01:00
Ruben Fiszel b19faa45cd fix compile 2024-12-05 15:17:24 +01:00
Ruben Fiszel 5404ec9b48 fix: pass USERPROFILE on windows 2024-12-05 10:12:44 +01:00
Ruben Fiszel e11cc13c92 fix: apply NO_PROXY and HTTP_PROXY, HTTPS_PROXY more consistently 2024-11-02 13:57:30 +01:00
Stephen Beckstrand 09a8a0361b chore: Update references to windmill-ee-full image (#4578)
* chore: Update windmill-ee-full image references

The messaging in a few backend executors references the need to
use the full enterprise image rather than `windmill-ee`. These
messages reference the `windmill-full-ee` image but its the image
is actually named `windmill-ee-full` under the available Packages.

* chore: Update refs to rust to ansible in ansible executor
2024-10-24 17:40:59 +02:00
Alexander Petric 6eb7f8ce81 feat(worker): support workers to run natively on windows (#4446)
* minimal code change to get windmill worker on windows for bun and python + rustfmt

* adding support for powershell

* compiling error on unix

* rust linting comments

* comments hugo: PSModulePath

* comments ruben, refactor to simplify

* adding build workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* editing workflow

* skip migration env, ee fixes

* improvements powershell

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* testing windows runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* install postgres on runner

* killing process tree in windows

* sqlx_offline

* install openssl for github windows runner

* used pre-installed openssl

* used pre-installed openssl

* build ee

* build ee

* build ee

* build ee

* adding commented out steps for artifact publishing

* build on tag matchinv v* pattern

* ren instead of mv on Windows

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix merging issue

* gate imports for windows

* fixing default cargo home path...

* fixing default cargo home path...

* comments ruben

* make pwsh default modules loading more robust on unix (#4448)

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: HugoCasa <hugo@casademont.ch>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2024-10-03 15:59:12 +02:00
Ruben Fiszel bd6ddc166b feat(cli): add queues, workers and worker-groups commands (#4439)
* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all

* all
2024-09-26 11:27:37 +02:00
wendrul 91d328b132 feat: add rust (#4253)
* Add rust on frontend

* Add parser for rust

* Handle rust job execution

* Main has to return a result that is serializable

* Update parser: parse many rust types

* Rust parser dependencies

* Frontend wasm parser

* Add windmill parser rust to windmill parser wasm (temporarily)

* Add rust as a tag on worker__default

* Change init code for rust

* Cleanup rust_executor.rs

* Remove tree-sitter dep and fix unused imports

* Add lockfile and build logic

* Add cargo and rustup to windmill image

* Fix env var to work on docker image and dev env

* modify package.json

* Deps after parser publish

* Add stashed migration

* Unify rust versions

* Add error message when php or cargo are not present to use another image

* Error message conditionally on feature flag

* all ci/cd changes

* all ci/cd changes

---------

Co-authored-by: Ruben Fiszel <ruben@rubenfiszel.com>
2024-08-30 17:01:04 +02:00