Commit Graph

114 Commits

Author SHA1 Message Date
Ruben Fiszel 61f3291b24 fix: actually isolate windows job children from CTRL_BREAK_EVENT + reap on worker death (#9563)
* fix: actually isolate windows job children from CTRL_BREAK_EVENT + reap on worker death

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* decouple LIMIT_WINDOWS_TO_1CU memory cap from DISABLE_PROCESS_GROUP escape hatch

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* create windows job children suspended, assign job, then resume (close post-spawn race)

Addresses Codex P1: AssignProcessToJobObject post-spawn could miss a grandchild forked before assignment. Creating the child suspended and resuming after assignment guarantees it is in the job before running any code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-14 09:57:52 +02:00
Ruben Fiszel aa12c66c25 feat(snowflake): derive public key from private key when omitted (WIN-1959) (#9251)
* feat(snowflake): derive public key from private key when omitted (WIN-1959)

Snowflake key-pair auth needs a SHA256 fingerprint of the public key for
the JWT iss claim, but the public key is mathematically derivable from
the RSA private key. Other tools (e.g. Power BI) only require the
private key, so requiring users to supply both is redundant. When
public_key is missing, fall back to deriving it from private_key (PKCS#8
or PKCS#1 PEM) instead of erroring out.

Fixes WIN-1959

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(snowflake): treat empty public_key/private_key as missing

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 06:48:10 +00:00
centdix 17cf538a2d refactor: move ai providers to windmill-ai (#9120) 2026-05-12 14:09:31 +00:00
Ruben Fiszel e1a7c75e19 perf(flows): cache resolved flow_env per flow execution (#9079)
* perf(flows): cache resolved flow_env per flow execution

* perf(flows): tighten flow_env cache cap to 1024 and clarify memory note

* perf(flows): don't cache transient flow_env resolution failures
2026-05-08 07:43:46 +02:00
centdix 26a6d1e4ce refactor: create windmill-ai crate (part 1 — types, traits, base modules) (#8530)
* refactor: create windmill-ai crate and move base AI types from windmill-common

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move worker AI types to windmill-ai crate

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: move QueryBuilder trait and StreamEventSink abstraction to windmill-ai

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add base64 dependency to windmill-ai for bedrock PDF support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add windmill-ai refactor plan

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor: address PR review — remove dead bedrock feature, add boxed_sink helper, move plan to docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-22 11:20:27 +00:00
hugocasa 172a7d16db fix: fix otel tracing on nativets
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-17 14:05:58 +00:00
Ruben Fiszel c57c769dea feat: add CI test scripts with auto-trigger on deploy (#8736)
* feat: add CI test scripts with auto-trigger on deploy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fix annotation parser early return and handle renames correctly

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move CI test results to top of script/flow detail pages

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: improve CI test results spacing, icon, and remove pass label

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: support one-line annotation and use script/path format

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI test trigger logic to EE

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: move CI badge next to New badge and add deduplicated CI summary

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add CI test e2e tests and fix nullable column annotations

Add integration tests for CI test annotation parsing (creates/removes
ci_test_reference rows) and the CI test results API (single + batch
endpoints). Add backend test for auto-trigger on deploy (private+python).

Fix sqlx LEFT JOIN LATERAL nullable column annotations in
get_ci_test_results and get_ci_test_results_batch queries — sqlx
cannot infer nullability from LATERAL subqueries, causing runtime
decode errors when no matching job exists.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix build/sqlx

* fix

* feat: CI test improvements and templates

- Fix windmill-dep-map/private feature propagation in worker, api-scripts,
  and api-flows Cargo.toml so CI test triggers actually fire in EE mode
- Clone ci_test_reference rows during workspace fork
- Add polling to CiTestResults component (refetch every 3s while running)
- Add running state and auto-refresh to ForkWorkspaceBanner CI summary
- Add yellow "CI test" badge on script list rows and detail page
- Fix Library badge border color (remove indigo border override)
- Add CI Test TypeScript and CI Test Python templates in ScriptBuilder
- Update sqlx offline cache
- Add debug tracing for CI test trigger in worker_lockfiles

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing children prop to WorkspaceDeployLayout

Fixes svelte-fast-check type error when passing named snippets as
children content inside the component tag.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review feedback

- Remove empty wrapper divs around CiTestResults, move mb-4 into component
- Add batch endpoint size cap (max 200 items)
- Add ON DELETE CASCADE to ci_test_reference workspace FK (new migration)
- Downgrade CI test trigger logs from info to debug
- Fix false-positive polling: only treat status='running' as running,
  not null status (CiTestResults, CompareWorkspaces, ForkWorkspaceBanner)
- Fix test numbering in integration tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to latest EE commit

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to d9d68c2406df0b59f413ea0b2cb24780a9817d04

This commit updates the EE repository reference after PR #516 was merged in windmill-ee-private.

Previous ee-repo-ref: d7ccd9b86da99ec056a0e8708e3637d64290387a

New ee-repo-ref: d9d68c2406df0b59f413ea0b2cb24780a9817d04

Automated by sync-ee-ref workflow.

* fix: treat queued jobs (job_id set, null status) as running

Jobs that have been pushed but not yet picked up by a worker have a
job_id but null status. Treat these as 'running' to avoid showing
misleading 'pass' badges or '0 passing'. Tests that were never
triggered (no job_id, null status) remain neutral/hidden.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: hugocasa <hugo@casademont.ch>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-04-09 17:21:36 +00:00
Ruben Fiszel d2d6810db9 feat: add LIMIT_WINDOWS_TO_1CU env var for Windows worker memory limits (#8681)
* feat: add LIMIT_WINDOWS_TO_1CU env var for Windows worker memory limits

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address CI review — stricter env var parsing and SAFETY comment

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-02 19:31:07 +00:00
Ruben Fiszel a46aa641f9 feat: add R language support (#8263)
* feat: add R language support

Add R as a new supported scripting language in Windmill, following the
same pattern used for Ruby. Includes:

- Backend: ScriptLang::Rlang enum variant, DB migration, tree-sitter-r
  parser crate with tests, WASM parser binding, R executor with NSJail
  sandboxing, job dispatch and signature parsing
- Frontend: language picker, R icon, syntax highlighting, editor bar
  insertions (Sys.getenv, get_variable, get_resource), schema inference,
  init code template, BETA badge
- CLI: .r extension mapping, sync support, bootstrap template

R scripts use `main <- function(...)` syntax, jsonlite for JSON
serialization, and system curl for the Windmill client helper.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add R package resolution and installation

Parse library()/require() calls from R scripts to extract dependencies.
Resolve versions from CRAN, cache lockfiles in pip_resolution_cache,
and install packages to a shared R library cache. The run step sets
R_LIBS_USER so installed packages are available to the script.

- Parser: parse_r_requirements() extracts package names from AST
- Executor: resolve() generates lockfile, install() installs from CRAN
- Worker lockfiles: wire up R resolve for dependency jobs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add nsjail sandboxing for R resolve and install phases

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix R get_variable/get_resource and add sandbox annotation + e2e tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: fix R arg inference with JS fallback parser and get_variable/get_resource

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix flake

* nsjail

* nits

* fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits

- Suppress renv verbose output during resolve and install (controlled by #verbose annotation)
- Filter renv from install list (already loaded, causes noisy restart message)
- Log compact "resolved N packages" instead of full renv.lock JSON
- Add R (r-base, r-cran-renv) to DockerfileFull and DockerfileFullEe
- Use disable_rl for nsjail install config (R compiles from source)
- Reduce default concurrency from 20 to 5
- Add rlang to openflow.openapi.yaml
- Fix MainArgSignature (no_main_func -> auto_kind) after main merge

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* final

* fix: remove accidental R install from multiplayer Dockerfile

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: remove R from Windows build and DockerfileExtra

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: rename R migration to avoid timestamp collision with trigger_filter_logic

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* fix: R install improvements - suppress verbose output, flat lockfile logging, Dockerfile R support, rlimits

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add clear error when Rscript binary is missing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: fix type errors in R fallback parser, use format! in wrap(), add R system prompts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: pyranota <pyra@duck.com>
2026-04-01 06:11:37 +00:00
Ruben Fiszel ce2e6c8c01 fix: add Authority Key Identifier to MITM proxy leaf certs (#8576)
* test: add x509-parser dev-dep for MITM proxy cert tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt for ssl-verify-fix branch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref.txt

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update ee-repo-ref to a90b083660b372bf1da1c18769cbd50936ea8040

This commit updates the EE repository reference after PR #494 was merged in windmill-ee-private.

Previous ee-repo-ref: db665a09d5b9a485977d73c22908629e3dda6200

New ee-repo-ref: a90b083660b372bf1da1c18769cbd50936ea8040

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 23:49:40 +00:00
Alexander Petric 56253c04cb feat: IAM RDS auth for PostgreSQL worker resources (#8573)
* feat: add IAM RDS auth support for PostgreSQL worker resources

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: use Config builder for IAM RDS connections

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for IAM RDS auth

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: update ee-repo-ref to ebea6ef1e5bfcfc3f0151da9687dac6c61bbfab6

This commit updates the EE repository reference after PR #493 was merged in windmill-ee-private.

Previous ee-repo-ref: 1228561a98c5195bb97a81d4a57ce2bb2ecfca79

New ee-repo-ref: ebea6ef1e5bfcfc3f0151da9687dac6c61bbfab6

Automated by sync-ee-ref workflow.

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2026-03-27 21:50:29 +00:00
Ruben Fiszel d578e40101 feat: add selfApproval option to WAC + inline approval buttons (#8440)
* feat: add selfApproval option to WAC waitForApproval + inline approval buttons

Add self-approval configuration to WAC workflows and inline
approve/reject buttons in WorkflowTimeline.

- TS SDK: add selfApproval option to waitForApproval()
- Python SDK: add self_approval param to wait_for_approval()
- Backend: store approval_conditions in flow_status for WAC,
  enforce self-approval checks on resume endpoints
- Frontend: show Approve/Reject buttons in timeline with form
  support (EE), gated by user permissions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: revert sqlx query change + regenerate system prompts

- Revert get_suspended_flow_info to use original sqlx::query_as!
  with COALESCE to avoid sqlx offline cache mismatch in CI
- Detect WAC by checking if FlowStatus parsing fails + suspend > 0
- Re-fetch flow_status column separately for WAC approval conditions
- Regenerate auto-generated system prompt files for SDK changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: use resume URLs for WAC inline approval buttons

- Backend generates HMAC-signed resume/cancel URLs when creating
  WAC approval, stores them in timeline entry and approval meta
- Frontend uses anonymous resume endpoint (like classic flows)
  with fallback to resumeSuspendedFlowAsOwner for admins
- Buttons show for everyone when URLs are present; server-side
  self_approval_disabled check enforces restrictions
- Show warning for admins/owners when self-approval is disabled
- selfApproval: false requires EE (errors at dispatch on CE)
- self_approval_disabled check moved outside user_auth_required
  gate so it works independently
- WAC detection no longer requires task import

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add resume_suspended and approval_info endpoints

- New approval_token DB table for token-based approval access
- New POST /jobs_u/flow/resume_suspended/{job_id} endpoint:
  - OptAuthed: works with login or approval_token
  - Checks approval_conditions (self_approval, groups, auth)
  - Admins/owners bypass rules
- New GET /jobs_u/flow/approval_info/{job_id} endpoint:
  - Returns form, rules, can_approve status
- HMAC anonymous endpoint now bypasses all approval_conditions
  (secret = full capability)
- getResumeUrls approvalPage URL now uses token format
- WAC approval dispatch generates and stores approval tokens
- Mark resumeSuspendedFlowAsOwner as legacy

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: simplify frontend to use resume_suspended endpoint

- OpenAPI spec updated with resume_suspended and approval_info endpoints
- WorkflowTimeline: removed URL parsing, now calls single
  resumeSuspended endpoint for both approve and reject
- Buttons show for any logged-in user viewing the job (backend
  enforces authorization rules)
- Kept self-approval warning for admins

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: stateless approval tokens, new approval page, FlowStatusWaitingForEvents update

- Replace DB-stored approval tokens with stateless HMAC derivation:
  token = HMAC(workspace_key, job_id + "approval_token")
  Verifiable without DB lookup, not reversible to resume secret
- Drop approval_token migration (no DB table needed)
- FlowStatusWaitingForEvents: use resumeSuspended endpoint instead
  of URL parsing + resumeSuspendedFlowAsOwner
- New approval page route /approve/{ws}/{job}?token= that uses
  approval_info and resume_suspended endpoints
- Old approval page route kept for back-compat

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: match old approval page content in new approval page

- Add FlowMetadata, JobArgs, FlowGraphV2, DisplayResult
- Add approvers with tooltips, flow arguments section
- Add admin self-approval bypass warning
- Add "Open run details" link
- Fetch full job alongside approval_info for all UI data

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: filter _MODULES from args, show 'workflow' for WAC approvals

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove deno template from approval/prompt SuspendDrawer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: approval page form display + hide deno from approval script picker

- Fix form schema rendering on new approval page by wrapping flat
  WAC form schemas in { properties, order } for SchemaForm
- Hide deno from the approval step language picker in flow editor

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove deno from canHaveApproval in script_helpers.ts

The insert menu uses canHaveApproval() from script_helpers.ts via
FlowInputsQuick, not the displayLang function in FlowInputs.svelte.
Revert the unnecessary FlowInputs.svelte change.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: return form schema and description in approval_info for classic flows

The approval_info endpoint was returning None for form_schema on
classic flows. Now fetches raw_flow to get suspend.resume_form
schema, hide_cancel, and the step's completed result for description.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: inline Login component on approval page instead of redirect

Show the Login component directly on the approval page when
authentication is required. On successful login, reloads user
and approval info without navigating away.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: show resume buttons for all users, not just owners

The resume_suspended endpoint handles authorization server-side,
so the frontend should always show the buttons. Remove isOwner
gate and the "cannot resume" message.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent layout shift on resume by removing spinner from cancel button

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prevent resume button expansion by using disabled instead of loading

The loading prop adds a Loader2 spinner that expands the button width.
Use disabled={loading} instead to prevent layout shift.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: approval page login redirects back with full page reload

Set rd to the full URL (starts with http) so Login.redirectUser()
uses window.location.href instead of goto(), triggering a full page
reload after login. This ensures the approval page re-fetches data
as an authenticated user.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: fetch flow definition from flow_version when raw_flow is null

Deployed flows don't store raw_flow on the job. Fall back to
flow_version table using runnable_id to get suspend settings
(form schema, hide_cancel) for the approval_info endpoint.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: show specific reasons when user cannot approve

Display whether denial is due to self-approval being disabled,
required group membership, or both.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: support both nested and flat form schema in waitForApproval

Users can now pass either:
  waitForApproval({ form: { schema: { name: { type: "string" } } } })
or:
  waitForApproval({ form: { name: { type: "string" } } })

Both WorkflowTimeline and approval page handle both formats.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: convert sqlx query macros to non-macro for CI offline cache

Replace sqlx::query! and sqlx::query_scalar! with sqlx::query and
sqlx::query_as to avoid SQLX_OFFLINE cache misses in CI.
Also remove unused LogIn import from approval page.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: suppress dead code warning + unused isOwner variable

- Add #[allow(dead_code)] to without_flow method (CI -D warnings)
- Rename isOwner to _isOwner in FlowStatusWaitingForEvents (unused)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: security and robustness fixes from PR review

- Add workspace_id verification in resume_suspended to prevent
  cross-workspace approval (#3)
- Fix token leakage: use relative path for login redirect instead
  of full URL with token (#4)
- Handle getJob failure independently from approval_info so the
  page works for unauthenticated users (#7)
- Clear error state on successful data load (#13)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address review feedback — shared token gen, rand resume_id, UX

- Move generate_approval_token to windmill-common::variables (shared
  between windmill-api and windmill-worker, eliminates duplicate HMAC)
- Use rand::random::<u32>() for resume_id instead of DefaultHasher
- Stop polling after approve/reject on approval page
- Add cancelLoading state to WorkflowTimeline Reject button

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 21:22:35 +00:00
Ruben Fiszel 5296adeddf test: add powershell module detection and execution tests (#8373)
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 04:11:43 +00:00
Ruben Fiszel 5f0ef936d1 feat: add sandbox annotations, volume mounts, for AI sandbox starting with claude (#8058) 2026-03-05 06:19:51 +00:00
Ruben Fiszel 6bf544f507 refactor: extract object store into dedicated crate with filesystem backend (#7996)
* refactor: extract object store code into windmill-object-store crate with filesystem backend

Consolidate all object_store-dependent code from windmill-common into a new
windmill-object-store crate. Add a filesystem-backed object store implementation
using LocalFileSystem for dev/testing without cloud credentials. Includes 30
comprehensive tests covering render_endpoint, lfs_to_object_store_resource,
duckdb_connection_settings, error mapping, and filesystem-backed integration tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* all

* all

* all

* fix: fix raw_app hardcoded path, add missing ObjectStoreResource import, and add tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: move S3ModeFormat to windmill-types, make windmill-parser-sql optional, restore debug logs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-18 12:24:45 +00:00
Ruben Fiszel 623407de6a chore: make rquickjs, kube, otel proxy deps optional behind feature flags
- rquickjs: gated behind `quickjs` feature in windmill-jseval, propagated through windmill-worker/windmill-api, added to oss_core
- windmill-autoscaling: made optional in windmill-api (was unconditional), enabled via enterprise feature
- opentelemetry-proto, prost, hudsucker, rcgen, hyper-http-proxy, hyper-tls, hyper-util: made optional in windmill-worker, enabled via enterprise feature

This significantly reduces compilation time for vanilla `cargo check` without features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 16:13:16 +00:00
Ruben Fiszel bb57f8cd29 remove unecessary deps for vanilla cargo run 2026-02-08 14:08:36 +00:00
Ruben Fiszel 3f68e3a58e refactor: isolate deno_core into windmill-runtime-nativets subcrate (#7848)
* refactor: isolate deno_core into windmill-runtime-nativets subcrate

Remove deno_core from flow eval and isolate nativets V8 runtime into a
dedicated subcrate so deno_core compilation no longer blocks
windmill-worker or windmill-api.

- Create windmill-jseval crate: QuickJS-based JS eval for flow
  expressions and batch rerun, extracted from windmill-worker
- Create windmill-runtime-nativets crate: all deno_core/V8 deps and
  nativets script execution, with build.rs snapshot generation
- Simplify windmill-worker: remove all deno_* direct deps, empty
  build.rs, gate nativets behind optional dep
- Update windmill-api: use windmill-jseval for batch rerun instead of
  deno_core, remove deno_core feature entirely
- Add nativets integration tests (nativets_jobs.rs) and parallel
  stress test (nativets_stress.rs, 8 workers x 200 jobs)
- Remove dead code: deno flow eval path, USE_QUICKJS env var,
  parity tests (replaced with 63 standalone expected-value tests)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR review feedback for deno_core isolation

- Deduplicate unsafe_raw() into windmill-common/src/utils.rs (single source)
- Delete orphaned runtime.js and windmill-client.js from windmill-worker/src/
- Fix operator precedence in windmill-jseval with explicit parentheses
- Remove unnecessary return keyword in heap limit callback
- Remove redundant as usize casts
- Remove ~150 lines of commented-out code from runtime.js
- Remove commented-out #[cfg] in build.rs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* otel ee

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 09:16:37 +00:00
Ruben Fiszel e1a815f6a0 refactor: extract windmill-dep-map crate for parallel api/worker compilation (#7846)
* refactor: extract windmill-dep-map crate for parallel api/worker compilation

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve WebhookShared type mismatch and missing enterprise propagation

- Make windmill-api webhook_util re-export from windmill-common instead of
  duplicating types, fixing Extension<WebhookShared> mismatch between
  windmill-store and windmill-api
- Add windmill-api-jobs/enterprise to windmill-trigger enterprise feature
  so check_license_key_valid is available when trigger subcrates enable
  enterprise on windmill-trigger

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: stop trigger features from unconditionally enabling enterprise

Move enterprise propagation for all trigger subcrates from individual
trigger feature definitions to the enterprise feature itself, so
enterprise is only enabled when explicitly requested.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: remove unused pub use re-exports and disable CI cargo cache

- Remove unused re-exports from windmill-worker/src/lib.rs:
  trigger_dependents_to_recompute_dependencies, handle_job_error,
  and unused bun/otel items
- Fix callers to use direct module paths instead
- Add windmill-dep-map as dev-dependency for tests
- Disable cargo cache in backend-check CI (faster from-scratch builds)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore bun re-exports used by tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* all

* chore: re-enable cargo cache for check_ee_full CI job

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-08 00:39:56 +00:00
Alexander Petric afa6e7ab5b feat: mssql integrated auth (gssapi) (#7760)
* feat: mssql integrated auth (kerberos/ntlm)

* install krb5 headers

* also make it work for windows

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-02-05 18:35:31 +00:00
Ruben Fiszel 5c20b37a53 feat: add QuickJS as alternative JS engine for flow expression evaluation (#7664)
* feat: add QuickJS as alternative JS engine for flow expression evaluation

Add rquickjs as an optional alternative to deno_core for evaluating
JavaScript expressions in flow transformations. QuickJS offers ~8-16x
faster startup times for simple expressions, making it ideal for
evaluating many small expressions in flows.

Key changes:
- Add new `quickjs` feature flag for windmill-worker
- Implement js_eval_quickjs.rs with true async Rust callbacks for
  variable(), resource(), and results.xxx access (no pre-fetching)
- Share expression transformation logic (replace_with_await,
  replace_with_await_result) between both implementations
- Add USE_QUICKJS_FOR_FLOW_EVAL env var to switch engines at runtime
- When only quickjs feature is enabled (no deno_core), QuickJS is
  automatically used
- Add comprehensive parity tests comparing QuickJS and deno_core output

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* all

* quickjs

* quickjs

* all

* all

* all

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 01:44:40 +00:00
Pyra c5963957b6 feat: add otlp/http internal collector (#7690)
* feat: add otlp/http internal collector [merge-ee-first]

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to b2e51eceaf00719b7b6794b516d42f20a1d96ff7

This commit updates the EE repository reference after PR #390 was merged in windmill-ee-private.

Previous ee-repo-ref: b576da804a0dd3fadf1ad2fbcc0a0df12684d795

New ee-repo-ref: b2e51eceaf00719b7b6794b516d42f20a1d96ff7

Automated by sync-ee-ref workflow.

* fix panic

Signed-off-by: pyranota <pyra@duck.com>

* finish

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to c23a9fb439a71bf574139422b8a43770167ed233

This commit updates the EE repository reference after PR #392 was merged in windmill-ee-private.

Previous ee-repo-ref: 657a61696699d7d9751dc8e30935372bb5af2db7

New ee-repo-ref: c23a9fb439a71bf574139422b8a43770167ed233

Automated by sync-ee-ref workflow.

* defensive programming

Signed-off-by: pyranota <pyra@duck.com>

* ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* Apply suggestions from code review

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* fix silly claude

Signed-off-by: pyranota <pyra@duck.com>

* chore: update ee-repo-ref to e9f7a4485b2d21746c6a4a4c38fa1cbac2b942af

This commit updates the EE repository reference after PR #393 was merged in windmill-ee-private.

Previous ee-repo-ref: bc6c149c01d7063171488b7930f40ffce73a65d3

New ee-repo-ref: e9f7a4485b2d21746c6a4a4c38fa1cbac2b942af

Automated by sync-ee-ref workflow.

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
2026-01-26 19:29:09 +00:00
centdix 2553d987d9 feat(ai): native bedrock compatibility (#7668)
* check endpoint

* use flag for cred check on worker

* use sdk for auth chat side

* cleaning

* cleaning

* also handle GET

* Add session token support in BedrockClient::from_credentials

Users with temporary STS credentials need session tokens for AWS
authentication. This adds the optional session_token parameter to
support these use cases.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Cache AWS SDK config loading to avoid repeated I/O

aws_config::load_defaults() performs environment variable lookups and
potentially file I/O on each call. Cache the result in a static OnceCell
to improve performance on high-frequency Bedrock requests.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add debouncing to Bedrock credential check button

Prevent rapid clicks from spawning multiple concurrent flow preview
jobs for the worker credential check.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove orphaned SigV4 signing comment

The manual SigV4 signing code was removed in favor of the AWS SDK,
but this documentation comment was left behind.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Remove verbose tracing info and change-related comments

- Remove per-request auth method info logs (too verbose for production)
- Simplify from_env log to single debug-level message
- Remove comments describing future changes rather than current behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* Fix Bedrock feature flag compilation warnings

Gate AWS-specific imports, struct fields, and methods behind the bedrock
feature flag to eliminate dead code warnings when building without the
bedrock feature enabled.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* bedrock flag

* cleaning

* Move BedrockCredentialsCheck from edit drawer to add drawer

The component was incorrectly shown when editing resources instead of
when adding new ones. Moved it from ResourceEditor.svelte to
ApiConnectForm.svelte.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* cleaning

* cleaning

* make aws-config optional

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2026-01-26 11:04:14 +00:00
Ruben Fiszel 0b6100c8b2 clean up cargo + update rmcp 2026-01-26 08:04:17 +00:00
Pyra 95df7b9a6a feat: otel REST tracing (#7571) 2026-01-20 13:38:02 +00:00
Ruben Fiszel d432cdbec1 split out windmill-mcp + pin tokio-utils 2026-01-09 06:16:26 +00:00
centdix a8e34de58d upgrade rmcp crate (#7518) 2026-01-07 16:02:51 +00:00
centdix 79ac6312e8 feat(ai): handle aws bedrock as provider (#7155)
* backend draft

* fix for tool and streaming

* do frontend side

* working

* working tools

* rm

* handle list endpoint

* handle for ai agents

* fix for models requiring inference id

* cleaning

* fix desc issue

* fix tool usage

* fix structured output

* cleaning

* fix for api

* rm

* fix input images

* cleaning

* chore: use aws sdk (#7156)

* feat(ai): Add AWS SDK dependencies for Bedrock integration

- Add aws-sdk-bedrockruntime v1.113.0
- Add aws-credential-types for bearer token authentication
- Update rustls to v0.23.35 for compatibility
- Dependencies added to windmill-common for AI features

* feat(ai): Add bearer token provider for Bedrock authentication

- Implement BearerTokenProvider using aws_credential_types
- Simple token-based auth using API keys from Windmill resources
- Add basic unit tests for provider creation
- Export bedrock_auth module in lib.rs

* feat(ai): Add Bedrock client wrapper with region extraction

- Implement BedrockClient wrapper around AWS SDK client
- Bearer token authentication integration
- Extract AWS region from Bedrock base URL automatically
- Comprehensive unit tests for region extraction
- Make aws-config non-optional dependency for AI features
- Update feature flags to reflect new dependency structure

* cargo

* feat(ai): Implement non-streaming Bedrock via AWS SDK

Use official AWS SDK instead of manual HTTP requests for better type safety and maintainability. Implements the Bedrock converse() API for non-streaming requests with proper bearer token authentication and message format conversion between OpenAI and Bedrock formats.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor(ai): Eliminate Simple* conversion types for Bedrock SDK

- Move AI types to windmill-common/src/ai_types.rs for shared access
- Update bedrock_converters to work directly with OpenAI types
- Remove ~200 lines of conversion boilerplate from ai_executor.rs and bedrock.rs
- Remove unused imports to clean compilation warnings
- Benefits: 50% fewer conversion steps, no information loss, easier maintenance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(ai): Add streaming support for AWS Bedrock SDK

- Implement converse_stream() for Bedrock streaming responses
- Use EventReceiver.recv() to process stream events
- Extract text deltas using bedrock_stream_event_to_text()
- Send TokenDelta events to StreamEventProcessor for real-time updates
- Refactor request building to eliminate duplication between streaming and non-streaming
- Clean, minimal implementation following AWS SDK patterns

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* revert flake change

* fix

* feat(ai): Add tool calls and image support for Bedrock streaming

**Phase 1: Streaming Tool Call Support**
- Add stream event processing functions in bedrock_converters.rs:
  - bedrock_stream_event_to_tool_start() - Extract tool use start from ContentBlockStart
  - bedrock_stream_event_to_tool_delta() - Extract tool input deltas from ContentBlockDelta
  - bedrock_stream_event_is_block_stop() - Detect ContentBlockStop events
  - streaming_tool_calls_to_openai() - Convert accumulated tool calls to OpenAI format
- Update ai_executor.rs streaming loop with tool call accumulator (HashMap)
- Track current tool use ID during streaming
- Send ToolCallArguments events to StreamEventProcessor
- Return accumulated tool calls instead of empty vector

**Phase 2: Image Input Support**
- Add parse_image_data_url() to extract format and base64 data from data URLs
- Add content_part_to_block() to convert ContentPart to Bedrock ContentBlock
- Refactor convert_message() to handle multi-part content with images
- Support ImageUrl conversion to Bedrock ImageBlock with proper format (png/jpeg/gif/webp)
- Import AWS SDK image types: ImageBlock, ImageSource, ImageFormat
- Keep content_to_text() helper for system message text extraction

**Benefits**:
-  Tool calling now works in both streaming and non-streaming modes
-  Images are properly converted instead of being silently dropped
-  Structured output works in streaming (uses tool calling)
-  Full feature parity with manual HTTP implementation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* fix(ai): Add S3 image support and structured output for Bedrock

**Fixes:**
1. **S3 Image Support**: Call prepare_messages_for_api() before Bedrock SDK path to convert S3Objects to ImageUrls
   - Downloads images from S3 and encodes as base64 data URLs
   - Ensures images are properly handled in both streaming and non-streaming modes

2. **Structured Output**: Add ToolChoice::Any when structured output tool is present
   - Forces Bedrock to call the structured_output tool
   - Ensures JSON schema compliance for structured output
   - Works in both streaming and non-streaming modes

**Changes:**
- ai_executor.rs: Call prepare_messages_for_api() for Bedrock SDK path
- ai_executor.rs: Set tool_choice to Any when structured_output_tool_name is present
- aws_bedrock.rs: Remove unused ToolChoice imports (used via full path in worker)

**Testing:**
-  S3 images are now downloaded and converted before API call
-  Structured output now forces tool usage with ToolChoice::Any
-  Both work in streaming and non-streaming modes

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* cleaning

* cleaning

* better error

* cleaning

* cleaning

* rm

* rename

* apply region

---------

Co-authored-by: Claude <noreply@anthropic.com>

* fix default

* no panic

* no print

* use utils file

* cleaning

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-17 18:57:59 +00:00
hugocasa 2e245accc5 fix(backend): improve sse parsing (#7100) 2025-11-10 14:32:16 +00:00
centdix 97ac1be036 feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
hugocasa d365014882 build on macos with deno_core (#6642) 2025-09-19 17:33:05 +00:00
centdix 3199f9fffd feat(ai agent): allow multiple images input for ai agent + code cleaning (#6591)
* add in frontend

* draft openai handling

* upload to s3

* simpler output

* return s3 directly if any

* low quality

* implement for gemini

* handle imagen model

* handle image input

* cleaning

* remove base64 from output

* cleaning

* fix timeout

* handle openrouter

* remove log

* allow image input when creating image

* cleaning

* increase stack size

* inline everything

* revert stack size

* refactor: move AI executor types to separate module

- Created ai module structure with types.rs
- Moved all type definitions from ai_executor.rs to ai/types.rs
- No functional changes, just code organization

* refactor: add QueryBuilder trait and provider detection utilities

- Created QueryBuilder trait for abstracting provider-specific logic
- Added helper functions for provider detection (is_anthropic_provider)
- Implemented placeholder QueryBuilder for all providers
- Updated OpenAIRequest to use slices instead of Vec references
- All providers now have QueryBuilder implementations (using default for now)

* feat: implement OpenAI query builder with image support foundation

- Created proper OpenAI query builder implementation
- Added image_handler module for S3 upload/download utilities
- Separated text and image request building logic
- Added prepare_messages_for_api to handle S3Object conversion
- Foundation laid for supporting tools with image output

* refactor(ai): complete AI executor refactoring with query builder pattern

- Created modular structure under ai/ module
- Moved all types to ai/types.rs
- Created QueryBuilder trait for provider abstraction
- Implemented OpenAI query builder with image+tools support
- Added unified agent runner supporting both text and image outputs with tools
- Refactored run_agent to delegate to new unified implementation
- Added image handler utilities for S3 operations
- Improved code organization and maintainability

* cleaning

* feat(ai): implement remaining provider query builders

- Added Anthropic query builder with proper message conversion
- Added Google AI query builder with Gemini API support
- Added OpenRouter query builder delegating to OpenAI for compatibility
- Added missing Anthropic and Gemini types to types.rs
- Fixed type references and compilation errors
- All providers now support the unified query builder interface

* fixes

* fixes

* mime type + cleaning

* image to images

* handle mutlitple images

* fix

* remove agent_runner file

* clean query builder logic

* cleaning

* cleaning

* hide structured_output based on output type

* fix

* user images and not nested

* better descriptions
2025-09-17 16:55:54 +00:00
centdix 20f48e6ded feat(ai agent): handle images in ai agent (#6572)
* add in frontend

* draft openai handling

* upload to s3

* simpler output

* return s3 directly if any

* low quality

* implement for gemini

* handle imagen model

* handle image input

* cleaning

* remove base64 from output

* cleaning

* fix timeout

* handle openrouter

* remove log

* allow image input when creating image

* cleaning

* increase stack size

* inline everything

* revert stack size

* cleaning

* fix for openai

* better mime type

* add descriptions
2025-09-15 07:51:09 +00:00
Diego Imbert 304f953710 Separate duckdb crate to fix c++ build/link issues (#6551)
* call ffi

* remove duckdb dep

* rename windmill_duckdb_ffi_internal

* static lib

* ci

* back to dylib, bug isn't fixed in static

* feature flag and copy dynamic lib

* fix dynlib in docker

* load libwindmill_duckdb_ffi_internal at runtime on usage

* lazy static deadlocks

* Cache dynamic library handles

* update auto s3 path insert from editor bar

* Fix duckdb S3 freezing worker because of blocking task in tokio async context

* build dll windows GH workflow

* try fix windows build

* revert build.rs

* nit fixes CI

* Dockerfile update (not tested yet

* build dev sh for duckdb lib

* mistake

* attach windmill_duckdb_ffi_internal.so artefact

* rhel9

* docker fixes

* fix dockerfile

* better err msg

* forgot lib prefix .so

* add column_order

* fix column_order
2025-09-10 14:43:46 +00:00
dieriba f10cac1c4b feat: conditional retry workflow (#6461)
* feat retry

* fix

* fix

* fix and nits

* fix query

* fix test

* nits and perf

* update sqlx

* once

* remove

* update sqlx
2025-09-03 20:24:38 +00:00
hugocasa 958e8af782 feat: ai agent steps (#6393)
* feat: ai agent steps base

* better backend and graph

* feat: anthropic, log viewer

* nit

* fix(frontend): hide tool nodes from timeline

* move ai agent actions from flow status to flow status module

* nits and workspace/hub scripts support

* tmp ref

* fix merge

* feat: display agent tools status in the graph

* fix reactivity

* fix flow status

* nit
2025-08-20 22:40:57 +00:00
pyranota 11dd4118ce feat: add ruby support (#5939)
* implement single line pin

* make panic-safe

* use pin even if multiple modules imported withing single statement

* add repins and make imports respect pins

* keep all pins

* Allow multiple pins

* add comments + handle stuff more safely

* fix fully qualified imports

* remove ignore

* sort nested

* apply unique to output requirements list

* fix typo

* remove mut

* update sqlx

* sort imports

* sort imports

* fix formatter and format

* feat: add ruby support

* fix

* update sqlx

* add ruby to flake.nix

Signed-off-by: pyranota <pyra@duck.com>

* upgrade par_install_language_dependencies helper

rename fields + allow passing custom payload

Signed-off-by: pyranota <pyra@duck.com>

* change migrations

Signed-off-by: pyranota <pyra@duck.com>

* make full pipeline work!

Signed-off-by: pyranota <pyra@duck.com>

* do a bit fixing

* stage

Signed-off-by: pyranota <pyra@duck.com>

* add ruby in dockerfiles

Signed-off-by: pyranota <pyra@duck.com>

* implement auto-require + 'windmill/inline'

Signed-off-by: pyranota <pyra@duck.com>

* create 'windmill/inline' only if does not exist

Signed-off-by: pyranota <pyra@duck.com>

* add nsjail

Signed-off-by: pyranota <pyra@duck.com>

* update global settings

Signed-off-by: pyranota <pyra@duck.com>

* add ruby icon

* generate lockfile on deploy (but broken)

Signed-off-by: pyranota <pyra@duck.com>

* fix(frontend): display deployed script lockfile and other info

Signed-off-by: pyranota <pyra@duck.com>

* feat: add mini-wmill client

Methods:
- get_resource
- get_variable

Additionally add shortcuts on frontend

Signed-off-by: pyranota <pyra@duck.com>

* add nit warning

Signed-off-by: pyranota <pyra@duck.com>

* support private repos credentials

Signed-off-by: pyranota <pyra@duck.com>

* fix compilation error

Signed-off-by: pyranota <pyra@duck.com>

* found rustc bug + refactor universal installer

Signed-off-by: pyranota <pyra@duck.com>

* more refactor + fixes

Signed-off-by: pyranota <pyra@duck.com>

* bug fixing + polishing

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* change 'windmill/inline'

Signed-off-by: pyranota <pyra@duck.com>

* delegate env_clear to universal installer

Signed-off-by: pyranota <pyra@duck.com>

* update init script

Signed-off-by: pyranota <pyra@duck.com>

* add cli support

Signed-off-by: pyranota <pyra@duck.com>

* unhardcode frontend

Signed-off-by: pyranota <pyra@duck.com>

* fix cli

Signed-off-by: pyranota <pyra@duck.com>

* fix init script

Signed-off-by: pyranota <pyra@duck.com>

* make it work on ms windows

Signed-off-by: pyranota <pyra@duck.com>

* do not clean env in the end

Signed-off-by: pyranota <pyra@duck.com>

* fix windows (again)

Signed-off-by: pyranota <pyra@duck.com>

* include PROXY_ENVS (untested)

Signed-off-by: pyranota <pyra@duck.com>

* feat: self-signed certs + proxy support (tested)

Signed-off-by: pyranota <pyra@duck.com>

* flake: pin ruby to 3.4

* merge follow-ups

Signed-off-by: pyranota <pyra@duck.com>

* remove obsolete libc crate

Signed-off-by: pyranota <pyra@duck.com>

* cleanup parser

Signed-off-by: pyranota <pyra@duck.com>

* fix java ci

Signed-off-by: pyranota <pyra@duck.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fix nits

Signed-off-by: pyranota <pyra@duck.com>

* Update backend/windmill-worker/src/universal_pkg_installer.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update backend/windmill-worker/src/ruby_executor.rs

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* add nsjail for lock stage

Signed-off-by: pyranota <pyra@duck.com>

* fix merge

Signed-off-by: pyranota <pyra@duck.com>

* Update common.rs

* regen windmill-api-client

Signed-off-by: pyranota <pyra@duck.com>

* fix tests

Signed-off-by: pyranota <pyra@duck.com>

* fix tests again

Signed-off-by: pyranota <pyra@duck.com>

* reference published ruby parser

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-08-15 13:03:32 +00:00
Ruben Fiszel b83aca30d2 feat: use process groups to improve zombie job handling (#6157)
* useProcessGroups

* nit
2025-07-10 08:49:56 +00:00
Diego Imbert 0e316239dd EE Refactor (#5844)
* app compiles with every ee substituted

* Replace all oss files content

* Revert "Replace all oss files content"

This reverts commit ea4017d59f.

* delete all ee

* hide all _ee files under private flag

* hide every oss stuff when private flag set

* pub use *

* gitignore and substitute script

* pub mod for ee needed for ee repo

* small mistakes

* remove oidc_oss impl

* ee ref (temp)

* ee ref

* fix --all-features selecting private in OSS CI

* ee repo ref

* allow unused
2025-06-02 22:12:33 +02:00
Ruben Fiszel 0bdcad80ce duckdb optional dep 2025-05-29 14:54:56 +02:00
Diego Imbert fdefd4be93 feat: duckdb sql lang support (#5761) 2025-05-27 15:52:57 +02:00
pyranota 26222539e6 feat(python): inline script metadata (PEP 723) (#5712)
* make resolver

* more updates

* fix build

* fix raw_dependencies job type

* compat with http agent workers

* refactor

* rename

* more refactor

* cleanup

* more tests

* fix s3

* small fixes

* more fixing

* fix endpoint

* nit: update comment

* update ee ref

* update ee ref

* update ee ref

* implement safer `list_available_python_versions`

* add tracing to get of authed client

* internal: Trigger claude when commenting with /aider (#5783)

* add claude instructions files

* call claude too when using aider

* fix

* add draft for linear claude integration

* fix build

* update ee ref

* ignore versions <=3.9

* fix windows build

* correct versions filter

* fix windows build (this time for real)

* inject error to debug CI

* update CI

* undo debug of CI

* fix tests

* remove outdated comment

* update ee repo ref

* Update ee-repo-ref.txt

* Update backend/parsers/windmill-parser-py-imports/src/lib.rs

Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>

* Update InstanceSetting.svelte

---------

Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Co-authored-by: centdix <40307056+centdix@users.noreply.github.com>
Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
2025-05-24 02:13:47 +02:00
Diego Imbert 669a95c3be oracle support for s3 streaming + misc (#5738)
* changed SQL templates to include S3

* oracle support for s3 streaming
2025-05-14 15:45:00 +02:00
Diego Imbert c7886ea07a feat: sql jobs outputting to s3 + streaming for high-number of rows (#5704)
* stream to s3 boilerplate

* S3 works with new syntax

* snowflake s3 streaming support

* postgres s3 support

* fix postgres stream format

* mysql s3 streaming

* mssql s3 streaming

* new s3 mode syntax

* optional folder param

* rename folder to prefix

* json_stream_arr_values

* cargo toml rollback

* convert_ndjson with datafusion

* format conversion kinda works

* Fixed not finishing the datafusion writer

* support for pg and mssql

* fix file ext

* bigquery conversion and works with s3 streaming

* fix s3 flag parser

* snowflake s3 streaming support

* factor out duplicate code

* remove anyhow

* Err case for parse s3 mode

* Send error to mpsc

* bigquery s3 streaming fix for huge queries

* remove extra stuff

* snowflake s3 streaming support

* small regex mistake

* cfg(not(feature = "parquet"))

* fix CI (unused import)

* error handling fix (graphite)
2025-05-13 10:19:44 +02:00
Ruben Fiszel d5186da271 allow multiple workers on agent mode (#5607) 2025-04-11 21:19:29 +00:00
Ruben Fiszel 63fa499015 feat: agent workers v2 using http (#5588) 2025-04-10 00:56:11 +00:00
pyranota 59740c0478 feat: add java support (#5458)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* start working on java

* do java boilerplate

* implement parser for java

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* java is S3 + Caching enabled

* install nsjail backup

* commit v0

* fix nsjail

* v0.1

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* refactor install phase

* implement .valid.windmill atomic verification

* implement java init functionality

* remove quick-xml

* fix windows not recognizing 'mvn'

* create empty settings.xml if there is no config provided

* clean up

* change default settings.xml

* change classpath format for windows

* docs to helper

* java copy bin cache instead of symlink

* remove comments

* merge

* fix package.json

* fix package.json 2

* minor fixing

* migrate to Coursier

* update misc

* Http(s) Proxy + CA certs

* remove unused .wasm

* make requirements insensitive to spaces

* update handle_child refs

* rework save_cache for directories

* fix s3 bug

* compile .wasm for cli

* remove uuid import

* fix compilation

* use reference

* fix zero-dep failure

* removing unsafe stuff

* remove unneeded imports

* revert: we still need winapi

* remove nix store from nsjail

* do not create cache_nomount

* add java to dnt

* remove duplicated dependency in init script

* fix typos

* fix CI

* use published parser
2025-03-27 15:12:55 +01:00
Alexander Petric f3c32c236f fix win build (#5494)
* add sysinfoapi feature flag for winapi dependency

* add ff

* add ff at the right place
2025-03-20 20:28:15 +00:00
pyranota a3faea16e7 feat: add nu-lang support (#5217)
* feat: add nu (nushell) support

* add worker tests

* deactivate tables and non-any types below top-level

full support will come in V1
for V0 it's better to keep things minimal and simple

* add syntax highlighting

used python's grammar, since nushell isn't supported by monaco nor svelte-highlights

for V1 nu will get it`s own grammar

* add logo

* partially implement plugin support

* change logo + ability to deploy + nsjail draft

* static variables + get_resource + get_variable

* lsp/dev.nu + initial nu lsp (not working yet)

* make it work with nsjail

* nullguard

* Much more flexible signature parsing and better error-messages

* add init script

* rename nulsp to nu

* install nu to dockerfile

* fix merge

* implement Default for MainArgSignature

* stage NU_CACHE_DIR

* improve dockerfiles

* dev.nu for parser-wasm + flake.nix

* update code for windows

* add nushell to flake

* upload Cargo.lock

* make build.sh work on nixos

* build wasm cli parsers

* add docs to README_DEV.md

* add helper script docker/dev.nu

* improve docker/dev.nu

* fix windows

* commit frontend/package(lock).json

* update cargo.lock

* correctly update cargo.lock

* remove lsp

* update flake.nix to include svelte server and nushell

* Revert base.sql to main

* remove PLUGIN_USE_RE

* make CARGO_PATH private

* add nu to cli

* Change flags to build wasm-nu-parser

* remove flake.nix from parser-wasm

* update wasm-build target

* remove unused import

* add cli support for nu

* update github workflows

* wasm-build 0.17 -> 0.19

* update build script

* update cargo.lock

* Fix typographical error

* update Cargo.lock

* update ENV_SETTINGS

* use published nu parser

* update package.lock

* rewrite parser in tree-sitter

* implement parser from scratch

* polishing

* change init script to match new parser

* fix imports

* fix cli build

* fix cli build

* merge

* update wasm

* use MiniPulledJob

* update cli

* change cli wasm schema

* change cli

* update deno.json

* make wasm modules load lazily

* regenerate parsers

* remove leftover

* update cargo.lock

* clean up dnt.ts

* add docs to cli/test.nu

* add schema validation option

* add Nu to try_validate_schema

* reference frontend to new parser version
2025-03-20 16:45:21 +01:00
Ruben Fiszel c6dbd239b4 update deno_core implementation to latest 2.2.1 2025-02-22 11:04:54 +01:00