Commit Graph

1785 Commits

Author SHA1 Message Date
centdix 3dd293e450 Revert "feat(ai): handle aws bedrock as provider (#7131)" (#7141)
This reverts commit 30eb9aae25.
2025-11-14 12:31:16 +00:00
centdix 30eb9aae25 feat(ai): handle aws bedrock as provider (#7131)
* backend draft

* fix for tool and streaming

* do frontend side

* working

* working tools

* rm

* handle list endpoint

* handle for ai agents

* fix for models requiring inference id

* cleaning

* fix desc issue

* fix tool usage

* fix structured output

* cleaning

* fix for api

* rm

* fix input images
2025-11-14 12:22:51 +00:00
Ruben Fiszel 839e092aba nit check 2025-11-13 12:34:53 +00:00
Ruben Fiszel 2ee17aea8b fix: handle string in input transform strings 2025-11-13 12:26:50 +00:00
Diego Imbert 0639623af6 Trigger kind filter (#7123)
* Pass JobTriggerKind to push() to store it in v2_job

* fix compiltion

* frontend job trigger kind filter

* SQL trigger_kind filtering

* fix animated button

* ee repo ref

* Fix "Show schedule" + trigger kind = schedule

* duplicate email

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

---------

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
2025-11-13 10:50:04 +00:00
Diego Imbert c079155e3e SQL result_collection flag (#7041)
* Return array of results duckdb

* Migration script to add result_collection=legacy

* migration script fixes

* app_version_lite not necessary

* New annotations macro that supports custom types

* pass unit tests

* nit style

* result_collection almost works for postgres

* fix last_statement

* frontend suggesitons

* fix column_order making columns disappear

* added version check for duckdb FFI lib to avoid crashes when changing FFI interface on Windows agent workers

* result_collection for duckdb

* Correct legacy behavior in DuckDB

* mysql result_collection

* mssql collection_strategy

* result_collection for oracle

* snowflake result_collection

* fix errors

* mistake, .clone() causing deadlock

* fix frontend errors on oracle / mssql

* fix legacy behavior on mssql

* add exception catch in migration

* missing app_script update

* change cache layout to avoid worker pulling old scripts
2025-11-12 15:32:07 +00:00
Stephan Fitzpatrick ea5b3e6b92 fix(backend): add sslmode parameter to DuckLake PostgreSQL connections 2025-11-11 08:56:53 +00:00
hugocasa 51cba95d39 fix(backend): improve pwsh param block parsing (#7096)
* fix(backend): replace regex with one-pass algorithm for PowerShell param parsing

The previous regex-based approach used  which would
fail when encountering nested parentheses inside the param block, such as
 or .

This commit replaces the regex with a proper one-pass parser that:
- Tracks parenthesis depth to correctly handle nesting
- Respects string quotes (both single and double quotes)
- Handles PowerShell's backtick escape character
- Works correctly with complex default values and function calls

Changes:
- Removed RE_POWERSHELL_PARAM regex constant
- Added extract_powershell_param_block() function for extracting param contents
- Added extract_powershell_param_block_full() function for extracting full param block
- Updated parse_powershell_file() in windmill-parser-bash
- Updated pwsh_executor.rs to use the new extractor
- Added comprehensive test cases for nested parens and quoted strings

Fixes #7079

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* refactor(backend): combine PowerShell param extraction functions

Merged extract_powershell_param_block() and extract_powershell_param_block_full()
into a single function with a boolean parameter to control output format:
- include_keyword=false returns just contents between parentheses
- include_keyword=true returns full param(...) block

This eliminates code duplication while maintaining all existing functionality.

Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>

* fix(backend): improve pwsh param block parsing

* chore: publish parser

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: hugocasa <hugocasa@users.noreply.github.com>
2025-11-10 15:08:20 +00:00
hugocasa ce8d5660fe continue streaming on malformed sse payload (#7102)
* fix(backend): continue streaming on malformed sse payload

* nit
2025-11-10 15:08:01 +00:00
hugocasa 2e245accc5 fix(backend): improve sse parsing (#7100) 2025-11-10 14:32:16 +00:00
Pyra 84992cd8ff fix: proper error handling in pulled job preprocessor (#7098)
* fix: proper error handling in pulled job preprocessor

Signed-off-by: pyranota <pyra@duck.com>

* follow up for merge

Signed-off-by: pyranota <pyra@duck.com>

* make it safe

Signed-off-by: pyranota <pyra@duck.com>

* clippy

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* use String instead of Value

Signed-off-by: pyranota <pyra@duck.com>

* update ee ref

Signed-off-by: pyranota <pyra@duck.com>

* implement Error for PulledJobResultToErr

Signed-off-by: pyranota <pyra@duck.com>

* updatesqlx

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-11-10 13:43:50 +00:00
dieriba fc5034e94d fix: preproccessor ui and expanding preprocessor support (#6872)
* preproccessor php

* fix

* ok

* remove folder

* chore: publish parser

---------

Co-authored-by: HugoCasa <hugo@casademont.ch>
2025-11-10 11:22:31 +00:00
Ruben Fiszel 3dcad57481 add debug_sse_stream 2025-11-09 10:53:37 +00:00
Ruben Fiszel f12be4eb19 fix direct access 2025-11-08 10:26:48 +00:00
dieriba 3a657b10e7 nit flow env (#7090) 2025-11-07 18:51:22 +00:00
dieriba c59183f5c3 feat: env var in flow (#6852) 2025-11-07 18:38:55 +00:00
centdix 49524d5b28 chore(aiagent): store memory in db (#7053)
* store to db

* no warning in context

* warning in frontend

* remove log

* 100kb + alert

* update sqlx

* update eeref

* Update ee-repo-ref.txt

* cleaning

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-11-07 15:27:00 +00:00
Ruben Fiszel ad43680bb9 fix: dep jobs improvements (#7081)
* dep jobs improvements

* update
2025-11-07 11:04:53 +00:00
Ruben Fiszel ad861f5312 formalize ai input transform (#7078)
* ai input transform

* fix
2025-11-07 08:03:44 +00:00
Ruben Fiszel 9e3ea76575 fix(bun): support relative imports without .ts but starting with /u/ or /f/ 2025-11-06 18:42:57 +00:00
centdix 5ed41c6d13 fix(aiagent): force structured tool usage for claude (#7052) 2025-11-04 17:06:33 +00:00
Alexander Petric 4220582daf fix: pass whitelist env vars to bun install (#7047) 2025-11-04 15:30:10 +00:00
Pyra 6052714833 fix(ruby): propagate error correctly (#7046)
Signed-off-by: pyranota <pyra@duck.com>
2025-11-04 16:01:49 +01:00
Ramtin Mesgari 5da494b197 refactor: remove legacy database views v2_as_queue and v2_as_completed_job (#6689)
* refactor: remove legacy database views v2_as_queue and v2_as_completed_job

Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com>

* fix tests

* fix jobs.rs

* end

* fix

* improvement

* improvement

---------

Signed-off-by: Ramtin Mesgari <26694963+iamramtin@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
2025-11-04 13:08:02 +00:00
Ruben Fiszel 2d1c1d81ba internal(backend): mini completed job + improvements (#7042)
* mini completed job

* mini completed job

* sqlx

* restore dedi

* restore dedi

* nits

* fix
2025-11-04 09:31:11 +00:00
Pyra f661caf2b1 fix: fix rebuild_dependency_map (#7026)
* nit: fix rebuild_dependency_map

Signed-off-by: pyranota <pyra@duck.com>

* ignore archived flows

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-11-01 15:07:38 +01:00
Ruben Fiszel a2223802cf nit 2025-11-01 09:43:21 +00:00
Ruben Fiszel 8e9ddb7b84 flow generate locks improvements 2025-11-01 08:53:52 +00:00
Pyra abfc1cba1c fix: redeployment of relative import overwrites lock from raw reqs (#7023)
Signed-off-by: pyranota <pyra@duck.com>
2025-10-31 23:52:31 +01:00
hugocasa cf86881528 fix(backend): add locks to inline preprocessor/failure/tool modules in flows (#6825)
* fix(backend): add locks to inline preprocessor/failure/tool modules in flows

* fix merge

* fix tests

* fix tests

* add test for ai agent tool

* nit

* fix build

* fix locking
2025-10-30 10:33:36 +00:00
Pyra 83c0c82124 fix: auto-unarchived scripts (#6998)
* fix: auto-unarchived scripts

Signed-off-by: pyranota <pyra@duck.com>

* also do for archive by hash

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
2025-10-29 21:09:36 +00:00
Alexander Petric 50a476b529 feat: Add AI_HTTP_HEADERS environment variable for custom AI request headers (#6994)
This commit adds support for setting custom HTTP headers for all AI API requests
via the AI_HTTP_HEADERS environment variable.

Usage:
  AI_HTTP_HEADERS="customheader1: hello, customheader2: world"

The environment variable accepts a comma-separated list of header:value pairs.
These headers will be applied to all AI requests made through both the worker
(AI agent jobs) and the API (AI proxy requests).

Changes:
- backend/windmill-worker/src/ai_executor.rs: Parse and apply custom headers
- backend/windmill-api/src/ai.rs: Parse and apply custom headers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-29 21:03:36 +00:00
Ruben Fiszel b5b0b36da0 add OTEL_JOB_LOGS 2025-10-29 18:20:15 +00:00
centdix 32180d636d fix(aiagent): use tool-based structured output for all claude models (#6979)
* fix(aiagent): use tool-based structured output for all claude models

Extended is_anthropic_provider to check if model starts with 'claude'
regardless of provider. This ensures the tool-based structured output
logic is used for all Claude models, not just when using Anthropic
or OpenRouter providers.

Closes #6977

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>

* fix

---------

Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-28 17:14:18 +00:00
Alexander Petric 038986d0b5 fix: fix windows bun bundler main.ts path detection (#6962) 2025-10-27 21:20:48 +00:00
Ruben Fiszel 5b1ab610b7 improve stack size usage (#6936)
* oom

* final

* final
2025-10-24 22:08:54 +00:00
Ruben Fiszel 4cee047ebc fix agent worker nits 2025-10-24 19:32:47 +00:00
hugocasa 7b46491578 fix(cloud): better errors when failing to get team plan status (#6908)
* fix(cloud): better errors when failing to get team plan status

* better errors

* fix build

* fix build
2025-10-23 20:19:10 +00:00
Pyra edece035f8 feat: job debouncing (#6878)
* feat(EE): job debouncing

Signed-off-by: pyranota <pyra@duck.com>

* remove 'script' file

Signed-off-by: pyranota <pyra@duck.com>

* more work

Signed-off-by: pyranota <pyra@duck.com>

* properly gate it behind enterprise

Signed-off-by: pyranota <pyra@duck.com>

* update ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* change ee repo ref again

Signed-off-by: pyranota <pyra@duck.com>

* remove unused variable

Signed-off-by: pyranota <pyra@duck.com>

* feat(EE): implement TODOs and enhance tracing for job debouncing

- Add database index on script(workspace_id, debounce_key) for efficient lookups
- Update minimum version requirement to 1.564.0 throughout codebase
- Add tracing warnings when debouncing is disabled due to worker version mismatch
- Fix all documentation links from TODO placeholders to proper URLs
- Replace Gauge icon with Timer icon for debouncing UI elements
- Update placeholder text and tooltips with clear descriptions

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* create -> crate

Signed-off-by: pyranota <pyra@duck.com>

* remove index

Signed-off-by: pyranota <pyra@duck.com>

* some updates

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* fix it once more

Signed-off-by: pyranota <pyra@duck.com>

* Remove flow step debouncing, keep top-level flow debouncing

- Remove debounce fields from RawScript and FlowScript FlowModuleValue variants
- Remove debounce fields from JobPayload::FlowScript and RawCode
- Update raw_script_to_payload function signature
- Remove debouncing UI from flow step runtime settings
- Remove debouncing toggle handler and indicator badge
- Preserve top-level flow debouncing in FlowSettings

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* cleanup

Signed-off-by: pyranota <pyra@duck.com>

* fixup claude's work

Signed-off-by: pyranota <pyra@duck.com>

* cleanup: remove dbg! statements, update min version to 1.566.0, add comprehensive comments

- Removed all dbg! macro calls from production code
- Updated MIN_VERSION_SUPPORTS_DEBOUNCING from 1.564.0 to 1.566.0
- Added comprehensive documentation comments explaining:
  - Debouncing feature purpose and mechanics
  - Database schema for debounce_key and debounce_stale_data tables
  - Version check logic and guard functions
- Improved code clarity and maintainability

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* improve fallback

Signed-off-by: pyranota <pyra@duck.com>

* remove comments from old migration

Signed-off-by: pyranota <pyra@duck.com>

* fix pull

Signed-off-by: pyranota <pyra@duck.com>

* fix once more

Signed-off-by: pyranota <pyra@duck.com>

* Update frontend/src/lib/components/ScriptBuilder.svelte

Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>

* add sqlx cache

Signed-off-by: pyranota <pyra@duck.com>

* fix: frontend CI - fix debounce field names and remove leftover flow step debouncing

- Fix ScriptBuilder.svelte: change custom_debounce_key to debounce_key
- Add debounce_key and debounce_delay_s fields to NewScript schema in openapi.yaml
- Regenerate frontend types from OpenAPI spec
- Remove leftover flow step debouncing code from FlowModuleComponent.svelte
- Remove debounce fields from RawScript in openflow.openapi.yaml
- Remove unused Timer import from FlowModuleHeader.svelte

All frontend checks now passing (0 errors, 0 warnings)

Co-authored-by: Pyra <pyranota@users.noreply.github.com>

* fix ci

Signed-off-by: pyranota <pyra@duck.com>

* remove unused import

Signed-off-by: pyranota <pyra@duck.com>

* fix ci again

Signed-off-by: pyranota <pyra@duck.com>

* udpate ee repo ref

Signed-off-by: pyranota <pyra@duck.com>

* CI doesn't want to be fixed but I still try

Signed-off-by: pyranota <pyra@duck.com>

* nits

Signed-off-by: pyranota <pyra@duck.com>

* ci...

Signed-off-by: pyranota <pyra@duck.com>

* Update ee-repo-ref.txt

* safer migration

Signed-off-by: pyranota <pyra@duck.com>

* reduce noise in logs

Signed-off-by: pyranota <pyra@duck.com>

* fix cli for scripts

Signed-off-by: pyranota <pyra@duck.com>

* nit

Signed-off-by: pyranota <pyra@duck.com>

---------

Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: Pyra <pyranota@users.noreply.github.com>
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-23 20:18:08 +00:00
centdix 53d8fbe508 feat(flow chat): display image outputs (#6880)
* support displaying images output in flow chat

* opti

* simplify

* better code

* fix
2025-10-23 17:19:57 +00:00
hugocasa e81d629705 feat(backend): stream early return (#6896)
* feat(backend): stream early return

* revert early return change

* nit

* sqlx

* fix build
2025-10-23 08:36:21 +00:00
Ruben Fiszel dc887e5f43 remove erronous error log in agent mode 2025-10-23 07:41:31 +00:00
centdix 2170d8dd32 feat(ai agent): handle inputTransforms for tools arguments (#6873)
* show inputs

* filter filled args

* merge args with input transforms

* working expr

* handle results expr

* cleaning

* cleaning

* cleaning

* cleaning

* Update SQLx metadata

* fix no previous step

* add ai option in frontend

* cleaning

* cleaning

* Update SQLx metadata

* fix

* fix reactive unmount issue with tool id

* use existing func

* only fetch if needed

* fix

---------

Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
2025-10-22 18:04:23 +00:00
Alexander Petric d777c7798b feat: allow optionally forcing ipv4 for reqwest (#6883) 2025-10-21 15:46:41 +00:00
centdix 97ac1be036 feat(aiagent): allow mcp as tools (#6790)
* draft mcp client

* testing

* fix

* cleaning

* mcp resource in inputtransforms

* cleaning

* big cleaning

* cleaning

* no arc

* add utils file

* refactor tools

* add mcp actions

* draft frontend

* send arguments from backend

* better frontend

* cleaning

* use token for auth

* add logo

* rm

* fix

* fix

* chore: refactor mcp for ai agents (#6829)

* Add Tool enum for AIAgent with backward compatibility

- Created Tool enum that can be either Windmill (FlowModule) or Mcp (resource reference)
- Created McpToolRef struct to hold MCP resource path
- Implemented custom Deserialize for Tool with backward compatibility:
  - New format: {type: 'windmill'|'mcp', ...}
  - Old format: FlowModule objects (automatically wrapped in Tool::Windmill)
- Updated AIAgent to use Vec<Tool> instead of Vec<FlowModule>
- Updated FlowValue::traverse_leafs to handle Tool enum
- Backward compatible: old flows with Vec<FlowModule> will deserialize correctly

* Refactor AI executor to process Tool enum instead of extracting MCP from input_transforms

- Separate Windmill tools and MCP resource paths from tools list
- Process Windmill FlowModules into Tool definitions
- Load MCP tools from resource paths in Tool::Mcp variants
- Remove old logic that extracted mcp_resources from input_transforms
- Import FlowModule, remove unused InputTransform
- Fix type issues: use .as_str() for path and handle Option<bool> properly

* handle in args

* mcp as flowmodule

* frontend

* config for mcp

* simplify logic

* fix ai executor logic

* cleaning

* clean frontend

* fix

* better resource picker

* fix and styling

* add endpoint to fetch tools

* apply tool filtering

* fix name validation

* better ui

* use cache

* fix

* fix merge

* refactor: Separate MCP tools from FlowModule in AIAgent

- Add new AgentTool, ToolValue, and McpToolValue types
- Update AIAgent to use Vec<AgentTool> instead of Vec<FlowModule>
- Implement From traits for clean conversion between AgentTool and FlowModule
- Add backward compatibility via custom deserializer for AgentTool
- Simplify resolve_module logic by reusing existing resolve_modules function
- Update traverse_leafs to handle AgentTool structure

This refactoring separates MCP tools from FlowModule tools, making the
type system clearer and eliminating the need to treat MCP servers as
a special case of FlowModule.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: Update ai_executor and worker_lockfiles for AgentTool

- Update ai_executor.rs to handle new AgentTool structure
  - Separate MCP tools from FlowModule tools using ToolValue enum
  - Convert AgentTool to FlowModule for backward compatibility
  - Add imports for AgentTool and ToolValue types

- Update worker_lockfiles.rs for lazy loading optimization
  - Convert AgentTool <-> FlowModule in insert_flow_modules
  - Preserve lazy loading for FlowModule tools via modules_node
  - Keep MCP tools inline (lightweight, no need for lazy loading)
  - Maintain backward compatibility with existing flows

This enables the lazy loading optimization for FlowModule tools while
keeping MCP tools inline, balancing performance and simplicity.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* cleaning

* adapt frontend

* cleaning

* cleaning

* type fix

* cleaning

* fix back comp

* move mcp button position

* nit

* cleaning

* fix nested removal

* cleaning

* opti

* fix chat markdown display

* fix chat messages layout

* fix back comp frontend

* fix deserializer

* nit

* simpler serializer

* use if else

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:28:52 +00:00
Ruben Fiszel 6ccccbcf9e fix(cli): improve generate-flow with raw requirements 2025-10-20 10:03:14 +00:00
Ruben Fiszel e5a11a7bc6 split main async with a box pin 2025-10-16 20:31:46 +00:00
Ruben Fiszel 3dd75ad18d fix rare stack overflow bc of async size 2025-10-16 20:05:09 +00:00
Ruben Fiszel 6543a83d9f fix rare stack overflow bc of async size 2025-10-16 20:01:30 +00:00
Pyra 952a15a877 remove dbg! leftovers (#6842)
Signed-off-by: pyranota <pyra@duck.com>
2025-10-16 17:29:57 +00:00