* backend draft
* fix for tool and streaming
* do frontend side
* working
* working tools
* rm
* handle list endpoint
* handle for ai agents
* fix for models requiring inference id
* cleaning
* fix desc issue
* fix tool usage
* fix structured output
* cleaning
* fix for api
* rm
* fix input images
* cleaning
* chore: use aws sdk (#7156)
* feat(ai): Add AWS SDK dependencies for Bedrock integration
- Add aws-sdk-bedrockruntime v1.113.0
- Add aws-credential-types for bearer token authentication
- Update rustls to v0.23.35 for compatibility
- Dependencies added to windmill-common for AI features
* feat(ai): Add bearer token provider for Bedrock authentication
- Implement BearerTokenProvider using aws_credential_types
- Simple token-based auth using API keys from Windmill resources
- Add basic unit tests for provider creation
- Export bedrock_auth module in lib.rs
* feat(ai): Add Bedrock client wrapper with region extraction
- Implement BedrockClient wrapper around AWS SDK client
- Bearer token authentication integration
- Extract AWS region from Bedrock base URL automatically
- Comprehensive unit tests for region extraction
- Make aws-config non-optional dependency for AI features
- Update feature flags to reflect new dependency structure
* cargo
* feat(ai): Implement non-streaming Bedrock via AWS SDK
Use official AWS SDK instead of manual HTTP requests for better type safety and maintainability. Implements the Bedrock converse() API for non-streaming requests with proper bearer token authentication and message format conversion between OpenAI and Bedrock formats.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor(ai): Eliminate Simple* conversion types for Bedrock SDK
- Move AI types to windmill-common/src/ai_types.rs for shared access
- Update bedrock_converters to work directly with OpenAI types
- Remove ~200 lines of conversion boilerplate from ai_executor.rs and bedrock.rs
- Remove unused imports to clean compilation warnings
- Benefits: 50% fewer conversion steps, no information loss, easier maintenance
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(ai): Add streaming support for AWS Bedrock SDK
- Implement converse_stream() for Bedrock streaming responses
- Use EventReceiver.recv() to process stream events
- Extract text deltas using bedrock_stream_event_to_text()
- Send TokenDelta events to StreamEventProcessor for real-time updates
- Refactor request building to eliminate duplication between streaming and non-streaming
- Clean, minimal implementation following AWS SDK patterns
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* revert flake change
* fix
* feat(ai): Add tool calls and image support for Bedrock streaming
**Phase 1: Streaming Tool Call Support**
- Add stream event processing functions in bedrock_converters.rs:
- bedrock_stream_event_to_tool_start() - Extract tool use start from ContentBlockStart
- bedrock_stream_event_to_tool_delta() - Extract tool input deltas from ContentBlockDelta
- bedrock_stream_event_is_block_stop() - Detect ContentBlockStop events
- streaming_tool_calls_to_openai() - Convert accumulated tool calls to OpenAI format
- Update ai_executor.rs streaming loop with tool call accumulator (HashMap)
- Track current tool use ID during streaming
- Send ToolCallArguments events to StreamEventProcessor
- Return accumulated tool calls instead of empty vector
**Phase 2: Image Input Support**
- Add parse_image_data_url() to extract format and base64 data from data URLs
- Add content_part_to_block() to convert ContentPart to Bedrock ContentBlock
- Refactor convert_message() to handle multi-part content with images
- Support ImageUrl conversion to Bedrock ImageBlock with proper format (png/jpeg/gif/webp)
- Import AWS SDK image types: ImageBlock, ImageSource, ImageFormat
- Keep content_to_text() helper for system message text extraction
**Benefits**:
- ✅ Tool calling now works in both streaming and non-streaming modes
- ✅ Images are properly converted instead of being silently dropped
- ✅ Structured output works in streaming (uses tool calling)
- ✅ Full feature parity with manual HTTP implementation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* cleaning
* fix(ai): Add S3 image support and structured output for Bedrock
**Fixes:**
1. **S3 Image Support**: Call prepare_messages_for_api() before Bedrock SDK path to convert S3Objects to ImageUrls
- Downloads images from S3 and encodes as base64 data URLs
- Ensures images are properly handled in both streaming and non-streaming modes
2. **Structured Output**: Add ToolChoice::Any when structured output tool is present
- Forces Bedrock to call the structured_output tool
- Ensures JSON schema compliance for structured output
- Works in both streaming and non-streaming modes
**Changes:**
- ai_executor.rs: Call prepare_messages_for_api() for Bedrock SDK path
- ai_executor.rs: Set tool_choice to Any when structured_output_tool_name is present
- aws_bedrock.rs: Remove unused ToolChoice imports (used via full path in worker)
**Testing:**
- ✅ S3 images are now downloaded and converted before API call
- ✅ Structured output now forces tool usage with ToolChoice::Any
- ✅ Both work in streaming and non-streaming modes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* cleaning
* cleaning
* cleaning
* better error
* cleaning
* cleaning
* rm
* rename
* apply region
---------
Co-authored-by: Claude <noreply@anthropic.com>
* fix default
* no panic
* no print
* use utils file
* cleaning
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat: support to run windows binary as service
* ee ref
* ee ref
* flags
* Update backend/src/main.rs
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
* fix ai commit
* fix
* ee ref
---------
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
Ruben Fiszel
centdix
Alexander Petric
hugocasa
wendrul