* feat: add Pydantic BaseModel and dataclass support for Python type inference
- Add AST-based detection of Pydantic BaseModel inheritance patterns
- Add AST-based detection of @dataclass decorator (all variants)
- Implement recursive field schema extraction with type inference
- Add thread-safe stack-based module storage for nested parsing
- Add RAII cleanup guard to ensure memory safety on all code paths
- Add security limits: 200 fields max, 10 recursion levels max
- Add comprehensive test coverage: 3 new tests for Pydantic/dataclass
- Maintain 100% backward compatibility with existing type system
This enables ML/AI practitioners to use Pydantic models as function
parameters with automatic UI generation from model schemas.
Implementation highlights:
- Zero code execution: Pure AST analysis for safety
- Thread-safe: Stack-based storage prevents race conditions
- Memory-safe: RAII pattern guarantees cleanup
- Security-hardened: Field count and recursion depth limits
- Performance-optimized: Depth-limited recursion, lazy parsing
Test results: All 12 tests passing (9 existing + 3 new)
Closes#4700🤖 Generated with Claude Code (https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: improve Pydantic/dataclass parser robustness and error handling
This commit addresses critical bugs and improves error handling in the
Python parser for Pydantic BaseModel and dataclass support.
## Critical Fixes
1. **Thread-local storage RAII pattern**: Fixed bug where parse failures
could leave the module stack in an inconsistent state. Now uses proper
functional composition with .ok().map() to ensure cleanup always happens.
2. **Recursion depth warnings**: Added explicit warning messages when the
recursion depth limit (10 levels) is reached during type extraction.
Made the limit a named constant for clarity.
3. **Unsupported type warnings**: Added informative warning messages for
unsupported type annotations (Union types and forward references) to
help users understand why their types aren't being inferred.
## Improvements
- Added 10 comprehensive test cases covering:
- Empty Pydantic models
- List[T] and Optional[T] types
- Dataclass with decorator arguments
- Dict types
- Regular classes (non-model types)
- Invalid syntax handling
- Datetime fields
- Multiple model definitions
- Nested models
- All 21 tests pass successfully
## Testing
Verified that:
- Parser handles malformed code gracefully
- RAII cleanup works correctly with early returns
- Warning messages are clear and actionable
- No memory leaks or panics
Closes#4700
* refactor: Separate Pydantic/dataclass code into dedicated module. Created src/pydantic_parser.rs with thread-local storage, model detection, and type extraction logic. Moved 12 Pydantic tests to tests/pydantic_tests.rs and removed duplicate code from lib.rs. All 21 tests passing.
* opti and publish
---------
Co-authored-by: Devdatta Talele <devtalele0@gmail.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: implement quiet mode to filter verbose logs
Add QUIET_MODE environment variable (set to "true" or "1" to enable) that:
- Filters logs with the "windmill_verbose" target via tracing_init
- Increases frequency of periodic logs by 10x when enabled
- Marks per-job worker logs as verbose so they are suppressed in quiet mode
Key changes:
- Add QUIET_MODE lazy_static and VERBOSE_TARGET constant to tracing_init.rs
- Create create_targets_filter() helper that filters windmill_verbose target
- Mark periodic "still running" and "memory snapshot" logs with VERBOSE_TARGET
- Mark per-job "fetched job", "started handling", "job finished" logs as verbose
- Increase memory snapshot interval from every 10 ticks to every 100 ticks in quiet mode
- Increase "still running" log interval from every tick to every 10 ticks in quiet mode
- Add quiet mode notification in job logs
Closes#7477🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* Update tracing_init.rs
* Update worker.rs
* Remove target from tracing info logs
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* use tini with unshare to preserve signals (e.g oom)
* fix ci
* ci as nsjail
* simplify
* fix flaky go integration test
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
When updating rows in the Database Manager with MariaDB tables containing
DATE columns with non-null values, the update query was failing silently
because the date-only format (YYYY-MM-DD) was not recognized by the date
parsing regex. This caused the WHERE clause to use '0000-00-00' instead
of the actual date value. Added a new regex pattern to handle the date-only format that MariaDB
returns for DATE columns, ensuring proper date comparison in UPDATE
queries.
Co-authored-by: Diego Imbert <70353967+diegoimbert@users.noreply.github.com>
* fix: add CORS headers to static assets for iframe context sharing
Add Cross-Origin-Opener-Policy, Cross-Origin-Embedder-Policy, and
Cross-Origin-Resource-Policy headers to static assets served by the backend
to match the vite preview config. This enables the TypeScript worker to work
correctly when ui_builder is loaded in an iframe.
Fixes#7453
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix: add CORS headers only for /apps_raw paths and force reload on navigation
- Only add Cross-Origin-Opener-Policy, Cross-Origin-Embedder-Policy, and
Cross-Origin-Resource-Policy headers for /apps_raw/ paths (not all static assets)
- Add frontend navigation handler to force page reload when navigating from
non-apps_raw paths to /apps_raw/add or /apps_raw/edit
- This ensures the TypeScript worker works correctly while avoiding CORS issues
for external iframe embeds in other parts of the app
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: also reload when navigating from /apps/get_raw/ to apps_raw editor
When viewing a raw app at /apps/get_raw/, the cross-origin isolation
headers are not present. Force a page reload when navigating from
this path to /apps_raw/add or /apps_raw/edit to ensure the headers
are fetched from the server.
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
* Update +layout.svelte
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Ruben Fiszel <rubenfiszel@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* fix frontend
Signed-off-by: pyranota <pyra@duck.com>
* chore: update ee-repo-ref to 5c2a8854e7ff014063a69dd8f7829a935129c31e
This commit updates the EE repository reference after PR #370 was merged in windmill-ee-private.
Previous ee-repo-ref: 8c75fc5b1372a790dc74b374cd72f9caf51a4595
New ee-repo-ref: 5c2a8854e7ff014063a69dd8f7829a935129c31e
Automated by sync-ee-ref workflow.
---------
Signed-off-by: pyranota <pyra@duck.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* refactor(ai): use responses API for OpenAI/Azure, create 'other' provider for completion endpoint
- Created new 'other.rs' provider using the OpenAI-compatible completion endpoint
- Refactored 'openai.rs' to use the Responses API for both text and image output
- Updated query_builder to route OpenAI/AzureOpenAI to OpenAIQueryBuilder
- All other providers (Mistral, DeepSeek, Groq, etc.) now use OtherQueryBuilder
- Updated OpenRouter to delegate to OtherQueryBuilder instead of OpenAIQueryBuilder
This prepares the codebase for adding websearch tool support using the Responses API.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(ai): add websearch as AI agent tool type
- Added WebsearchToolValue to ToolValue enum in flows.rs
- Updated all pattern matches to handle websearch tool type
- Added has_websearch parameter to run_agent function
- Websearch tools don't require additional configuration
This prepares the backend for provider-specific websearch implementations.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(frontend): add websearch as tool option in AI agent UI
- Added WebsearchTool type and utility functions to agentToolUtils.ts
- Added "Web Search" option in tool insertion menu (InsertModuleInner)
- Updated NewAIToolNode to handle pickWebsearchTool event
- Updated AIToolNode to render websearch tools properly
- Updated FlowModuleSchemaMap to create websearch tools
Users can now add websearch tools to AI agents through the UI.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(ai): implement websearch for OpenAI provider
- Added has_websearch parameter to BuildRequestArgs
- OpenAI provider now adds web_search tool to requests when enabled
- Uses OpenAI Responses API web_search tool type
- Websearch tool is added before other custom tools in the request
Implements websearch functionality for OpenAI and Azure OpenAI providers.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(ai): implement websearch for Anthropic provider
- Created anthropic.rs provider with native Anthropic API format
- Added web_search tool to Anthropic requests when enabled
- Anthropic uses /messages endpoint with x-api-key authentication
- Updated query_builder to route Anthropic to new provider
Implements websearch functionality for Anthropic Claude models.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* feat(ai): implement websearch for Gemini/GoogleAI provider
- GoogleAI now uses completion endpoint (other.rs) for text instead of responses API
- Added Google Search grounding when websearch is enabled
- Uses google_search_retrieval tool in request when has_websearch is true
- Updated parse methods to use OtherQueryBuilder for completion endpoint
Implements websearch functionality for Google Gemini models.
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix frontend
* fix anthropic and openai
* better for gemini
* structured output
* cleaning
* fix validate tool
* fixes
* cleaning
* cleaning
* fix for openai
* no responses api for azure
* fixes
* fix
* add tests for ai agent
* avoid panic
* better tests
* test user images
* fix tool choice
* always use streaming backend side
* big cleaning
* show annotations plus agent action for open ai websearch use
* show annotations plus agent action for anthropic websearch use
* show annotations plus agent action for google websearch use
* nit forntend
* rm
* fix
* add test for image ouptut
* fix for azure
* add in openflow
* fix
* fix
* nit tests
* fixes
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: centdix <farhadg110@gmail.com>
* handle messages array for ai agent
* better
* nit
* make tool_calls and tool_call_id nullable
* fix empty json behavior
* nits
* cleaning
* feat(backend): replace messages/messages_context_length with history oneOf field
Replace the separate 'messages' array and 'messages_context_length' fields with
a single 'history' field that uses a oneOf discriminator.
The 'history' field can be either:
- 'auto' mode: automatically manages conversation history with memory, takes a
'context_length' number parameter
- 'manual' mode: bypasses memory and uses explicitly provided messages array
Backward compatibility is maintained: if 'messages_context_length' is provided
in the old schema format, it is automatically converted to 'auto' mode with the
specified context_length.
Co-authored-by: centdix <centdix@users.noreply.github.com>
* feat(frontend): replace messages/messages_context_length with history oneOf field
Replace the separate 'messages' array and 'messages_context_length' fields with
a single 'history' field in the AI agent schema.
The 'history' field uses a oneOf discriminator with two modes:
- 'auto': { mode: 'auto', context_length: number } - automatically manages
conversation history with memory
- 'manual': { mode: 'manual', messages: array } - bypasses memory and uses
explicitly provided messages
The schema includes comprehensive descriptions for each mode explaining the
behavior. The order array has been updated to include 'history' in place of
the old 'messages_context_length' and 'messages' fields.
Co-authored-by: centdix <centdix@users.noreply.github.com>
* fix(frontend): add support for 'mode' discriminator in oneOf rendering
Update ArgInput.svelte to properly handle oneOf schemas that use 'mode' as the
discriminator field, in addition to the existing 'kind' and 'label' support.
Changes:
- Updated tagKey derivation to check for 'mode' first, then 'kind', then 'label'
- Added 'mode' to the onOneOfChange function to track mode changes
- Added 'mode' to the list of keys excluded from enum validation
- Added 'mode' to hiddenArgs to prevent it from being shown in the form
- Added title fields to the history oneOf variants in flowInfers.ts
This allows the AI agent's history field to properly render with toggle buttons
for 'auto' and 'manual' modes.
Co-authored-by: centdix <centdix@users.noreply.github.com>
* fixes
* frontend fix
* nit
* cleaning
* cleaning
* better
* reword
* reword
---------
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: centdix <centdix@users.noreply.github.com>
* Fix MS SQL Numeric rounding
* Fix result collection to avoid JSON parsing
* don't reparse json through sqlx, pass raw string to postgres
* Revert "don't reparse json through sqlx, pass raw string to postgres"
This reverts commit 355691fa7d.
* mistake
---------
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* fix: Migrate to astral-tokio-tar to address CVE-2025-62518
Migrates from the abandoned tokio-tar library to the actively maintained astral-tokio-tar to address CVE-2025-62518 (TARmageddon), a high-severity RCE vulnerability (CVSS 8.1). The vulnerability involves a boundary-parsing bug in PAX/ustar header handling that enables file smuggling attacks via malicious TAR archives.
This is a drop-in replacement requiring only dependency updates in Cargo.toml files. The astral-tokio-tar library uses the same tokio_tar module name, so no source code changes are needed. All references to the vulnerable tokio-tar v0.3.1 have been removed from Cargo.lock and replaced with the patched astral-tokio-tar v0.5.6.
Related to previous PR #6943 which was closed due to CLA issues. Security disclosure available at https://edera.dev/stories/tarmageddon and patch details at https://github.com/astral-sh/tokio-tar/releases/tag/v0.5.6
* update
---------
Co-authored-by: Devdatta Talele <devtalele0@gmail.com>
Co-authored-by: Devdatta Talele <50290838+devdattatalele@users.noreply.github.com>
* data tables settings ui
* install runed
* zod 4 fixes
* use new toJSONSchema
* Migrate ducklake catalogs to more generic custom instance databases
* fix compilation
* Safety conversion for old duckdb ffi
* data tables settings
* ts client basis
* inline run works
* datatables work
* Revert "datatables work"
This reverts commit 6e1588d59e.
* datatables work (without leaking pg credentials)
* println
* separate sqlUtils.ts
* nit
* Separate custom instance db Select and Wizard components
* nit
* nit wording
* add tags to custom instance dbs
* error when trying to use ducklake as datatable or opposite
* show status in dropdown
* data table instance setup works
* sqk function for ducklake
* factorize logic
* fix temp reactivity
* Data table assetexplore
* Migrate S3 permissions to modal
* Revert "Migrate S3 permissions to modal"
This reverts commit 0631d03cb0.
* nit query -> fetch
* Custom instance setup new look
* run_language_executor separate fn
* run_inline param
* nit wording
* Better typed client
* Data tables display as assets in frontend
* asset db icon
* nit
* cleaner errors
* nit
* Fix sed calls in mac
* run_inline_script_preview in python client
* basic python datatable client
* datatable and datalake parser in python
* ducklake client python
* nit fix
* Fix migration producing NULL instead of {} when no custom databases
* merge conflict fail
* python ducklake client arg fix
* parse or infer sql types in ts client
* ts asset parser, detect datatable & ducklake R/W
* fix sql repl for other read ops than select
* export type SqlTemplateFunction
* rename list_custom_instance_pg_databases
* typecheck datatable and ducklake name in Typescript
* Fix typecheck datatable and ducklake in TS
* declare module overriding instead of extending
* infer_sql_type in python client
* SqlQuery object in python
* fix merge conflicts
* update const_format
* CI fix
* factor out to var_identifiers
* sqlx prepare
* unnecessary security (admin is required)
* clearer comment
* ee repo ref
* nit snake case
* claude step 1: detect var declarations
* move detect_sql_access_type to common mod
* claude step 2: detect when saved vars are queried
* Revert "claude step 2: detect when saved vars are queried"
This reverts commit 1e1f930568.
* Revert "claude step 1: detect var declarations"
This reverts commit f866f4819d.
* remove ducklake/datatable and default
* detect data table assigns in var_identifiers
* Python parser successfully infers R/W/RW from ducklake / datatable
* still register ducklake/datatable if not used as unknown R/W
* Go to settings button in Assets Dropdown on not found
* nit
* sqlx prepare fail
* manual fix, somehow sqlx prepare won't do it
* fix frontend ci
* ee repo ref
* ducklake_user doesnt exist in unit tests
* nit fix
* ui nit
* nit
* nit missing clone
* fork ducklakes and datatables
* fix surface hover bug
* stupid mistake
* better deeply reactive mutable derived
* Ducklake picker
* Editor bar data tables
* DuckDB supports datatables
* datatable in duckdb asset parser
* duckdb asset parser var_identifiers
* Revert "duckdb asset parser var_identifiers"
This reverts commit 88068b1a77.
* sqlx prepare
* Box pin in test_workflow_as_code to fix stack overflow
* stash
* sql asset parser parses most s3 literals
* nit
* Detect attach + handle returning RW
* detect assets used with dot notation
* detect implicit access with USE dl; syntax
* Add assets as unknown if var was never used
* Support default ducklake/datatable main in parser
* ignore asset parsing errors in frontend (avoid flow layout shift)
* super weird duplication (merge conflict ?)
* nits
* fix duckdb parser detecting too much as asset when RW ctx is unknown
* fix transparent assets btn
* missing arg
* nit styling
* asset parser specific table parsing
* fix resource specific table parsing
* More concise asset display in flows + better icons
* fix assets page filtering out resources with added table
* Fix frontend to support specific table assets
* Open DB Manager to specific table
* Specific table parser in Python and TS + unit tests
* Fix UPDATE setting access to None
* fix flow edge rendering on top of output picker
* python parser fix var override bug
* add ts test
* fix compilation
* sqlx prepare
* update parsers version
* fix missing schema key onDelete
* Grant permission to create schemas in custom instance databases
* Update pg query to return empty schemas
* Create schema
* Select nits
* support schemas in sql parser
* ts parser handle schema with sql parser result
* detect .schema() syntax
* detect schema syntax in python
* support .schema() in ts and py SDK
* open db manager to specific schema
* support reassignment in ts parser
* nit better unitest
* : syntax in ts
* datatable:schema syntax in python
* fix client py
* nit select dropdown darkmode
* object | null fetchOne
* ts client nits
* parse_sql_client_name fn
* getImportWmillTsStatement refactor in EditorBar
* text to json() in python client
* update parser versions
* pkg lock
* Sql query details in TS asset parser
* code transformation with type parameter in Editor
* Custom Language Worker, code substition works !
* Error marker mapping works
* hover info is correct
* completions work correctly
* other overrides
* type inference kinda works
* Position mapping tests
* refactor prepare_queries
* Refactor PgDatabase to share common code
* Pgdatabase in prepare_queries
* TokioPgConnection refactor
* refactor prepare_queries
* type parameter to sql function
* Fix deadlock
* nit fix
* Fix worker async call freezing because of svelte Proxy
* Force worker to recompute when we set queries
* nit refactor
* nits console logs
* wait that ts worker initialize
* monaco change file version
* update diagnostics
* Refactor for errors
* Show SQL errors in Monaco
* improve sdk
* cleaning refactor + MapResource + usePreparedAssetSqlQueries
* Fixes
* Fix error position mapping
* cache in typescript worker
* fix insert no values
* don't inject type if already present
* Support schema in prepare queries
* update parsers
* ChangeOnDeepInequality
* inferAsset ScriptEditor usage refactor
* sql query typecheck work in flow editor
* Assets and SQL Query check in Raw App Inline Editor
* pkg lock
* Fix DatatableSqlTemplateFunction nit
* prepare query schema nit
* duplicate diagnostics
* nit getScriptVersion mock
* Reprepare queries when switching workspaces
* nit fix
* nit fix
* fetch_one_scalar and execute in python client
* limit pg_connections
* -- prepare flag in postgres
* skip serializing
* fix destructuring undefined
* Prepare queries in workers instead of backend
* nit
* Execute search_path instructions normally
* nit fix
* Fix SET search_path issue in prepare
* only support preparing single-statement queries for now
* update parsers
* safety
* better remove_comments
* Fix getQueryStmtCountHeuristic
* getQueryStmtCountHeuristic tests
* comment out failing tests
* Fix getQueryStmtCountHeuristic impl
* only datatable
centdix
hugocasa
Alexander Petric
Diego Imbert
Devdatta TaleleandDiego Imbert
Aananth K
