mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-21 16:02:28 +00:00
f2abf0e94de94035c51bd441fc39ab18a2ede6fb
242 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
cd06bd25e1 | nit(python): add mapping for azure-identity #7922 | ||
|
|
8e19f9652d |
Added serpapi import handling (#7871)
Added the handling for serpapi import. The name of the Python library to be installed is google-search-results. Source: https://pypi.org/project/google-search-results/ |
||
|
|
ff70a4e9d1 |
fix: parse Python datetime.datetime and datetime.date type annotations (#7856)
* fix: parse Python datetime.datetime and datetime.date type annotations correctly The Python parser only matched ExprKind::Name for type annotations, so `datetime.datetime` (an Attribute expression) silently fell through to Typ::Unknown and no datetime picker was shown in the UI. - Extend parse_expr to resolve `datetime.*` attribute access (alongside the existing `wmill.*` handling) - Add Typ::Date variant for `datetime.date` → JSON schema format "date" - Update python worker to import and convert `date.fromisoformat()` - Update argSigToJsonSchemaType, AI types, schema validation, and SQL datatype wasm for the new Date variant Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * all * all * all * all --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
3476ef4b9c |
fix: fix DB Manager not working with db resources with 4+ path segments (#7809)
* support more than 3 path segments * Fix explore db resource not working with 4+ path segments * don't assume 3 segments * ?table= syntax impl * update parsers * more nit fixes * fix sql query * claude nit * Update SQLx metadata --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
|
c9ee675de8 |
Fix asset parser integration test (#7794)
* fix ci * Fix TS asset parser CI |
||
|
|
c760a33ccd | fix ci (#7790) | ||
|
|
0caa533fbd |
feat: column-level asset tracking for ducklake and datatables (#7774)
* Refactor 1
* claude tmp1
* fixes1
* support for insert and update
* Fix returning
* docs: add nuanced mutex selection guidance to Rust backend skill (#7737)
Add "Mutex Selection in Async Code" section explaining when to use
std::sync::Mutex vs tokio::sync::Mutex based on official Tokio docs.
std::sync::Mutex is preferred for data protection as it's faster;
tokio::sync::Mutex only needed when holding locks across .await points.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix(cli): revert findCodebase change that broke ../shared codebases (#7740)
* fix(cli): revert findCodebase relative_path check that broke ../shared codebases
The previous change added a check to ensure script paths start with the
codebase's relative_path. However, this broke cases where relative_path
uses parent directory references (e.g., "../shared") because:
1. path.join normalizes paths, so "/project/../shared/f/script.ts" becomes
"/shared/f/script.ts"
2. FSFSElement strips the cwd prefix, resulting in "f/script.ts"
3. The check "f/script.ts".startsWith("../shared/") failed
The original behavior was correct - relative_path indicates where to find
codebase files, while includes/excludes patterns match against the normalized
paths that get passed during sync.
Fixes regression reported in #7729 comments.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* test(cli): add preview test for codebase with imports
Tests that codebase bundling correctly includes imported modules,
which is the key functionality needed for ../shared codebases.
The test creates a helper module and a main script that imports
from it, then verifies the bundled script executes correctly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore(main): release 1.621.2 (#7735)
* chore(main): release 1.621.2
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
* feat: workspace dedicated workers (#7741)
* feat: workspace dedicated workers
* ref
* chore: update ee-repo-ref to a18ac31062ac092cb9a5fc87629e217d97f4911d
This commit updates the EE repository reference after PR #398 was merged in windmill-ee-private.
Previous ee-repo-ref: 98cfe3fef764d9d815d326d5056c734a03689d33
New ee-repo-ref: a18ac31062ac092cb9a5fc87629e217d97f4911d
Automated by sync-ee-ref workflow.
* fix(frontend): workspace script in flow steps
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix: remove uuid-ossp extension requirement for RDS compatibility
The uuid-ossp extension was created in the first migration but never
actually used - the codebase uses gen_random_uuid() which is built-in
to PostgreSQL 13+. This allows Windmill to run on AWS RDS where
application users may not have CREATE SCHEMA privileges.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: indexer build error (#7744)
* fix: indexer build error
* prepare sqlx
* Remove changes from Cargo.toml
* refactor: remove seed parameter from AI chat completions (#7745)
* better claude
* refactor: remove seed parameter from AI chat completions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* fix: visibility bug on deployment UI (issue when renaming items) + add tracking of folders and resource types (#7739)
* fix: Raw apps deployment UI (and merge UI)
* Add folders and resource tpyes to merge UI
* claude first pass on adding the new arg for h_deploy_metadata
* Add missing argument to handle_deployment_metadata in all its calls
* Add support for folders and resource types in merge UI
* Update eereporef for CI
* Update ee repo
* Add migration to reset cached diff with potential artifacts
* fix type in frontend
* Preapare sqlx
* Remove unused import and logs
* update ee-repo
* Update eerepo
* chore: update ee-repo-ref to aca38475afd2cafaf63f4bbffc65be9437d57d86
This commit updates the EE repository reference after PR #397 was merged in windmill-ee-private.
Previous ee-repo-ref: 19c64cf8c61d83f45047b37660054b29658cd403
New ee-repo-ref: aca38475afd2cafaf63f4bbffc65be9437d57d86
Automated by sync-ee-ref workflow.
* Make integration test for workspace comparisons
* Update SQLx metadata
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
* fix: require AGENT_TOKEN and BASE_INTERNAL_URL for agent mode
- Add AgentConfig struct to validate required env vars on startup
- Change build_agent_http_client to require explicit token and URL
- Remove DEFAULT_BASE_INTERNAL_URL fallback (no more silent localhost:8000)
- Exit immediately if agent cannot connect to server on initial load
- Update integration tests to use dynamic port for BASE_INTERNAL_URL
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: forward teams error to client (#7746)
* fix: forward teams error to client
* chore: update ee-repo-ref to 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d
This commit updates the EE repository reference after PR #400 was merged in windmill-ee-private.
Previous ee-repo-ref: 25d35a8de1cd70e281dc876e51cd30402580b5c0
New ee-repo-ref: 9a3d71f2c6a41ed4d17111a8c05d8e1d4933898d
Automated by sync-ee-ref workflow.
* fix
* fix
* fix
* al
* sqlx
* sqlx
* all
* all
---------
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
* feat: add token usage tracking to AI agent output (#7738)
* feat: add token usage tracking to AI agent output
Add TokenUsage struct to track input/output/cache tokens from AI providers.
Currently implemented for Bedrock provider, with infrastructure in place
for other providers. Usage is included in the AI agent result alongside
output and messages when available.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add token usage extraction for Anthropic provider
Extract usage from message_delta SSE event and convert to TokenUsage.
Includes input_tokens, output_tokens, cache_read_input_tokens, and
cache_creation_input_tokens (mapped to cache_write_input_tokens).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add token usage extraction for Google AI/Gemini provider
Extract usage from usageMetadata in Gemini SSE events and convert to TokenUsage.
Maps promptTokenCount -> input_tokens, candidatesTokenCount -> output_tokens,
totalTokenCount -> total_tokens.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add token usage extraction for OpenAI Responses API provider
Extract usage from response.completed SSE event and convert to TokenUsage.
Maps input_tokens, output_tokens, and total_tokens directly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: add token usage extraction for Azure OpenAI / Chat Completions API
Add stream_options.include_usage to request and parse usage from final
SSE chunk for providers using the standard OpenAI Chat Completions API
(Azure OpenAI, Mistral, DeepSeek, Groq, TogetherAI, CustomAI).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: cleanup token usage tracking - remove unused Image usage field and accumulate across iterations
- Remove unused `usage` field from ParsedResponse::Image variant
- Add TokenUsage::accumulate() method to sum usage across agent iterations
- Accumulate input/output/total/cache tokens instead of replacing with last iteration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: remove verbose debug logging from AI providers
Remove tracing::info!("[debug] ...") statements that were too verbose
for production. These logged raw events on every streaming event.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: add retry mechanism for OpenAI-compatible providers without stream_options support
Some OpenAI-compatible providers don't support the stream_options parameter
for usage tracking. This adds a retry mechanism that:
- First attempts the request with stream_options.include_usage
- If it fails with 400 and error mentions stream_options/include_usage,
automatically retries without the parameter
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: remove unnecessary text parsing overhead in image response handlers
Revert debugging changes that read response as text before parsing JSON.
Using response.json() directly is more efficient.
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: centralize TokenUsage conversion with constructor methods
Add new(), from_input_output(), and with_cache() constructors to TokenUsage
to eliminate duplicate conversion logic across providers. Also fixes potential
truncation in Bedrock cache token conversion by using i32::try_from with
fallback to i32::MAX.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: simplify Anthropic usage extraction and add Default derive
- Use idiomatic `if let` pattern instead of `is_some()` check for usage extraction
- Add Default derive to OpenAIChatUsage for consistency with other usage structs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: use saturating_add to prevent overflow in token accumulation
In long-running agents with many iterations, token counts could
potentially overflow. Using saturating_add ensures values cap at
i32::MAX instead of wrapping around.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* better claude
* nit
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore(main): release 1.622.0 (#7742)
* chore(main): release 1.622.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
* fix npm check
* fix: add schema compatibility layer for MCP clients like n8n (#7747)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* nit ui nextcloud triggers (#7749)
* feat: add PocketID OAuth provider support (#7318)
* feat(oauth): add Pocket-ID OAuth provider component
- Implements PocketIdSetting.svelte following Keycloak pattern
- Configures OIDC endpoints for Pocket-ID (/authorize, /api/oidc/*)
- Supports standard OIDC scopes (openid, profile, email)
- Uses passkey-only authentication via Pocket-ID
Refs #5678
* feat(oauth): register Pocket-ID in SSO provider list
- Import PocketIdSetting component
- Add Pocket-ID to provider list in SSO tab
- Update exclusion filter to prevent duplicate custom entries
Refs #5678
* fix(oauth): add missing PocketID icon and fix component integration
- Create PocketIdIcon.svelte component with user profile icon
- Register pocket-id in APP_TO_ICON_COMPONENT mapping
- Fix PocketIdSetting to use IconedResourceType pattern matching other OAuth providers
This resolves the issue where PocketID toggle was not appearing in SSO settings.
Refs #5678
* refactor: migrate PocketIdSetting to Svelte 5 runes syntax
- Use $props() with $bindable() for reactive prop binding
- Use $state() for local reactive state
- Use $derived() for computed values
- Use $effect() for reactive side effects
- Replace on:change with onchange event handler
- Pre-populate base URL from existing config when editing
- Clean up bracket notation to dot notation for value properties
Addresses reviewer feedback
* fix: rename pocket-id to pocketid for naming convention compliance
Change identifier from 'pocket-id' to 'pocketid' to match Windmill's naming convention.
No OAuth provider uses hyphens - all custom SSO providers (keycloak, authentik, authelia,
kanidm, zitadel) use no separator.
Changes:
- AuthSettings.svelte: oauths['pocket-id'] → oauths['pocketid'] (2 locations)
- PocketIdSetting.svelte: name={'pocket-id'} → name={'pocketid'}
- icons/index.ts: 'pocket-id': PocketIdIcon → pocketid: PocketIdIcon
Note: PocketID does not need oauth_connect.json entry as it's a custom SSO provider
with user-configured endpoints, similar to Keycloak/Authentik.
Addresses reviewer feedback
* fix: use TextInput component for consistency
---------
Co-authored-by: hugocasa <hugo@casademont.ch>
* fix: preserve script envs field during sync push
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* nit frontend fix
* chore(main): release 1.623.0 (#7748)
* chore(main): release 1.623.0
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
* fix: resolve infinite effect loop in PocketIdSetting component (#7753)
* fix: prevent retention cleanup from deleting jobs of active flows (#7755)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* chore(main): release 1.623.1 (#7754)
* chore(main): release 1.623.1
* Apply automatic changes
---------
Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com>
* feat: default to quickjs on ce for flow eval (#7756)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: runtime assets (#7656)
* Runtime assets
* Nits
* Revert "Nits"
This reverts commit
|
||
|
|
635a24f82c |
feat: runtime assets (#7656)
* Runtime assets
* Nits
* Revert "Nits"
This reverts commit
|
||
|
|
00fc86d099 |
Add grpc entry to mapping in mapping.rs (#7616)
Map `grpc` to [`grpcio`](https://pypi.org/project/grpcio/). |
||
|
|
0f2b417ff5 |
feat: add Pydantic BaseModel and dataclass support for Python type inference (#7497)
* feat: add Pydantic BaseModel and dataclass support for Python type inference - Add AST-based detection of Pydantic BaseModel inheritance patterns - Add AST-based detection of @dataclass decorator (all variants) - Implement recursive field schema extraction with type inference - Add thread-safe stack-based module storage for nested parsing - Add RAII cleanup guard to ensure memory safety on all code paths - Add security limits: 200 fields max, 10 recursion levels max - Add comprehensive test coverage: 3 new tests for Pydantic/dataclass - Maintain 100% backward compatibility with existing type system This enables ML/AI practitioners to use Pydantic models as function parameters with automatic UI generation from model schemas. Implementation highlights: - Zero code execution: Pure AST analysis for safety - Thread-safe: Stack-based storage prevents race conditions - Memory-safe: RAII pattern guarantees cleanup - Security-hardened: Field count and recursion depth limits - Performance-optimized: Depth-limited recursion, lazy parsing Test results: All 12 tests passing (9 existing + 3 new) Closes #4700 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> * fix: improve Pydantic/dataclass parser robustness and error handling This commit addresses critical bugs and improves error handling in the Python parser for Pydantic BaseModel and dataclass support. ## Critical Fixes 1. **Thread-local storage RAII pattern**: Fixed bug where parse failures could leave the module stack in an inconsistent state. Now uses proper functional composition with .ok().map() to ensure cleanup always happens. 2. **Recursion depth warnings**: Added explicit warning messages when the recursion depth limit (10 levels) is reached during type extraction. Made the limit a named constant for clarity. 3. **Unsupported type warnings**: Added informative warning messages for unsupported type annotations (Union types and forward references) to help users understand why their types aren't being inferred. ## Improvements - Added 10 comprehensive test cases covering: - Empty Pydantic models - List[T] and Optional[T] types - Dataclass with decorator arguments - Dict types - Regular classes (non-model types) - Invalid syntax handling - Datetime fields - Multiple model definitions - Nested models - All 21 tests pass successfully ## Testing Verified that: - Parser handles malformed code gracefully - RAII cleanup works correctly with early returns - Warning messages are clear and actionable - No memory leaks or panics Closes #4700 * refactor: Separate Pydantic/dataclass code into dedicated module. Created src/pydantic_parser.rs with thread-local storage, model detection, and type extraction logic. Moved 12 Pydantic tests to tests/pydantic_tests.rs and removed duplicate code from lib.rs. All 21 tests passing. * opti and publish --------- Co-authored-by: Devdatta Talele <devtalele0@gmail.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> |
||
|
|
3affbb3321 |
feat: type-checked data tables v0 (#7381)
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit |
||
|
|
da500fcf3e |
fix: Python Enum types generate proper dropdown schemas with descriptions (#7400)
* Fix Python Enum and Literal schema generation with docstring descriptions - Extract Enum class definitions and their string values - Parse docstring Args: sections for parameter descriptions - Map Enum type annotations to string enums with proper values - Handle Enum.VALUE default values correctly - Store descriptions in Arg.otyp field - Add test case for enum with docstring parsing * perf: optimize enum parser and fix default value handling - Combine enum extraction and docstring parsing into single AST pass (2x performance improvement) - Add support for IntEnum, StrEnum, Flag, IntFlag types - Fix default values to use actual enum values (e.g., 'red') instead of member names (e.g., 'RED') - Improve docstring parsing robustness with proper indentation tracking - Clean up code structure with EnumInfo type for better maintainability All tests pass. This addresses code review feedback for performance and correctness. * perf: implement true lazy evaluation for enum parsing - Only parse metadata when unknown types encountered - Two-pass approach: parse types first, extract only if needed - Zero overhead for scripts without enums - Keyword checks + prepass filtering when extraction needed |
||
|
|
75fdc2cdc9 |
feat: data table schemas (#7353)
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit |
||
|
|
c6c7f3415a |
Specific asset tables (#7323)
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit |
||
|
|
9bbab3321e |
feat: Data tables (#7226)
* data tables settings ui * install runed * zod 4 fixes * use new toJSONSchema * Migrate ducklake catalogs to more generic custom instance databases * fix compilation * Safety conversion for old duckdb ffi * data tables settings * ts client basis * inline run works * datatables work * Revert "datatables work" This reverts commit |
||
|
|
d38c96db36 |
feat: workspace dependencies (#7124)
* commit raw requirements Signed-off-by: pyranota <pyra@duck.com> * raw requirements Signed-off-by: pyranota <pyra@duck.com> * implement `parse_annotation` Signed-off-by: pyranota <pyra@duck.com> * more progress on wdeps Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * more progress Signed-off-by: pyranota <pyra@duck.com> * fixes Signed-off-by: pyranota <pyra@duck.com> * cli improvements + raw deps Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix python versions Signed-off-by: pyranota <pyra@duck.com> * progress Signed-off-by: pyranota <pyra@duck.com> * update :) Signed-off-by: pyranota <pyra@duck.com> * add MaybeLock Signed-off-by: pyranota <pyra@duck.com> * go WIP Signed-off-by: pyranota <pyra@duck.com> * fix python ignoring py version from requirements Signed-off-by: pyranota <pyra@duck.com> * optimize php Signed-off-by: pyranota <pyra@duck.com> * require admin to alter Signed-off-by: pyranota <pyra@duck.com> * fix(cli): flow generateLocks raw deps Signed-off-by: pyranota <pyra@duck.com> * progress in checklist Signed-off-by: pyranota <pyra@duck.com> * fix agent workers Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * nit: remove default features Signed-off-by: pyranota <pyra@duck.com> * oh-wow Signed-off-by: pyranota <pyra@duck.com> * remove dbg! Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * add indexes Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * nits Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * add debug flag Signed-off-by: pyranota <pyra@duck.com> * cli: remove noise Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * remove todos Signed-off-by: pyranota <pyra@duck.com> * trigger deps correctly Signed-off-by: pyranota <pyra@duck.com> * fix frontend Signed-off-by: pyranota <pyra@duck.com> * fix frontend again Signed-off-by: pyranota <pyra@duck.com> * finally fix frontend Signed-off-by: pyranota <pyra@duck.com> * ee repo ref Signed-off-by: pyranota <pyra@duck.com> * fix all Signed-off-by: pyranota <pyra@duck.com> * more fixes... Signed-off-by: pyranota <pyra@duck.com> * remove test Signed-off-by: pyranota <pyra@duck.com> * Update backend-test.yml * comment out legacy test Signed-off-by: pyranota <pyra@duck.com> * fix ci Signed-off-by: pyranota <pyra@duck.com> * fix ci? Signed-off-by: pyranota <pyra@duck.com> * comment out thing Signed-off-by: pyranota <pyra@duck.com> * ignore test Signed-off-by: pyranota <pyra@duck.com> * ci Signed-off-by: pyranota <pyra@duck.com> * base fixture Signed-off-by: pyranota <pyra@duck.com> * fix regression Signed-off-by: pyranota <pyra@duck.com> * fix docs links Signed-off-by: pyranota <pyra@duck.com> * update min version Signed-off-by: pyranota <pyra@duck.com> * simplify * implement cache for get_latest Signed-off-by: pyranota <pyra@duck.com> * move to workspace settings Signed-off-by: pyranota <pyra@duck.com> * sqlx + migration Signed-off-by: pyranota <pyra@duck.com> * more migrations Signed-off-by: pyranota <pyra@duck.com> * use box pin Signed-off-by: pyranota <pyra@duck.com> * nit Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
|
67895abc9f | add option to build wasm packages for nodejs | ||
|
|
b66e038a0f | feat: add support for validateset in pwsh (#7158) | ||
|
|
c16bef8f29 | feat: add support for switch and attributes in pwsh params (#7143) | ||
|
|
51cba95d39 |
fix(backend): improve pwsh param block parsing (#7096)
* fix(backend): replace regex with one-pass algorithm for PowerShell param parsing The previous regex-based approach used which would fail when encountering nested parentheses inside the param block, such as or . This commit replaces the regex with a proper one-pass parser that: - Tracks parenthesis depth to correctly handle nesting - Respects string quotes (both single and double quotes) - Handles PowerShell's backtick escape character - Works correctly with complex default values and function calls Changes: - Removed RE_POWERSHELL_PARAM regex constant - Added extract_powershell_param_block() function for extracting param contents - Added extract_powershell_param_block_full() function for extracting full param block - Updated parse_powershell_file() in windmill-parser-bash - Updated pwsh_executor.rs to use the new extractor - Added comprehensive test cases for nested parens and quoted strings Fixes #7079 Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> * refactor(backend): combine PowerShell param extraction functions Merged extract_powershell_param_block() and extract_powershell_param_block_full() into a single function with a boolean parameter to control output format: - include_keyword=false returns just contents between parentheses - include_keyword=true returns full param(...) block This eliminates code duplication while maintaining all existing functionality. Co-authored-by: hugocasa <hugocasa@users.noreply.github.com> * fix(backend): improve pwsh param block parsing * chore: publish parser --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: hugocasa <hugocasa@users.noreply.github.com> |
||
|
|
fc5034e94d |
fix: preproccessor ui and expanding preprocessor support (#6872)
* preproccessor php * fix * ok * remove folder * chore: publish parser --------- Co-authored-by: HugoCasa <hugo@casademont.ch> |
||
|
|
a9b99e9467 | fix(cli): fix generate-metadata wasm parsers | ||
|
|
5b1ab610b7 |
improve stack size usage (#6936)
* oom * final * final |
||
|
|
4b26def0cd | chore(parsers): publish bash parser with CRLF handling (#6905) | ||
|
|
fa76105d14 |
fix: detect preprocessor in re-exported named exports (#6899)
Fixes #6894 The TypeScript parser now correctly detects preprocessor functions that are re-exported from other modules using named exports like: export { preprocessor } from "./other_module"; Previously, only function declarations were detected. Now the parser also checks ExportNamed AST nodes for any specifier named 'preprocessor'. This allows developers to easily reuse preprocessor functions across multiple scripts without the workaround of wrapping them in a new function. Added comprehensive tests covering: - Simple re-export: export { preprocessor } from "./other" - Re-export with renaming: export { preprocessor as preprocessor } - Mixed exports: export { foo, preprocessor, bar } - Negative case: exports without preprocessor |
||
|
|
5529736784 |
fix(parser): Handle CRLF line endings in bash and PowerShell parsers (#6889)
Fixes #6867 The bash and PowerShell parsers were not properly handling CRLF (Windows-style) line endings when parsing script arguments. The regex patterns were only matching LF line endings, causing scripts with CRLF to fail metadata generation. Changes: - Updated RE_BASH regex to optionally match \r before line end (\r?$) - Updated RE_POWERSHELL_ARGS regex to optionally match \r (\r?) - Added test case test_parse_bash_sig_with_crlf() to verify CRLF handling This fix ensures that bash scripts created on Windows (or with editors using CRLF) will correctly parse arguments and generate proper metadata via 'wmill script generate-metadata' command. Python parser was verified to work correctly as it uses an AST parser that inherently handles line endings properly. |
||
|
|
83337b2035 |
Ansible repo update hubPath + fixes (#6843)
* Allow ssh auth * ssh changes pt2 * Fix unused * Update package.json and hubPaths |
||
|
|
32fae7a10c |
feat: ansible playbook execution git repo mode (repo viewer + UI utils) (#6831)
* Improve minio on flake.nix * Add first asset parsing logic for ansible * Correct html gt sign * Decouple s3 file picker from drawer * Factor duplicate code into snippet * Update S3FilePickerInner to be compatible * Fix pane shrinking issue * Git repo viewer * Change GitRepoViewer * Endpoints for git repo visualizer * Move git repo viewer to its own component * Add button to populate git repo viewer * Update parser yaml for new ansisble features (repo viewer) * Reflect parser changes for ansible * Add button to add the git repo mode of declaration for ansible * Factor function * Playbook + inventories into the drawer * Add button to add inventories from s3 * Move tests to lib.rs * Inventory loading from s3 * Move get github app token logic to be reused by ansible * Update parser and ansible executor * Use the correct path for inventories * Add nushell to flake for wasm builds * Add published parser * Update hubPaths with clone and upload to s3 * Update ee-repo to the branch ref * Fix npm run check * Update cargo.lock * Change labels on buttons * Remove debug log * Update ee-repo-ref * Fix ee issues * Update ee-repo ref * Fix typo * Fix ee * Update ee-repo-ref * Fix missing imports * Unused var * Fix typo * Layout improvents * Fix typos * Remove unused function and log --------- Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
|
e2feba391c |
feat(rust): add resource types (#5843)
* feat(rust): add resource types * write tests * copy from go-resources * implement frontend for rust * update * update cli * update tests * update Signed-off-by: pyranota <pyra@duck.com> * fix package-lock * fix ci * update parser Signed-off-by: pyranota <pyra@duck.com> * Update frontend/src/lib/components/Editor.svelte Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
|
59cdb141c3 |
NULL Toggle in InsertRow drawer (#6729)
* NULL toggle in InsertRow * fix long type parsing in postgres * nits * graphite catch * lazy_static * support for time/timestamp/tz long forms in pg parser * graphite suggestion |
||
|
|
898eb6231b |
feat(backend): array and object params support in pwsh (#6706)
* feat(backend): array and object params support in pwsh * add bool and improve arg conversion * fix bash * update parser * sqlx |
||
|
|
8518b7c897 | check | ||
|
|
673b4d2a4c |
Advanced S3 permissionning (#6617)
* Advanced permission rules UI * stash * first iteration for s3 rule parsing * Move to glob based approach * cache expiry * fix popover positioning * ee * unused imports * forgot windmill_uploads * Check S3 permissions for apps * nit * typo * ee repo ref * forgot get_workspace_s3_resource_and_check_paths in oss |
||
|
|
f3f330dd2a |
feat: windmill dyn multiselect (#6488)
* done * fix * better * typo * use old key * chore: publish pkgs and update deps --------- Co-authored-by: HugoCasa <hugo@casademont.ch> |
||
|
|
8c1be78f61 | fix(parser): detect assets in Python named args (#6518) | ||
|
|
97ed4a539b | nits cleanup + faster script index #6450 | ||
|
|
11dd4118ce |
feat: add ruby support (#5939)
* implement single line pin * make panic-safe * use pin even if multiple modules imported withing single statement * add repins and make imports respect pins * keep all pins * Allow multiple pins * add comments + handle stuff more safely * fix fully qualified imports * remove ignore * sort nested * apply unique to output requirements list * fix typo * remove mut * update sqlx * sort imports * sort imports * fix formatter and format * feat: add ruby support * fix * update sqlx * add ruby to flake.nix Signed-off-by: pyranota <pyra@duck.com> * upgrade par_install_language_dependencies helper rename fields + allow passing custom payload Signed-off-by: pyranota <pyra@duck.com> * change migrations Signed-off-by: pyranota <pyra@duck.com> * make full pipeline work! Signed-off-by: pyranota <pyra@duck.com> * do a bit fixing * stage Signed-off-by: pyranota <pyra@duck.com> * add ruby in dockerfiles Signed-off-by: pyranota <pyra@duck.com> * implement auto-require + 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * create 'windmill/inline' only if does not exist Signed-off-by: pyranota <pyra@duck.com> * add nsjail Signed-off-by: pyranota <pyra@duck.com> * update global settings Signed-off-by: pyranota <pyra@duck.com> * add ruby icon * generate lockfile on deploy (but broken) Signed-off-by: pyranota <pyra@duck.com> * fix(frontend): display deployed script lockfile and other info Signed-off-by: pyranota <pyra@duck.com> * feat: add mini-wmill client Methods: - get_resource - get_variable Additionally add shortcuts on frontend Signed-off-by: pyranota <pyra@duck.com> * add nit warning Signed-off-by: pyranota <pyra@duck.com> * support private repos credentials Signed-off-by: pyranota <pyra@duck.com> * fix compilation error Signed-off-by: pyranota <pyra@duck.com> * found rustc bug + refactor universal installer Signed-off-by: pyranota <pyra@duck.com> * more refactor + fixes Signed-off-by: pyranota <pyra@duck.com> * bug fixing + polishing Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * change 'windmill/inline' Signed-off-by: pyranota <pyra@duck.com> * delegate env_clear to universal installer Signed-off-by: pyranota <pyra@duck.com> * update init script Signed-off-by: pyranota <pyra@duck.com> * add cli support Signed-off-by: pyranota <pyra@duck.com> * unhardcode frontend Signed-off-by: pyranota <pyra@duck.com> * fix cli Signed-off-by: pyranota <pyra@duck.com> * fix init script Signed-off-by: pyranota <pyra@duck.com> * make it work on ms windows Signed-off-by: pyranota <pyra@duck.com> * do not clean env in the end Signed-off-by: pyranota <pyra@duck.com> * fix windows (again) Signed-off-by: pyranota <pyra@duck.com> * include PROXY_ENVS (untested) Signed-off-by: pyranota <pyra@duck.com> * feat: self-signed certs + proxy support (tested) Signed-off-by: pyranota <pyra@duck.com> * flake: pin ruby to 3.4 * merge follow-ups Signed-off-by: pyranota <pyra@duck.com> * remove obsolete libc crate Signed-off-by: pyranota <pyra@duck.com> * cleanup parser Signed-off-by: pyranota <pyra@duck.com> * fix java ci Signed-off-by: pyranota <pyra@duck.com> * cleanup Signed-off-by: pyranota <pyra@duck.com> * fix nits Signed-off-by: pyranota <pyra@duck.com> * Update backend/windmill-worker/src/universal_pkg_installer.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * Update backend/windmill-worker/src/ruby_executor.rs Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> * add nsjail for lock stage Signed-off-by: pyranota <pyra@duck.com> * fix merge Signed-off-by: pyranota <pyra@duck.com> * Update common.rs * regen windmill-api-client Signed-off-by: pyranota <pyra@duck.com> * fix tests Signed-off-by: pyranota <pyra@duck.com> * fix tests again Signed-off-by: pyranota <pyra@duck.com> * reference published ruby parser Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com> |
||
|
|
e6f1211d31 |
feat: Ducklake native support (#6268)
* upgrade duckdb * basic ducklake works * ducklake works with custom db catalogs * fix: pwsh skip already installed modules outside of cache (#6037) * improve query performance of user stats * separate ducklake_catalog db * ducklake settings * DucklakeSettings frontend * Ducklake ws settings saved in backend * fetch ducklake catalog resource * Ducklake works with configured s3 storage * Ducklake as asset * ducklake asset icon * Fix duckdb array and object args not working properly (#6254) * Fix bug with comments in duckdb * Avoid multiple queries when doing ATTACH ducklake * trunc sig no longer needed now that comments are trimmed * cache DuckdbConnectionSettingsResponse * duplicated code * transform_attach_ducklake contributes to duckdb_connection_settings_cache * eliminate the need for used_storages * nit * cleaner management of the bigquery credentials file * DBManagerDrawer refactor to prepare for Ducklake * get ducklake schema * implement delete for ducklake * load column metadata for ducklake * Select query works for ducklake, basic db explorer works ! * duckdb count query * Support all db ops for ducklake * clean migrations * SQL repl for Ducklake * fix broken database studio * nit * assert function * Ducklake in Editor Bar * default ducklake syntax + allow extra args * DucklakeCatalogWizard UI * nit + remove extra $ * modal when databases do not exist * cannot be windmill * Ducklake works safely with instance database * Avoid sending instance db credentials on network * resource leak security * remove fetch_attach_db_conn_str * prevent instance pg password leak * hide asset usage count when not available * case unsensitivity duckdb * warnings * disable instance catalog * use shorthand syntax when inserting with EditorBar * Instance ducklake catalog is now safe to use * use safer argon2 pwd * update package json parsers * update package json * better msgs * tooltips * disable explore button until saved * nit * fix warnings * better ducklake_user password management * nit * Sanitize passwords from errors in ducklake * DisplayResult broken in job result * remove superadmin requirement to check databases_exist * duckdb_connection_settings_v2_inner * Ducklake works on agent worker (finally) * ci * #[allow(dead_code)] * fix openapi missing response * Separate +Database button for DuckDB in EditorBar * Fix dropdown in ducklake settings * Attempt to fix migration race condition in CI * update sqlx failing for some offline queries * avoid temp password for ducklake_user * nits * ducklake settings nits * update duckdb default script * fix sql repl resetting text on refresh * avoid pgcrypto extension --------- Co-authored-by: HugoCasa <hugo@casademont.ch> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
|
9931311650 | fix: resource-type-ts-parser (#6289) | ||
|
|
10befb995d |
feat: local type references parsing support for main function args (#5995)
* add base struct * feat resolve interface and type declarion in entrypoint param's function * nits * fix reset dependencies * update package * fix handle infinite recursion * add depth level and handle enum for referenced type * nits * nits * nits * perf * fix * done * fix schema form cache inconsistency * fix default type and nits * remove * update Object typ for parser * one level ref from from parent when resolving types and use format for resource * update cli and use resource type * nits * update parsers * fix: use specific parser versions --------- Co-authored-by: HugoCasa <hugo@casademont.ch> |
||
|
|
91849baf34 | fix: include export from ts relative import tracking | ||
|
|
2062a634f6 | Fix duckdb array and object args not working properly (#6254) | ||
|
|
3efc329fc0 |
build: combine build-pkgs*.sh into build.nu (#6231)
* build: combine build-pkgs* into build.nu Signed-off-by: pyranota <pyra@duck.com> * nit refactor Signed-off-by: pyranota <pyra@duck.com> --------- Signed-off-by: pyranota <pyra@duck.com> |
||
|
|
2ab5345e61 |
Assets refactor (#6217)
* Moved logic to FlowAssetsProvider * Remove assetsMap in flow * do not parse everything on mount + only check for missing assets fields * add assets field in backend * remove fallbackAccessTypes * better structure and less queries / parsing * Fix assets not showing when pulling raw_flow from jobs * flow assets ctx for job run * Fix transitive assets fetching * Fix input args asset node * enablePathScriptAndFlowAssets flag * edit btn for variable * untrack refresh * move parseInputArgsAssets * Assets tab in runs * Update FlowStatusViewerInner to svelte 5 + fix asset sync bug * avoid toast error on bad resource * fetch res metadata for input arg asset * Job assets viewer in run page * r/w selector * remove indigo badge * store alt_access_type state in ScriptEditor * Don't parse assets in flow script editor * Add alt_access_type in backend * show Read as selected by default to avoid giving the feeling of having made a decision * keep alt_access_type when reparsing in flow raw scripts * Remove variable asset kind, and save assets for scripts * remove all backend asset parsing * R/W/RW selector button nits * fix insert into assets not saving alt access type * support named arguments in python asset parser * improve asset usage drawer R/W indicator * update legacy $res: syntax * reactivity issue * remove last variable asset stuff * sqlx prepare * tooltip explainer * deprecated variable asset nit |
||
|
|
433341b295 |
feat: assets as a primary concept (#6125)
* assets migration * parse assets (duckdb) * iterate on assets * S3 object Preview * remove pagination * filterText * better occurence list * tweak * assets in JobPreview * clone impl * AssetsDetectedBadge * improve DbManagerButton + asset dropdown button * edit resource btn * warning when incorrect resource * +Resource in DuckDB * +S3 Object editor bar * nit fix rename * flow asset badge * More Generic OnChange * Highlight assets used in modules * Show occurence count in flow * Better UX, avoid moving parts * nit * Asset nodes * move to dedicated Asset ctx * fix layoutNodes not handling first assetsMap * explore asset btn in flow asset node * correct offset * single computeAssetNodes function * Fix y positioning of nodes with assets * resource editor * write mode node (ui) * accessType in ctx + fix insert button positioning * right positioning when mixing read and write nodes * right positioning when mixing R and W assets * Better layout fix algorithm * listAssetsByUsage and asset nodes on transitive usages * refactor + remove linkAssets * Refactor to allow for custom R/W modes * AssetsDropdownButton in flow script editor * R/W/RW selection and changes node pos in flow * layoutNodes doesnt need recompute now * fix wrong assumption that nodes recompute when assets change * r/w/rw multi toggle * MultiToggle cool animation + clearable * rename + 1px nit * remove mini toggle button group, use ToggleButtonGroup * Combinator parser that detects R / W asset context * nit fix missing flex-1 * missing order by * better ui indication for access type * special x offset case when only one asset node for clarity * parse getResource in TS with swc ecma parser * support load and write s3 detection in TS * Python asset parser * support wmill api calls without special $res: or s3:// syntax * detect out of context asset uris python * do not use access type override when not ambiguous in flow graph * parse_assets match case in rust * AsRef<str> refactor * From impl * Save flow assets * Save script asset usages + fixes + save fallback access types * asset sub icon * max total asset node width to avoid overlap * small refactor * don't parse comments in duckdb assets * fix assets clearing on parse error * fix script asset save in wrong place * load initial asset fallback access types * support variables * ui fixes * Support S3Object as URI in TS client * support new syntax in python client * Support +S3Object in EditorBar for TS and python * Reduce resource requests in assets page * import windmill client when necessary * update s3Types.d.ts * nit fix * Show input resources and s3 objects as assets * improve asset icons * DarkModeObserver refactor * asset page tabs * Moved resource variables and s3object pages to assets tabs * fetch resource usages * Get variables usages * move assets usage dropdown to component * Revert "move assets usage dropdown to component" This reverts commit |
||
|
|
900c8edd7b | fix: ignore type only imports when computing ts lockfiles | ||
|
|
9c2f6a757f | fix: consistency of root job propagation fixing cases where runFlow in scripts would fail | ||
|
|
f70b6f3052 | update python paser | ||
|
|
f32159f412 |
fix: do not split main.py on 'def main' (#5895)
doesn't work well in edge cases:
```
@schedule(
bla=True
)
def main(x: str):
return x
```
```
error_message = """
Please ensure you have defined main(username, password) correctly.
The function def main( should be at the end.
"""
def main(x: str):
return x
```
Co-authored-by: Ruben Fiszel <ruben@windmill.dev>
|
||
|
|
88ab1a5136 | Use regex to split input to python import parser (#6007) |