mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-19 16:02:14 +00:00
ruben/win-2151-wdyt-of-httpsgithubcomwindmill
3349 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ruben Fiszel
|
8dd5e48a68 |
chore(main): release 1.751.0 (#9965)
* chore(main): release 1.751.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
97d14d979f |
bun bootstrap housekeeping on the migrator's held connection (#9970)
migrate() and fix_flow_versioning_migration re-acquired a second connection from the pool while already holding one (the migrator's checked-out, advisory-locked connection). That deadlocks any backend limited to one connection at a time — connection-constrained managed Postgres, PgBouncer transaction pooling, or an embedded single-connection dev database. Route those housekeeping queries onto the already-held connection via a new CustomMigrator::connection() accessor. Fewer connections during migration and, for fix_flow_versioning, the existence check and write now run on the same advisory-locked connection. Default multi-connection behavior is unchanged. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 hugocasa
|
fd8e64d11f |
feat: add cosmetic dev/staging label for dev workspaces (#9959)
* feat: add cosmetic dev/staging label for dev workspaces Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: prefill dev fork name and use a link to switch its label Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: reword the dev/staging label link copy Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style: preview the dev/staging label as a badge in the switch link Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: show the dev/staging badge in the session diff drawer header Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5fe7e1f3e8 |
chore(main): release 1.750.0 (#9952)
* chore(main): release 1.750.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
 windmill-internal-app[bot]
|
aaeb9524b3 |
chore: refresh vendored docs snapshot (#9955)
Co-authored-by: hugocasa <15649739+hugocasa@users.noreply.github.com> |
||
|
Ruben Fiszel
|
891b32195a |
chore(main): release 1.749.0 (#9938)
* chore(main): release 1.749.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
46be39dfb7 |
fix(pipelines): order data_test relationships refs before the tested script in a cascade (#9934)
* fix(pipelines): order data_test relationships refs before the tested script in a cascade Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): key custom-test reads by (usage_kind, path) to avoid same-path flow collisions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
799b9e3b7c |
chore(main): release 1.748.0 (#9914)
* chore(main): release 1.748.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
39eb9de1bc |
feat(pipelines): fork data environments for ducklake materialization (dev data) (#9915)
* feat(pipelines): fork-scoped ducklake namespaces with read-defer to parent Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork graph indicator + fork ducklake namespace cleanup endpoint Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork_views-keyed view transition, fork lineage clone, design doc Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): review hardening - fork DATA_PATH last-wins, registry cache TTL, defer tests Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): per-lake isolated/shared choice at fork creation Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): chain-aware defer discovery + per-location fork namespace registry Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): lake-scoped fork schemas, catalog identity in registry, chain-aware graph chips Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): cleanup deletes fork data from the registered storage identity Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): collapse fork data-path segment to one component (slash-safe ids) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): per-catalog ancestor checks, ancestor extra_args passthrough, test compile fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): invalidate fork ancestor-chain cache on lineage mutations Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): sweep descendant ancestor-chain caches on delete/reparent Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): run fork ducklake cleanup inline in delete_workspace Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): resolve fork cleanup credentials pre-commit, destroy post-commit Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): shared dev-workspace authz gate for namespace drop, invalidatable registration cache, segment-boundary delete filter - extract require_prod_admin_for_dev_workspace, used by both delete_workspace and drop_forked_ducklake_namespaces so the gates cannot drift - key FORK_DUCKLAKE_REGISTERED per workspace and invalidate it in cleanup_fork_ducklake_namespaces so a same-id fork recreated within the TTL re-registers its namespaces - filter listed object locations to the segment boundary before deletion Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): keep orphaned wm-fork-* workspaces ducklake-isolated parent_workspace_id is ON DELETE SET NULL, so a fork can outlive its parent with an empty ancestor chain while its cloned config still points at the shared lake. Key the isolation gate on the wm-fork- prefix as well as the chain (mirroring workspace_is_fork): orphaned forks get the write redirect, registration and cleanup with zero ancestors (no defer), and keep their 'fork' graph chips. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): attach orphaned wm-fork-* ancestors at their fork namespace Chain position alone classified the last ancestor as a root, but an orphaned wm-fork-* ancestor (its own parent deleted, SET NULL) ends the chain the same way while its data lives in its fork namespace — its descendants' defer views bound the dead root's lake instead. Key the root-vs-fork decision on the wm-fork- prefix too, matching the resolution gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): never inherit shared lake opt-out; durable cleanup ledger for failed fork deletions - fork creation strips cloned fork_behavior stamps before applying the request's shared_ducklakes list: sharing is a per-creation choice, a fork of a shared fork defaults back to isolated - fork_ducklake_namespace loses its ON DELETE CASCADE FK: rows are the durable cleanup ledger and outlive the workspace when physical cleanup fails post-commit; fork creation retries leftover rows for the reused id and refuses to create while a metadata schema still cannot be dropped (data-file leftovers alone are inert once the schema is gone and are swept by the next successful same-prefix cleanup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): make orphaned-namespace cleanup retries independent of deleted fork resources - ledger rows gain a schema_dropped phase flag: set when the schema drop succeeded but data cleanup failed, so later retries skip the schema phase and need no catalog credentials at all; registration resets it on re-attach (ON CONFLICT DO UPDATE) since attaching recreates the schema - retry-path $res: resolution falls back to the workspace being forked (the deleted fork's resources were clones of a parent's); live paths (delete_workspace prepare, drop endpoint) pass no fallback Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): fork tables from failed-after-commit runs stay fork-owned in defer and graph A failed materialization must not disguise a physically existing fork table as deferred: CREATE VIEW IF NOT EXISTS silently yields to the table, so reads hit fork data while the graph claims parent defer. - record_mat upsert preserves the last committed snapshot_id on failure - defer discovery and graph chips treat fork rows with a committed snapshot as fork-owned even when status is failed - inspect_fork_catalog also lists live fork tables (same round trip) and the defer list is filtered against them — covers rows recorded before this fix and tables created by raw SQL - drop stale FK-cascade wording in the design doc and sidebar comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): fork-mode ducklake settings — per-lake isolated/shared chips + banner, fork_behavior round-trip The workspace-settings ducklake editor had no fork awareness: no reminder of each lake's isolated/shared choice and no warning about what edits mean in a fork. It also rebuilt each lake explicitly on save, silently dropping fork_behavior — any settings save in a shared fork flipped the lake back to isolated. - fork detection mirrors the backend gate (parent link or wm-fork- prefix) - info banner explaining isolated vs shared semantics in a fork - per-lake chip (emerald 'isolated' / amber 'shared with parent') with tooltips, matching the pipeline graph chip colors - fork_behavior added to DucklakeSettingsType and preserved through convertDucklakeSettingsToBackend Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
33521505db |
feat(ducklake): scheduled lake maintenance (expiry, compaction, orphan cleanup) (#9916)
* feat(ducklake): scheduled lake maintenance (snapshot expiry, compaction, orphan cleanup) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): review fixes — starts_with not LIKE, CE license-lapse escape Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): auth-contract docs + _unchecked rename per codex review Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ducklake): move maintenance payload construction into EE module Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): fall through to script resolution for non-managed reserved-prefix schedules Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ducklake): document accepted pre-existing-schedule limitation on the reserved prefix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ducklake): CE save-off clears the managed schedule row and queued occurrence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 2fab310d4f50ed7c34857d69c9b854f4491bf217 This commit updates the EE repository reference after PR #645 was merged in windmill-ee-private. Previous ee-repo-ref: fff1fd830a36beba732486f05941ec243cf6b640 New ee-repo-ref: 2fab310d4f50ed7c34857d69c9b854f4491bf217 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
42e11c6570 |
feat(pipelines): schema contracts — save-time consumer checks vs captured schemas (#9917)
* feat(pipelines): schema contracts — save-time consumer checks vs captured schemas Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: move schemaContractContext above schemaCanEvolve doc comment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: emit scd2/on_schema_change in CLI local graph, address review notes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: gate editor _current ignore-suppression on scd2, matching backend Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5d7fb6deca |
feat(pipelines): asset freshness — fresh/stale badge (CE) + watchdog (EE) (#9909)
* feat(pipelines): passive asset freshness tracking on the graph Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(pipelines): drop dead freshness-enforcement stub, document query ordering Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(pipelines): freshness watchdog (EE) — auto re-run stale producers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): watchdog review fixes — archived workspaces, badge kind parity, scan index Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — no singlestepflow in freshness, +N parity, completion-time fallback Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(pipelines): CI review — history completedAt, freshness/asset trigger UI metadata Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to 6f5fe0f7f56696fbef5a8349da38496c32e71666 This commit updates the EE repository reference after PR #643 was merged in windmill-ee-private. Previous ee-repo-ref: 1f13380354bf591ae25a2c20d36917534bcc5459 New ee-repo-ref: 6f5fe0f7f56696fbef5a8349da38496c32e71666 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
df6e511763 |
chore(main): release 1.747.0 (#9901)
* chore(main): release 1.747.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
hugocasa
|
d600c7ecfe |
fix(ai): route Azure Foundry Claude models via Anthropic Messages API (#9908)
* fix(ai): route Azure Foundry Claude models via Anthropic Messages API Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai): keep explicit Azure OpenAI deployment base URLs intact build_azure_openai_url only appends /openai/v1 for a bare resource root; any base with an explicit path (e.g. .../openai/deployments/<id>) is preserved. Adds a regression test and a unit test for usesAnthropicMessagesApi. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): enable Claude extended thinking on Azure Foundry Route azure_foundry+Claude through the Anthropic reasoning branch (adaptive thinking + output_config.effort) instead of the gpt/o gate, and recognize claude-sonnet-5. Live-verified: sonnet-5 and opus-4-8 on Foundry accept the low/medium/high/xhigh/max ladder and render summarized thinking. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
3ec1f164be |
fix: strip NUL characters from app values at save time (#9903)
App values are persisted to a json column, which permits the JSON NUL escape (backslash-u-0000), but are later converted to jsonb (e.g. a workspace fork clone_apps, search indexing), which rejects it with "unsupported Unicode escape sequence" -- silently making the app un-forkable. The usual source is a binary file such as .DS_Store accidentally bundled into a raw app file map. A real NUL is unstorable in jsonb either way, and frontend code that needs the character writes it as the source escape (which JSON-encodes to an escaped backslash + literal u0000 and is left untouched), so rather than hard-failing the save we strip genuine NULs and warn. Add strip_null_chars and apply it at both app_version insert sites (create_app_internal and update_app_internal, covering the regular and raw create/update routes). It removes a genuine NUL escape (odd run of backslashes before u0000) while preserving an even run. Returns a borrowed Cow (no allocation) when the value is already clean. Covered by unit tests. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
fad5419b9d |
chore(main): release 1.746.0 (#9872)
* chore(main): release 1.746.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
84141add1d |
feat(pipelines): workspace duckdb macro libraries (// macros / // use) (#9890)
* feat(pipelines): parse duckdb macro-library annotations (// macros, // use) * feat(pipelines): duckdb macro registry tables + deploy-path validation and writes * feat(pipelines): inject workspace duckdb macros into consumer jobs at run time * feat(pipelines): surface macro libraries and lib-consumer edges in asset graph api * feat(frontend): macro-library nodes, lib-consumer edges and scaffold in pipeline graph * docs: mark dbt gap #7 (packages/macros) shipped via workspace macro libraries * fix(pipelines): review fixes - char-safe parsing, local macros win, fork clone, trust-model docs * feat(frontend): duckdb macro autocomplete + workspace macro explorer drawer * fix(pipelines): address CI review - use-setup retention, splice past local defs, orphan filter, full consumer rescan, index-keyed strip * fix(pipelines): inject provider library setup for implicitly-called macros too * fix(pipelines): rls-gate macro listing + honor library-level // use transitively * fix(pipelines): weave injected macros around local definitions by bind order * fix(pipelines): injected library setup always runs before user blocks * perf(pipelines): cache macro registry per workspace with notify-event invalidation * perf(pipelines): disable macro registry cache on cloud |
||
|
Ruben Fiszel
|
af01e90b5c |
feat(s3): replace CE 50MB upload cap with 10GiB workspace storage quota (#9874)
* fix(s3_proxy): enforce CE 50MB upload cap on multipart uploads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): replace CE 50MB upload cap with 10GiB workspace storage quota Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): gate CE quota OSS stubs to not(enterprise) to match callers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): delta-aware CE storage quota + guard usage-load retry loop Account for the overwritten object's size in the quota check so valid same-size overwrites near quota are not rejected (Codex review), and stop the storage-usage $effect from re-firing on persistent API errors (Pi review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): count chunked PUTs; revert overreaching volume quota copy Volumes write to workspace storage via a separate worker-side path with its own 50MB-per-file cap that this PR does not change, so revert the drawer copy that claimed they count toward the 10GiB quota (Codex review). Bump ee-repo-ref for the chunked-PUT accounting fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): add SQLx cache for CE usage-bump/quota queries; exclude volumes Regenerate the missing offline SQLx cache for the not(enterprise) bump and remaining-quota queries so the private CE offline build compiles, and bump ee-repo-ref for the volumes/-prefix exclusion from the counted quota (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): always HEAD for CE upload delta so overwrites don't inflate usage Bump ee-repo-ref for the fast-path overwrite-accounting fix (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): reserve volumes/ prefix on CE write surfaces to close quota bypass Reject direct writes to the reserved volume prefix on the app-upload surface and add the OSS stub; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): refuse new multipart work when CE workspace is at quota Bump ee-repo-ref for the multipart-initiate/part quota gate (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): reserve in-flight multipart bytes against CE storage quota Add workspace_multipart_inflight table + grants, SQLx cache for the reservation queries, and bump ee-repo-ref. Bounds abandoned multipart uploads that the list-based recount can't see (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): clear multipart reservation only after a successful complete Add exclude-upload arg to the OSS quota stub/caller and the SQLx cache for the updated remaining-quota query; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): per-part multipart reservation; commit only on part success Per-part workspace_multipart_inflight schema (upload_id, part_id) so retries replace rather than double-count; SQLx cache for the reworked queries; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(s3): HEAD the multipart overwrite target once per upload, not per part SQLx cache for the stored-credit lookup; bump ee-repo-ref. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to bea5a8b5120d6d69cab1ad4611ebe463559bd200 This commit updates the EE repository reference after PR #640 was merged in windmill-ee-private. Previous ee-repo-ref: 6e6ff86f1939cf74736b7d435bf6851416437523 New ee-repo-ref: bea5a8b5120d6d69cab1ad4611ebe463559bd200 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
7c7d7474cc |
feat: support workspace forks on cloud using parent workspace limits (#9864)
* feat: support workspace forks on cloud using parent workspace limits Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: clarify count_paid_seats approximates rather than mirrors billing seats Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: non-admin fork UI, attach cap, and fork-count for cloud forks Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: cloud fork billing cache on rename, usage display, attach cap edge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: fork count in cloud quotas + fork billing points to parent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate billing/fork caches on fork deletion for id reuse Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: gate fork usage remap on CLOUD_HOSTED, not just the cloud feature Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: note cloud feature vs CLOUD_HOSTED gating in backend guide Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: reserve fork-cap slots for an attach candidate's whole subtree Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate team-plan cache on delete, raise fork depth cap Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: cap fork nesting depth (MAX_FORK_DEPTH, default 5) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fork count/height robust to cycles and deleted intermediates Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): reset fork button loading state on creation error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate billing cache for attached fork subtree; helper auth docs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
d9b080f57f |
feat(ai): add Azure AI Foundry as a native AI provider (#9879)
* feat(ai): add Azure AI Foundry as a native AI provider Adds `azure_foundry` as a new AIProvider variant wired through the AI chat (copilot) and AI agent flow steps. Foundry's chat completions API is OpenAI-compatible and uses Azure conventions (api-key header, Azure URL building), so it reuses the existing OpenAI-compatible query builder and proxy path via the shared `is_azure` helper (renamed from `is_azure_openai`). Backend (windmill-ai): - New `AzureFoundry` enum variant (serde `azure_foundry`) - `get_base_url` requires a resource base URL (like Azure OpenAI / Custom) - `is_azure()` covers Azure OpenAI + Foundry (api-key auth, Azure URL) - Added to OpenAI-compatible proxy support and HttpForward proxy mode - New proxy URL unit test Frontend (copilot): - New provider entry, completion config, model-token handling, streamed usage tracking, and reasoning registry (all model-id-gated, so a no-op for Foundry's non-OpenAI catalog) - Treated as a chat-completions provider, not the OpenAI Responses API OpenAPI: - `azure_foundry` added to AIProvider (openapi.yaml) and AIProviderKind (openflow.openapi.yaml); regenerated CLI guidance Note: the `azure_foundry` resource type (base_url + optional api_key) is hub-managed and must be published to the Windmill Hub separately. Fixes WIN-2122 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai): add azure_foundry to copilot flow Zod provider enum The tracked copilot flow schema (openFlowZod.gen.ts and its openFlow.json source) still carried the old AIProvider enum, so validateFlowModules / validateSpecialFlowModule rejected AI-generated flow edits that create or update an aiagent module with provider kind "azure_foundry" before they could be saved. Add the value to both (preserving the generated single-line format) and a regression test over the flow-module validation path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): lead provider list with OpenAI, Anthropic, Google AI Reorder AI_PROVIDERS so the three primary direct providers come first. The AIProviderPicker renders the first three entries as quick-access buttons, so these become the defaults (previously OpenAI, Azure OpenAI, Azure Foundry); Azure OpenAI / Azure Foundry stay adjacent right after. No logic depends on provider order (only per-provider defaultModels[0] is read). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
53bbb92953 |
feat(pipeline): backfill a range of partitions from the asset drawer (#9885)
* feat(pipeline): backfill a range of partitions from the asset drawer (ee) * feat(pipeline): cancel in-flight backfill job and show cancelling state * refactor(pipeline): move backfill range logic behind private feature * fix(pipeline): close backfill cancel-launch race and record dispatch intent * docs(openapi): producer_path also covers SDK write-edge producers * chore: update ee-repo-ref to c3852ecb36bd0be1a74c63169e513888f3347850 This commit updates the EE repository reference after PR #641 was merged in windmill-ee-private. Previous ee-repo-ref: 7c1450ef89fbc9e844a121b39cafe0d7235d704b New ee-repo-ref: c3852ecb36bd0be1a74c63169e513888f3347850 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
e77b7523a5 | nit enterprise implies license feature | ||
|
Ruben Fiszel
|
9a24cd2bef |
chore(main): release 1.745.0 (#9858)
* chore(main): release 1.745.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
1a9debb689 |
fix(jobs): give flow dynselect a path and its worker tag, like scripts (#9867)
Fetching options for a `dynselect`/`dynmultiselect` input was inconsistent between deployed scripts and deployed flows: - scripts ran through `push_script_job_by_path_into_queue` — a `script` job with the script's path, tag, lock and codebase resolution; - flows ran their schema dyn-select code as an anonymous `preview` with no path and no tag (always the language default), and reported access failures as a raw `SqlErr: no rows`. Deployed scripts are left exactly as they were (that path already handles tag/lock/codebase/on-behalf-of correctly). The flow branch now: - carries the flow path on the preview job, - reads the flow's `tag` under RLS and routes the job to it (falling back to the language default when unset), matching the script's worker group, and - runs `check_tag_available_for_workspace` on that tag — the same gate a normal flow run and the script path apply — so a caller who can read the flow but is not allowed to use its (custom/scoped) worker tag is rejected consistently. The flow's tag read runs on every request, so it also serves as the per-request access check, replacing the raw error with a clean `NotAuthorized` / `NotFound`. Entrypoint-name validation now covers all branches (it is interpolated into the generated wrapper). Inline is unchanged: a `preview` with no path on the language default, blocked for operators. Fixes WIN-2118 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
20cd1a02d5 |
feat(forks): partial-visibility deploy + surface hidden items (#9868)
* fix(forks): let partial-visibility users deploy the visible subset The fork Compare & Deploy page hid the deploy button entirely whenever the comparison reported any item not visible to the user (all_ahead/all_behind flags), telling them to hand the deploy to someone with full access. But the non-visible items are already filtered out of the diff list, and the UI already supports deploying an arbitrary subset via per-item selection — so blocking everything was inconsistent and, for stale/phantom rows, blocked on items that don't even exist. Show the deploy footer regardless; the user acts on the visible/selected items (the per-item disabled conditions are unchanged). The hidden-items notice is kept but downgraded to a non-blocking, direction-scoped banner that explains the excluded items instead of removing the action. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(forks): surface hidden-item counts by kind + admin path list WIP: expose items dropped by the visibility filter (hidden_ahead/hidden_behind in the compare response): by-kind counts for everyone, kind+path only for admins. * fix(forks): don't close the deployment request on a partial (hidden-items) deploy Making the deploy button reachable in the partial-visibility case exposed a bug: a clean merge-into-parent deploy unconditionally closed any open fork deployment request as "merged" — marking its comments obsolete and notifying the requester and assignees of a merge — even when hidden ahead changes were excluded from the list and left undeployed. Only close the request as merged when the full ahead set was visible (all_ahead_items_visible); otherwise leave it open (with a toast) so someone with full access can finish it. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
76a9523009 |
feat: use derived username instead of email for non-member superadmins (#9857)
* feat: use derived username instead of email for non-member superadmins Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address review - drop redundant username cache, guard whoami membership by email Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: use explicit non_member boolean instead of role string for superadmin banner Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve email from password table for non-member superadmin permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin drafts via shared username->email resolver Adds resolve_username_to_email (usr, then super_admin password fallback for both derived-username and email modes) and uses it in get_email_from_permissioned_as and the drafts get/list endpoints, so a non-member superadmin's drafts resolve and no email leaks into the drafts payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: superadmin-not-in-workspace schedule uses derived username as permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin identity in draft owner-circles, username_to_email, and home filter Applies the password-fallback username resolution to the script/flow/app/draft owner-circle subqueries and the username_to_email endpoint (was an admins-workspace 'username == email' hack), and switches the home items-list user-folder filter to the non_member flag instead of the now-broken username-contains-@ heuristic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: backfill non-member superadmin favorites from email to derived username Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: propagate DB errors in username resolution instead of leaking email (CI review) Addresses cubic-dev-ai P2: get_instance_username_or_fallback_to_email now returns Result and only falls back to the email for a genuine 'no derived username'; a query error propagates so callers fail closed rather than leaking the raw email as the acting username. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: clarify non-member superadmin popover (username used + admin permissions) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep username_to_email endpoint member-only to not disclose non-member superadmin email (CI review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: forbid disabling automate_username_creation once usernames assigned (CI review) Makes the setting effectively one-way once instance-wide usernames exist, so the global-uniqueness invariant that keeps stored u/<username> identities (schedules/triggers/drafts/superadmin ownership) unambiguous can never be dropped back to workspace-local uniqueness. Re-saving false on an already-disabled instance stays a no-op. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
cfcc0b9453 |
chore(main): release 1.744.0 (#9839)
* chore(main): release 1.744.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
74f579e6d9 |
feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) (#9840)
* feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) Add the local edit→preview→run loop for data pipelines (folders of `// pipeline` scripts), the analog of `wmill dev` / `wmill app dev`, usable from a code editor or an agentic loop — without deploying. No backend changes: full body inference comes from the same wasm the frontend uses (windmill-parser-wasm-asset), which returns assets + pipeline annotations in one call; local runs reuse runScriptPreview with _wmill_skip_asset_dispatch. - localGraph.ts: wasm-backed working-tree → asset-graph builder (the enabler) - pipeline show/run --local; new pipeline docs (PIPELINE.md/AGENTS.md) subcommand - pipeline dev watcher + /pipeline_dev page (PipelineDevView) rendering the same PipelineGraphEditor from the pushed local graph, run via preview - cascadeRun.ts: reusable run primitives extracted from the route page - regenerated CLI agent docs See docs/pipeline-local-dev.md for the full design, test steps, and handoff TODOs. The live `pipeline dev` browser preview is implemented but not yet stack-verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): improve local dev preview (run, activity, responsive) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): dev-preview args, multi-root run, ws auto-reconnect Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): connect managed-materialize producer in local dev graph The CLI pinned windmill-parser-wasm-asset ^1.728.1, which predates managed-materialize support (added in 1.733.1); the frontend already pins 1.740.0. The CLI's wasm therefore never emitted `// materialize`, so the producer had no output edge and showed disconnected from its `// on` consumers. Bump the CLI to 1.740.0 (matching the frontend) and translate the parsed materialize target into the producer's write edge + materialize_target, mirroring frontend resolveGraph.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): harden local-dev CLI (bare-.sql crash, defaultTs, docs clobber) Review fixes, complementary to the dev-preview/materialize/multi-root work already on the branch (none overlap those commits): - localGraph: a bare `.sql` (no dialect) made inferContentTypeFromFilePath throw and abort the whole graph build — and wedge `pipeline dev` at startup. Skip the unclassifiable file instead. Also map `bunnative` → parse_assets_ts and add ruby/rlang/nu/powershell to the `#`-comment fallback. - show/run/docs/dev: thread the resolved `wmill.yaml` defaultTs into the graph builder so `.ts` infers under the workspace's runtime (bun vs deno) instead of always bun — `opts.defaultTs` was always undefined (no such CLI flag). - dev: wrap the startup graph build so a half-written file can't abort the watcher. - docs: don't clobber a user-authored AGENTS.md/CLAUDE.md — only (over)write the pointer when absent or already a generated `@PIPELINE.md` pointer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): bind dev WS to loopback + local-graph regression tests - pipeline dev WS broadcast the folder's full script source (scripts[].content + temp_script_refs) unauthenticated on 0.0.0.0:3201 — bind 127.0.0.1 so it's not LAN-reachable (webview localhost + SSH/devbox port-forward still work). - Add regression tests for the just-landed local-graph fixes: bare .sql is skipped (was a build/dev-startup crash), defaultTs threads into .ts runtime inference (bun vs deno), and #-comment languages (ruby) use the # annotation fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --frontend flag for pipeline dev page origin wmill pipeline dev opens <remote>/pipeline_dev, but that route only exists in this build's frontend, so it 404s against a remote whose deployed frontend predates it. --frontend <origin> points the page at a locally-run frontend (REMOTE=<remote> npm run dev) while the API/token still target the remote — enabling the live preview against a real backend before the PR is deployed. No behavior change when omitted. Regenerated CLI agent docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): WS session token + details-pane live-reload refresh Addresses CI review (Codex/Pi/Claude): - dev WS: a browser tab could open ws://localhost:<port>/ws and receive the folder's full source (browsers don't enforce same-origin on WS, loopback bind alone doesn't help). Gate the upgrade on an unguessable per-session token carried in the dev-page URL (verifyClient → 401 without it). Verified: no-token/bad-token connections get 401 with no bundle. - details pane: scriptRes keyed on [workspace, selection, draftScript] didn't re-run on a pipeline dev live-reload (same selection), so the open pane showed stale source. Thread a localScriptsVersion (the pushed bundle) into the key. Verified: editing a selected node's file updates the pane source without reselect. - docs/pipeline-local-dev.md: refresh the stale 'not yet exercised' status + done TODOs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): emit volume: annotation assets in local dev graph Addresses CI review (Codex P1 / Pi P1): the wasm body parser doesn't surface `// volume: <name>` annotations — the frontend (infer.ts:parseVolumeAnnotations) and backend (asset_inference.rs) parse them separately and merge as rw volume assets. localGraph didn't, so a `# volume: cache` producer had no write edge and showed disconnected from its `// on volume://cache` consumer (and pipeline run --local wouldn't schedule downstream). Mirror the leading-comment-block scan (SQL excluded, matching both reference parsers) and merge into inferScriptAssets. Regression test added; verified producer -> volume://cache -> consumer connects. Also (Codex P2): docs/pipeline-local-dev.md manual browser URL omitted the new ws_token param — without it the WS upgrade is rejected and the page sits disconnected. Doc now says to copy the URL the CLI prints (carries wm_token + ws_token) and recommends --frontend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): runAll excludes event roots + review polish Addresses CI review (Codex P1, Claude P2/P3): - pipeline run runAll: derive the whole-pipeline selection from validStarts + descendants instead of all runnables, so an unqualified 'pipeline run <folder>' no longer fires event-trigger roots (kafka/mqtt/…) with empty args/side effects. Verified: a kafka root is excluded from the plan. - cascadeRun.ts runBoundedCascade: use buildLineageDownstreamMap (read-aware) so a pure-reader runs after its producer, and return cyclic — parity with the route page's bounded run (the file is meant to be THE shared correct primitive). - PipelineGraphEditor: storedRightPaneSize starts at 0 so the orientation-aware default (55% stacked / 40% side-by-side) actually applies on first open. - localGraph fallbackParse (go/bash): scan only the leading comment header (no body-comment phantom triggers) and strip key=value options from the asset URI; regression test added. - docs: reject '..' in the folder arg (it writes files under f/<folder>). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): route local previews to the // tag worker Addresses CI review P1: the local graph/bundle dropped the parsed `// tag`, so a node annotated `// tag gpu` ran on the default worker in both `pipeline run --local` and `/pipeline_dev`, while the deployed pipeline routes it to that worker tag. Carry the tag through LocalScript / the pushed bundle / LocalScriptContent and pass it to runScriptPreview at all three launch sites. Verified: a duckdb node tagged `bash` produces a job tagged `bash`; regression test added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): add asset partitions/schemas routes to OpenAPI, use generated client The ducklake asset panels (PartitionStatusGrid, SchemaHistoryPanel) hit /assets/partitions and /assets/asset_schemas via raw fetch with cookie-only auth, because those backend routes were never added to openapi.yaml so the generated client had no methods for them. On /pipeline_dev (token-via-URL, no session cookie) the raw fetches 401'd. Add both GET routes + MaterializedPartition/AssetSchemaVersion schemas to openapi.yaml and call them through AssetService, which injects the bearer token, types, and cancellation automatically. Verified: Partitions + Schema tabs load in /pipeline_dev. (backfill stays a raw fetch — it's an EE-only route not in the OSS spec — with the token added inline.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): regenerate bun.lock for windmill-parser-wasm-asset package.json / package-lock.json carry windmill-parser-wasm-asset@1.740.0 but the tracked bun.lock (the CLI installs/builds/tests via bun) was stale, so fresh bun installs would resolve a different graph than the committed lock. Regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): show asset producer + its runs in the dev-preview panel Selecting a ducklake/asset node in /pipeline_dev showed 'No producer for this asset' because selectionProducers wasn't passed (it's derived from the deployed graph on the route page, absent here). Compute it from the local graph's w/rw write-edges (incl. the // materialize target) and pass it through, mirroring the route page — so the panel shows the producing script and its (preview) runs, including data-test failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): carry annotation metadata onto local-graph runnables The local graph emitted only path/usage_kind/in_pipeline/materialize_target per runnable, so /pipeline_dev and pipeline show --local weren't the same surface as the deployed graph for annotated scripts — missing the badges/lineage the shared canvas renders. Map the wasm-parsed partition_kind, freshness, tag, retry, data_tests, column_lineage, and materialize_strategy (derived append/merge/replace) onto each runnable, mirroring the deployed AssetGraphRunnableNode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude event handlers that are lineage descendants from runAll The runAll guarantee ('never fires an event handler with empty args') only held for event ROOTS — validStarts excludes them, but runAll then unions in descendants(dag, start), so a kafka/mqtt/... handler that also reads an upstream pipeline asset (a lineage descendant of a valid start) still landed in the plan. Add eventTriggerScripts() and subtract it from the selection after the descendant union. +unit test. Also: docs/pipeline-local-dev.md recipe used 'pipeline docs demo_pipeline' without --local (default queries the deployed graph → hits the empty hint); add --local. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): whole-pipeline run cuts at event handlers (drop their downstream too) The prior runAll fix subtracted event handlers from the selection but left their downstream: for manual_root → asset_x → kafka_handler → asset_y → consumer, deleting only kafka_handler left consumer selected, and topoOrder then ran it as a root with missing/stale event-derived inputs. Replace the descendant-union+delete with reachableCutting(dag, validStarts, eventHandlers): traverse from valid starts but treat event handlers as cut points, so a node reachable ONLY through an event handler is dropped while one reachable via a non-event path stays. +unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): recover // tag in the go/bash annotation fallback The wasm path carries out.tag, but the go/bash fallback (and the wasm-error degradation path) only recovered pipeline + on, so a // tag gpu on a bash/go node — or a temporarily-unparseable ts/py/sql node — silently routed the local preview to the default worker while the deployed pipeline routes to the tag. Scan for // tag in fallbackParse too. +test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): extract shared assetProducers helper The 'who writes this asset' write-edge derivation was copied verbatim in PipelineDevView and the pipeline route page — two copies that would drift. Extract assetProducers(graph, selection) into graphTraversal.ts and use it from both, keeping the dev view and route page in lockstep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): only overwrite AGENTS.md/CLAUDE.md when it's the exact generated pointer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): wire local-dev runs into the selected-node runs pane Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude data_upload/webhook entrypoints from auto CLI runs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --upload binds an object to a data_upload/webhook entry point Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): add "Run + downstream" to the dev preview detail form Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): cut non-autorun triggers on all run paths; multi-binding --upload Address CI review: apply the data_upload/webhook/event barrier cut to the single-root and bounded (--from/--to) paths, not just whole-pipeline; accumulate repeatable --upload bindings per script (were overwritten); scope dev upload keys by script+param to avoid basename clobbering; drop <script> from help text. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reseed dev run form when a local edit changes the script's args The read-only pane is keyed on script.path only, so in /pipeline_dev the selected node re-resolves on every WS bundle without remounting; PipelineScriptView cloned script.schema once, so adding/removing args left the run form on a stale schema (could run with missing inputs). Extract PipelineRunForm (owns the SchemaForm clone) and key it on the serialized schema: a real arg change reseeds the form, an unchanged re-resolve keeps in-progress input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): don't cut a scheduled/manual root that also has a non-autorun trigger Address Codex P1: the barrier set subtracted only --upload-bound scripts, so a script with both `// on schedule` and `// on data_upload` resolved as the start yet was also a barrier — reachableCutting skipped it, giving an empty run plan. Subtract all valid starts (schedule/manual roots + bound handlers) from barriers: a legitimately-scheduled root runs on its schedule path even if it also carries a caller-input trigger; pure input-only roots stay cut. Adds a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): deployed non-autorun enrichment, s3:// storage, --to cut accounting, tag regex Address CI review (Codex P1/P1/P2, Pi P2): - Deployed `pipeline run` recovers marker-only data_upload/webhook/email triggers from script bodies (like the `show` path) so input-only entrypoints are cut instead of auto-run empty on the deployed graph. - `--upload s3://<storage>/<key>` keeps the named storage (authority) instead of folding it into the key, matching the S3Object round-trip convention. - Bounded `--to` targets cut by a barrier are reported in droppedEnds (+warning), not reachableEnds. - fallbackParse `// tag` matches a single token (\S+), rejecting multi-word prose. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): header-only deployed marker scan, fail-closed enrichment, default-storage s3 keys Address CI review (Codex P2, cubic P1/P1/P2): - Deployed marker recovery scans the LEADING comment header only (shared recoverHeaderMarkers helper, reused by the show enrichment too) so a body comment `// on data_upload` can't inject a phantom trigger and over-cut. - Deployed run enrichment fails CLOSED: a script-body fetch error aborts the run instead of silently letting an input-only entrypoint run with empty args. - Revert `--upload s3://` to default-storage whole-path keys (matching pipeline `s3://` asset-URI semantics); named-storage authority-splitting broke nested default keys like `s3://raw/2026/events.csv`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reject trailing content on fallback native markers; trim s3:/// key Address CI review (Codex P2, cubic P3): - fallbackParse now requires a native marker (`// on data_upload`) to stand alone; a line with trailing content (`// on data_upload f/foo`, `# on kafka topic`) is rejected, matching the canonical parser and keeping local/deployed parity. - s3UriKey trims a leading slash so the canonical empty-authority default form `s3:///key` doesn't leak a leading slash into the object key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): persist dev WS token per-port so reconnect survives a CLI restart Address Codex P2: the /pipeline_dev auto-reconnect reuses the ws_token from the page URL, but `pipeline dev` minted a fresh random token each start, so a restart on the same port left the open page rejected by verifyClient forever. Persist the token per-port under the user-private config dir (0600) and reuse it on restart, so an already-open page reconnects — matching the reconnect behavior's intent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): scope persisted dev WS token by workspace+folder+port Address cubic P2: keying the persisted token by port alone let a stale browser tab from a previous folder's session on the same port reconnect and receive a different folder's source. Scope the token file by workspace+folder+port so a same-session restart still reconnects, but a different folder on the same port gets a distinct token that rejects stale cross-folder tabs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): caller args can't override skip-dispatch guard; hash the dev token key Address CI review (Codex P1, cubic P2): - makeLaunch / CLI run build args with `_wmill_skip_asset_dispatch` LAST (and drop any caller-supplied copy) so a run-form/`--upload` arg can't re-enable backend asset dispatch while the client orchestrates the cascade (double-run / running deployed subscribers from a local preview). Adds a cascadeRun guard test. - Dev WS token file key is a sha256 of NUL-delimited workspace+folder+port, so different folders (`a/b` vs `a_b`) can't collide onto the same token file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): canonical s3://storage/key --upload parsing; scope dev token by remote+root Address Codex P1/P1: - Restore canonical S3Object URI parsing for `--upload` s3 sources, matching the frontend's `parseS3Object` (`s3://<storage>/<key>`, empty authority ⇒ default, `s3:///key`/`s3:///nested/key` for the default store). `s3://secondary/k.csv` → `{ s3: "k.csv", storage: "secondary" }` so a named-storage object is read from the right store. (This is the canonical convention; the default-storage nested key is served by the `s3:///` form.) - Scope the persisted dev WS token by remote+workspace+root+folder+port (was workspace+folder+port), so two profiles on different remotes (or local checkouts) with the same workspace/folder/port don't share a token — a stale tab can't reconnect across a workspace/remote boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
1c46f899ca |
fix(mcp): stop double-escaping string query params in build_query_string (#9855)
MCP tool arguments were converted to URL query values via `value.to_string()`
+ `trim_matches('"')`. For string values containing JSON (e.g. the `args`/`result`
filters on job listing, `args` on schedule listing), `to_string()` JSON-encodes the
string and escapes inner quotes with backslashes; stripping the outer quotes leaves
`{\"k\":\"v\"}`, which the backend's `serde_json::from_str` then fails to parse,
falling back to `FALSE` and returning zero results.
Use `value.as_str()` to emit the raw string content for `Value::String`, falling
back to `value.to_string()` for non-string types (numbers, booleans). Adds
regression tests covering JSON-string, non-string, and plain-string params.
Fixes WIN-2114
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
hugocasa
|
b4b0c6a93e |
feat: add dev workspaces paired with a lockable prod workspace (#9793)
* feat: add dev workspaces paired with a lockable prod workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: gate dev-workspace prod-lock on admin and prevent attach cycles Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: redirect locked-prod edits into the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make dev-workspace settings tab available on CE (was EE-gated) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: lock prod against forking too and funnel edits to the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: open dev item page on edit and tailor dev-workspace lock messages Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: prevent nested dev workspaces and hide dev option when one exists Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: drop the redundant already-has-dev hint on the fork form Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: badge dev workspaces and sort them ahead of forks in the tree/switcher Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: label dev workspaces as 'Dev workspace of X' instead of 'Fork of X' Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: label edit as 'Edit in <dev>', cover editor headers, auto-expand dev in tree Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: split prod lock into separate block-deploy and prevent-forking toggles Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: make resources/variables workspace-specific from compare page Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: steer AI-chat sessions to the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: refine session fork options and lock guidance for dev/prod Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: session picker reads prod's real rules, default to current ws Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: copy members into forks and clarify dev-workspace root labeling Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: place the workspace id field under the fork name Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address dev-workspace review findings and harden fork detection Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: regenerate sqlx offline cache Restores entries dropped during the origin/main merge and adds the dev-workspace queries (is_dev_workspace, ws_specific, has_parent). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address second-round dev-workspace review findings Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address Pi and Codex review findings on dev-workspace endpoints Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: gate locked-dev git-branch fork on admin and validate ws_specific path Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: clear prod dev-lock when deleting an attached dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: consolidate dev-workspace migration and scope all-group join to attach Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: restore dev-workspace CHECK into consolidated migration and scope all-group join Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: drop copy_members from the dev-workspace attach path Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: dev-workspace lifecycle/auth fixes from Codex review round Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: explicit create-in-other for workspace-specific items Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make create-in-other strictly create-only (never overwrite target) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: return 403 (not 401) for dev-workspace permission denials Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: allow attaching a same-family fork as a dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: emphasize the go-to-dev action in the no-direct-deploy alert Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: seed a resource's linked variables when creating it in the other workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: judge workspace deploy/fork locks against the user's identity in that workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: clarify create-in help text in workspace-specific panel Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: admin-gate dev-workspace creation and harden lock/seed edges Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: preserve a staged fork's source on picker create-mode re-entry Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: clear dev flag on archive and check dev existence server-side Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make create-in-other atomically create-only via direct create Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: create-only resource insert, ws-specific list scopes, archive lock guard Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: reserve the dev_workspace_lock protection-rule name from the public API Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: reattach create_protection_rule doc comment to its function Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: make dev-archive pairing teardown atomic with the archive Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: follow deploy_to on root rename; show dev pairing to non-member prod admins Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: copy creator metadata on fork; invalidate fork routing cache on rename Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: accept g/ paths in set_ws_specific; gate copy_members to dev workspaces Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a9ffdb996b |
chore(main): release 1.743.0 (#9837)
* chore(main): release 1.743.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
96c0ff65bd |
chore(main): release 1.742.0 (#9830)
* chore(main): release 1.742.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
75ba81b2d2 |
fix(audit): don't read pg_authid from an elevated context in S3 export migration (#9832)
* fix(audit): don't read pg_authid from an elevated context in S3 export migration Migration 20260626132251 aborted instance startup on managed Postgres (e.g. Cloud SQL) with "Modifying pg_authid or pg_auth_members is not allowed in elevated context": the audit S3 export "oldest in-flight xact_start" floor probe calls pg_has_role(...), which reads pg_authid, and managed providers forbid that read from an elevated context. The migration ran the probe inline in its UPDATE, so the whole migration — and the instance boot — failed. Extract the probe into a shared SQL function audit_logs_s3_oldest_inflight_ts() that returns the oldest in-flight xact_start (when cluster-wide stats are visible) or NULL otherwise. The pg_has_role read is wrapped in a plpgsql BEGIN/EXCEPTION subtransaction, so a pg_authid failure returns NULL (callers fall back to a conservative 7-day window / reject) instead of aborting. is_superuser (a GUC, no catalog read) is checked first to short-circuit. The migration's trigger and UPDATE, the OSS backfill try_start, and the EE exporter/startup anchor (companion windmill-ee-private PR) all route through it. Because 20260626132251 already shipped, it is added to the potentially_stale list in windmill-api/src/db.rs: on startup the stale _sqlx_migrations row (checksum mismatch) is deleted and the fixed, idempotent migration re-applies, so already-migrated instances upgrade without a checksum-mismatch boot failure. Fixes WIN-2108 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 95352c13c4c82247d8cfd80936f9203aeb079802 This commit updates the EE repository reference after PR #635 was merged in windmill-ee-private. Previous ee-repo-ref: 136f49a52af922868acac33abf8198913a9e835c New ee-repo-ref: 95352c13c4c82247d8cfd80936f9203aeb079802 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
9172a0945b |
chore(main): release 1.741.0 (#9804)
* chore(main): release 1.741.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
577ceeee86 |
perf(audit): re-anchor S3 audit export on enable + opt-in backfill (#9818)
* [ee] perf(audit): re-anchor S3 audit export on enable + opt-in backfill
The S3/GCS audit-log export's steady-state query filters by `age(xmin)`
(unindexable), so the only scan bound is the timestamp floor. On a fresh
enable the floor was epoch, and on a re-enable the cursor resumed from its
pre-disable position — either way the first run scanned the whole
`audit_partitioned` table. Under a `statement_timeout` (e.g. Aiven) that scan
never completes: the cursor never advances, nothing is exported, and the
repeated full scans saturate the database.
Re-anchor on enable (EE companion, windmill-ee-private#634):
- New trigger migration records a recent timestamp floor instead of the epoch
sentinel and `DO UPDATE`s the cursor to the current snapshot xmin on
re-enable, so the export always resumes from ~now and never rescans history.
Includes a one-time fixup for legacy epoch-sentinel checkpoints on upgrade.
Opt-in historical backfill (new `audit_logs_s3_backfill` module + endpoints):
- Exports a chosen `[from, to)` window on demand, scanning strictly by
`timestamp` (the partition key) in bounded keyset pages — each query is an
index scan capped at one page (verified via EXPLAIN: later partitions
`never executed`, ~11ms/page), so it stays well under any statement timeout
regardless of window size. Writes alongside the steady-state objects under
logs/audit/, without touching the xmin cursor.
- POST /settings/audit_logs_s3_backfill {from,to} (super-admin + Enterprise),
GET /settings/audit_logs_s3_backfill_status.
Also repurposes the status endpoint's `bootstrapping` flag to mean "draining a
backlog" (the cursor is capped and catching up), and updates the setting
description to point operators at the backfill.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): heartbeat backfill lease per object; bump EE ref
Address review (cubic): persist progress (refreshing the lease heartbeat) after
every object PUT in the backfill page loop, not only once per page, so the gap
between heartbeats stays well under STALE_HEARTBEAT_SECS even on slow uploads
and another replica can't re-claim mid-page and run a concurrent backfill.
Bumps ee-repo-ref.txt to pull in the EE test-race fix (folding the backlog-drain
regression into the single audit e2e test).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject unstable backfill windows; bump EE ref
Address review (P1): the backfill keyset-pages over rows visible at scan time
and declares completion when the scan runs dry, but a row's `timestamp` is its
inserting transaction's `xact_start`. A window whose upper bound is recent or in
the future could silently omit a transaction that started inside `[from, to)`
but commits after the scan passed that timestamp. `try_start` now rejects any
`to` newer than the oldest in-flight `xact_start` (everything strictly older
than the oldest running transaction is committed and stable), using the same
trustworthy stats gating as the exporter's floor (restricted role / 2PC → a
7-day-old cutoff).
Bumps ee-repo-ref.txt for the EE monotonic-checkpoint fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): re-anchor legacy epoch checkpoints instead of synthetic floor
Address review (P1): the legacy-checkpoint fixup stamped last_oldest_inflight_ts
to now()-7d while leaving the old last_xmin in place. On an instance that
enabled export on the old code >7 days ago and got stuck before the first
successful batch, the next run would filter post-enable rows older than 7 days
out via `timestamp >= ts_floor` while still advancing last_xmin over the
interval — silently dropping them (the same floor-vs-cursor loss class fixed
elsewhere in this PR), and contradicting the "nothing committed after enabling
is skipped" guarantee.
A stuck epoch-sentinel checkpoint cannot be safely resumed (its backlog can be
arbitrarily old, so any recent floor prunes rows the cursor then skips, and an
epoch floor reintroduces the full scan). Re-anchor it to the migration's current
snapshot xmin instead — exactly like a fresh enable — so the export resumes
cleanly from ~now and the never-exported pre-upgrade window is recovered via the
opt-in backfill rather than silently dropped. Reword the setting description so
it no longer implies the disabled/legacy window is covered by the cursor.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(audit): end-to-end integration tests for the object-store backfill
The backfill previously had only SQL-level/EXPLAIN validation. Add real
integration tests (in-memory object store, sqlx::test) exercising the public
path:
- backfill_exports_window_in_pages: with the page size forced to 2 rows, a
settled 3-day window is exported across multiple keyset pages; asserts every
in-window row lands exactly once, rows outside [from,to) are excluded, a day
that straddles a page boundary yields more than one object, progress counts
match, and a re-run is idempotent (deterministic keys overwritten, no dupes).
- backfill_rejects_unstable_window: a future/live `to` is rejected as unstable,
a window safely in the past is accepted.
Adds a test-only PAGE_ROWS override so multi-page behaviour is exercised with a
handful of rows.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(audit): note backfill scope is audit_partitioned only
Make explicit that, like the steady-state export, the backfill reads only
audit_partitioned; the pre-partitioning `audit` table is intentionally out of
scope (not a missed case).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject backfill windows before the partitioned boundary
Address review (Codex P1): the backfill reads only audit_partitioned, but
pre-partitioning history lives in the legacy `audit` table (still read by audit
list/get via UNION ALL, and retained for the configured period — 365 days by
default on EE). Since the setting text points operators at this API for
"pre-existing history", a window overlapping legacy rows would report completion
while silently omitting them.
Per the decision to not export the legacy table, reject instead of silently
omit: try_start now rejects a `from` earlier than the oldest audit_partitioned
timestamp (every legacy row predates the partition cutover, so a `from` at/after
that boundary can never overlap them). Reworded the setting text to scope the
backfill to the partitioned era. Added a regression test, plus an RAII guard
(cubic P2) so the test-only globals are restored even if an assertion panics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): backfill object keys per-window; require trustworthy settled cutoff
Address review (two P1s):
- Object-key overwrite loss: keys were `dt=<day>/audit_backfill_<min_id>.ndjson`.
A narrower, overlapping backfill can start a day's page at the same first row
(same min_id) but hold fewer rows, and `put` would overwrite a broader run's
object — silently dropping the rows only that object held. Include the
requested window in the key so different ranges write disjoint objects (same
window re-runs stay idempotent; consumers dedupe overlapping rows by id). New
regression test (verified red→green).
- Untrustworthy settled cutoff: when min(xact_start) isn't trustworthy (role
lacks pg_read_all_stats/superuser, or a prepared 2PC txn exists), the old
now()-7d fallback could still let an old transaction commit rows inside an
accepted window after the scan, so a "complete" backfill silently missed them.
Since a backfill asserts completeness, reject in those cases instead of
falling back. (The continuous exporter keeps its 7-day fallback — it only
claims bounded lag.)
Also makes the tests robust under the parallel runner: run_backfill takes the
store as a param, so tests pass a local in-memory store (no global
OBJECT_STORE_SETTINGS race) and serialize on the PAGE_ROWS override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(audit): reject backfill overlapping legacy table; regen deref openapi; trim migration comment
Address review (1 P1 + 2 P2):
- Empty-partition backfill (P1): the min(audit_partitioned) guard no-ops when
audit_partitioned is empty, so an upgraded instance with legacy `audit` rows
but no partitioned rows yet would accept a window and complete with zero rows,
silently omitting the legacy rows. Check the legacy `audit` table directly:
reject any window that overlaps a legacy row (subsumes the boundary check and
covers the empty-partitioned case). Test updated accordingly.
- openapi-deref (P2): regenerate openapi-deref.yaml/json (served via include_str!)
so /openapi.{yaml,json} expose the new backfill endpoints.
- Migration comment (P2): trim the PR-history narration to the durable
constraints (why a recent floor and a monotonic cursor are required), per
AGENTS.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* chore: update ee-repo-ref to b821fecccbcba2efed544890576bf2b84321d70d
This commit updates the EE repository reference after PR #634 was merged in windmill-ee-private.
Previous ee-repo-ref: 6b191b77aabcf77658ad4f9031576e0d7b66bf89
New ee-repo-ref: b821fecccbcba2efed544890576bf2b84321d70d
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben Fiszel
|
b7a227f860 |
chore(main): release 1.740.0 (#9776)
* chore(main): release 1.740.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
0dbd9c1231 |
perf: eliminate dual-connection DB pool contention across worker, queue, and api (#9798)
* perf: eliminate dual-connection DB pool contention across worker, queue, and api Reuse the held transaction (or move pool reads before begin()) instead of checking out a second pool connection while a tx is open, extending the fix from #9789/#7861. Targets the per-worker pool (max 5) hot paths plus several server-pool API handlers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: pass owned pool to get_email_from_permissioned_as in http trigger handler The generified signature takes impl PgExecutor; the http trigger handler passed &db where db is already &DB, yielding &&Pool which does not impl PgExecutor (only surfaced under the full feature set in CI). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep RLS-exposed reads on the non-RLS pool and isolate flow-eval reads in a savepoint Addresses review of the dual-connection sweep: - worker_flow: wrap the stop_after_all_iters_if reads in a SAVEPOINT. The caller swallows the error and keeps using tx, so a DB read failure must not leave the outer transaction aborted (it would fail the later commit). Matches the previous pool-read semantics. - Revert reads that were moved onto an RLS (user_db) transaction back to the non-RLS pool, since RLS row-visibility/role context can change results: push_scheduled_job (email/tag/settings lookups; reachable with a user_db tx from api-schedule/api-flows), push_inner native-retry dedicated_worker routing (RLS isolation variants), resources.rs app-namespace folder auto-create (non-admins must not be blocked), and the script archive/delete UPDATEs. Non-RLS db.begin() reuse and move-before-begin are kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: failpoint proving the stop_after_all_iters_if savepoint isolates an aborted read Adds a worker-crate failpoints feature and a data-driven hook: when the stop_after_all_iters_if expr is the magic sentinel, the in-evaluation read runs SELECT 1/0 to abort its (savepoint) transaction. The test asserts the flow still completes (iteration marked failed) — which only holds if the savepoint keeps the outer status-update transaction committable. Without the savepoint the abort would poison the outer tx and the job would never complete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
ba768fee88 |
feat(api): add structured endpoint for flow logs (#9797)
Add `GET /w/{workspace}/jobs_u/get_flow_all_logs_structured/{id}` as a
JSON alternative to `get_flow_all_logs`. It returns the same flow log
tree as an array of per-job entries (job_id, label, kind, step path,
depth, parent module type, sibling index/count, and resolved logs)
instead of a single delimited text blob, so callers can render or
process logs per-step without parsing the `=== ... ===` markers.
The shared auth, recursive-CTE query, and label-building logic is
extracted into `collect_flow_log_entries`; the existing text endpoint
now formats those entries and produces byte-identical output.
Fixes WIN-2102
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
6664ce6dc0 |
fix(frontend): apply script editor timeout to preview/Test runs (#9794)
The custom timeout configured in the script editor settings was only
honored for deployed script runs: it is persisted on the script row and
passed as custom_timeout when running by hash/path. Preview ("Test")
runs derive their timeout solely from the `timeout` query param of
/jobs/run/preview, which the editor never sent, so Test silently fell
back to the instance default.
Forward the editor's timeout setting through ScriptBuilder ->
ScriptEditor -> JobLoader.runPreview as the preview run's timeout query
param. The backend already clamps custom_timeout against the instance
max in resolve_job_timeout, so previews get the same ceiling as deployed
runs.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
 centdix
|
9d61e4e59e |
feat: self-host docs search for chat, mcp, cli; drop inkeep (#9772)
* feat: self-host docs search for chat, mcp and cli; remove inkeep
Embed a vendored docs snapshot (llms.txt/llms-full.txt) in the backend and
serve ranking + page rendering from GET /api/docs/{search,page}. The AI chat,
the MCP searchDocs/readDocsPage tools, and 'wmill docs' all consume it, so docs
search works with no runtime egress and is no longer EE-gated. Removes the
inkeep proxy. EE companion deletes inkeep_ee.rs (ee-repo-ref bumped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor: name read_docs_page param `url` instead of `path`
search_docs returns each hit's `Source` URL, so the read tool now takes a
`url` argument to match — the AI/MCP loop reads "search gives a Source URL,
read takes that url" rather than copying a `Source:` URL into a `path` slot.
A bare `/docs/...` path is still accepted and canonicalized before lookup.
Regenerated openapi-deref, the MCP endpoint tools, and the frontend client.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ci: add scheduled workflow to refresh the vendored docs snapshot
The backend embeds docs_snapshot/*.gz at build time, so the in-product docs
corpus is otherwise only as fresh as the last manual fetch.sh run. This adds a
weekly (and manually dispatchable) job that re-runs fetch.sh, sanity-checks the
result against truncation/garbage, and opens a PR via the internal app when the
snapshot changed — so a human reviews the docs diff before it rides into the
next release build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: make docs tool-result strings caller-neutral
The search/page endpoints back three differently-named consumers (the AI chat
`read_docs_page` tool, the MCP `readDocsPage` tool, and the `wmill docs` CLI),
so the shared rendered text shouldn't name one of them. Refer to "the docs
page-reading tool" and its `url` argument instead, and add tests pinning the
caller-neutral follow-up guidance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: point ee-repo-ref at inkeep-removal companion rebased on EE main
The companion branch now carries only the inkeep_ee.rs deletion on top of EE
main (was based on the native-job-retry EE line, which polluted the EE PR diff).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(docs): expose docs:read in token catalog; precompute lowercased corpus
Addresses two review nits on the self-hosted docs PR:
- docs:read was enforced (ScopeDomain::Docs) but missing from the token scope
catalog (token.rs ALL_SCOPES), so it couldn't be selected when creating a
standard scoped token in the UI — leaving scope-restricted CLI/MCP docs use
effectively ungrantable. Add a read-only "Documentation" group (no write
surface) and a test asserting it is exposed.
- search ran page.body.to_lowercase() on the whole corpus per query. Lowercase
body/title/description once at parse time (into the OnceLock corpus) and scan
the precomputed copies instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* chore: update ee-repo-ref to 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
This commit updates the EE repository reference after PR #630 was merged in windmill-ee-private.
Previous ee-repo-ref: c7ec3a0c2fa38d4cb5e50bf0265eef4710de4860
New ee-repo-ref: 27a4f41b8e5603d6e444efcfc420bd1c44a07eed
Automated by sync-ee-ref workflow.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com>
|
||
|
Ruben Fiszel
|
12f92e3ab7 |
[ee] feat(backend): native script retry without one-step-flow wrapping (#9688)
* feat(backend): native script retry without one-step-flow wrapping Schedules and data pipelines that retry a single script previously wrapped it in a one-step flow (JobKind::SingleStepFlow), creating extra job rows, a v2_job_status row, and UI projection complexity. This adds native retry on a plain JobKind::Script job. - RetrySettings: flatten Retry into a deduped retry_settings table, carried via the existing runnable_settings_handle (lazy, off the hot path). - push() materializes a bare-script-with-retry SingleStepFlow into a native Script job (gated on min-version + no handlers/retry_if). - add_completed_job re-pushes the next attempt on failure with backoff, tracking the attempt counter in v2_job_queue.extras and the chain via parent_job; schedule completion handlers fire only on the terminal attempt. - frontend: ScriptRetryChain shows the attempt chain on the run page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(backend): native retry_if eval + per-occurrence schedule handlers Extends native script retry to the two cases that previously stayed on the one-step-flow path: - retry_if: evaluated natively on the failure path via a feature-gated windmill-jseval dep (quickjs) over the failure result + flow_input; push materializes such policies natively only when quickjs is available. - on_failure_times / on_recovery: apply_schedule_handlers now resolves each past scheduled occurrence's terminal status across its native-retry chain (root OR any parent_job=root child succeeded) and excludes the current occurrence, so the counting is per-occurrence rather than per-attempt. All scheduled-script retries now go native (schedule.rs gate removed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(backend): always materialize retry_if natively; unsupported without quickjs retry_if is evaluated by the worker (which always has quickjs), not the pusher, so gating materialization on the pusher's feature was wrong. The flow path was never a real fallback either — the flow runtime needs quickjs to evaluate retry_if too. retry_if now always goes native; on a worker without quickjs it is unsupported and fails closed (no retry). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(backend): un-park asset-cascade (pipeline) retry Native retry resolves the blocker that parked pipeline retry: a retried subscriber is now a Script job (not a one-step flow / flow step), so it stays eligible for asset dispatch and can trigger its own downstream on recovery. - scripts.rs: persist // retry <count> [<delay>] to script_trigger on asset edges (was dropped with a TODO warning). - asset_dispatch.rs: is_eligible_kind keys off flow_step_id, not parent_job, so native-retry attempts dispatch on success while flow steps stay excluded. - tests: retry-bearing subscriber now dispatches as a native Script carrying the policy in runnable_settings_handle; native-retry attempt is eligible. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): cap native retry interval, lazy result serialization, idempotent retry push Hardening from a self-review of the native retry path: - Cap the backoff at MAX_RETRY_INTERVAL to match the flow-runtime path (evaluate_retry); the exponential formula could otherwise schedule up to ~18h vs the flow path's 6h. - Serialize the failure result lazily (only when a retry_if policy needs it), so the common failure no longer pays the serialization on the failure path. - Push each retry with a deterministic id per (root, attempt). If a worker dies between enqueueing the retry and finalizing the current attempt, the reaper re-handles the attempt and lands here again — push rejects the duplicate id, so the retry is enqueued exactly once (no double-retry). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): defer schedule handlers idempotently on retry-push replay (review P1) Address local-review findings: - P1: retry_pending was derived from the retry push *result*, so on a worker crash + reaper replay the duplicate-id push returned Err → retry_pending flipped to false → apply_schedule_handlers fired for the non-terminal attempt (and the terminal attempt later fired them again). Pre-check whether the deterministic retry id already exists and report it as pending without re-pushing, so the handler-deferral invariant is crash-idempotent too. - P2: refresh the stale 'wrap the script in a one-step flow' comment in the asset-cascade retry push — it now materializes a native Script. - Add RetrySettings <-> Retry round-trip unit tests (clamping edges). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): native retry chain + per-occurrence status sqlx tests Close the two integration-test gaps flagged in local review: - chains_attempts_and_is_idempotent: drives maybe_enqueue_native_script_retry through attempt0 -> retry1 -> retry2 -> exhausted (counter, backoff, max-attempts) and asserts crash-replay idempotency (the P1 fix: a replayed completion reports pending without double-enqueueing). - per_occurrence_status_counts_recovered_as_success: pins the exact per-occurrence terminal-status query from jobs_ee::apply_schedule_handlers — a retried-but- recovered occurrence counts as success, retries (parent_job set) are excluded from occurrence counting, and the current occurrence is excluded. - canceled_job_does_not_retry: cancellation wins over a pending retry. Runtime sqlx API (no .sqlx cache entry needed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): exclude schedule handlers from the retry-attempt chain The retry chain listed all script children of the root by parent_job, but schedule completion handlers (on_failure/on_recovery/on_success) are also script children — when the occurrence has no retries, the handler's parent is the root itself, so a successful, never-retried job rendered a bogus 'Retries (1)' badge pointing at the handler. Filter children to re-runs of the same script (matching script_hash); real retries keep the root's hash, handlers run a different script. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(frontend): surface schedule handlers on the run page Extend the run-page chain component with schedule completion handlers: - A 'Handlers' row on a scheduled job links to the on_failure/on_recovery/ on_success runs that fired for that occurrence (found as children of the terminal attempt, identified by their synthetic created_by). - A handler's own run page now shows a 'Failure/Recovery/Success handler' label with a link back to the run it handled and its schedule. on_recovery and on_success share created_by, disambiguated by the recovery-only error_started_at arg. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): restore folder_default_permissioned_as sqlx caches dropped by prepare An earlier `cargo sqlx prepare` on this branch ran before #8801's folder_default_permissioned_as test merged in, so it pruned the 3 query caches that test needs; cargo_test then failed under SQLX_OFFLINE. Restore them from main. * fix(backend): only cascade assets from native retry attempts, not handlers (review P1) is_eligible_kind keyed dispatch on flow_step_id alone, so every parented Script child became asset-eligible — including schedule/error/recovery handlers (Script jobs with parent_job set and no flow_step_id). A handler that declares assets would then trigger a cascade the old parent_job IS NULL guard prevented. Gate parented jobs on being a genuine retry attempt: a re-run of the SAME runnable as its chain parent (handlers run a different script). Runtime query, no sqlx cache. * fix(backend): cache the private-gated retry_setting asset-dispatch test query The same prepare-without-private that dropped the folder_default caches also pruned the cache for the retry_setting_dispatches_subscriber_as_native_script test query (asset_trigger_dispatch.rs:721). Regenerated with --features private. * fix(backend): exclude handler children from per-occurrence recovery (review) A scheduled occurrence's on_failure/on_success handler runs as a successful child (parent_job = occurrence), and the per-occurrence success EXISTS counted ANY successful child — so a failed occurrence whose error handler succeeded was marked 'recovered', breaking on_recovery (test_script/flow_schedule_handlers in the merge) and on_failure_times counting. EE query now scopes the EXISTS to same-runnable children (only native retry attempts); regenerate sqlx cache + bump ee-repo-ref. native_retry_test gains a handler-child regression case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(backend): scheduled-script retry is a native Script, not SingleStepFlow test_push_script_with_retry / test_try_schedule_with_retry (from main) asserted the old SingleStepFlow wrapping for scheduled-script retry; this PR makes it a native Script. Update both to assert kind='script' and that the retry policy is carried via runnable_settings_handle. * fix(backend): preserve dedicated_worker on native retry + saturate count casts (cubic) Address cubic CI review: - P1: the SingleStepFlow->native Script materialization dropped dedicated_worker, so a dedicated-worker scheduled script lost its dedicated pool on retry. Resolve it from the script row in push so the materialized Script keeps the dedicated tag. - P2: saturate the u32->i32 retry-attempt narrowings (RetrySettings::from) and the u32->i16 // retry count narrowing (scripts.rs) instead of wrapping. * fix(backend): use a retry-specific signal, not runnable equality (codex review) Address Codex CI review: - P1: is_native_retry_attempt treated any same-runnable parented Script child as a retry. WAC v2 inline children have that exact shape, so an inline child of an asset producer would cascade. Use a retry-specific signal instead: the job carries a retry_settings policy (always re-inserted by maybe_enqueue) and has no flow_innermost_root_job. Apply the same flow_innermost guard to the EE per-occurrence EXISTS (WAC inline children must not count as a recovery). - P1: the deterministic retry-id pre-check raced with push; a concurrent duplicate now resolves as 'retry pending' (re-check on the duplicate-id error) instead of flipping retry_pending to false and firing handlers early. - Tests: native_retry + asset_trigger_dispatch gain WAC-inline-child cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(backend): explicit native_retry_attempt marker, drop heuristics Replace the per-site "is this a retry?" inference (parent_job + runnable match + flow_innermost / retry_settings) with one explicit marker: a sparse native_retry_attempt(job_id, attempt) table, written in maybe_enqueue. The marker also carries the attempt counter (previously in v2_job_queue.extras), so it's the single source of truth. - asset_dispatch: is_native_retry_attempt is now one indexed EXISTS on the marker. - EE per-occurrence query: joins the marker instead of guessing by runnable/flow_innermost. - maybe_enqueue: reads/writes the marker (persistent) instead of queue extras. - Lifecycle: swept with the job in retention (log_cleanup), no FK to keep bulk delete cheap. - Eliminates handler / WAC-inline-child misclassification by construction. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): sweep native_retry_attempt markers in the periodic retention path too (codex) The marker has no FK and relies on retention cleanup; log_cleanup.rs swept it but the periodic monitor.rs path deleted v2_job rows without it, orphaning markers. Add the same WHERE job_id = ANY(...) sweep there. * fix(backend): widen native_retry_attempt.attempt to integer (cubic) The smallint column was cast to/from u32 and could wrap a retry chain longer than i16::MAX into premature exhaustion. Use integer, matching the retry policy's i32 attempt count, so no narrowing occurs on the maybe_enqueue read/write path. * feat(frontend): mark retries via is_retry on listJobs; drop SAVEPOINT - Expose an is_retry flag on jobs (UnifiedJob/CompletedJob/QueuedJob + openapi), computed from the native_retry_attempt marker. The run-page chain now filters retry attempts by is_retry instead of the script_hash heuristic, so WAC v2 inline children (same script, parent_job) no longer render as retries (codex). - Revert the marker-cleanup SAVEPOINT (an unused pattern in this codebase): keep the plain catch-and-continue matching the other side-table deletes; the table is created by a startup migration so it always exists when cleanup runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): mark is_retry sqlx(default) so non-list job queries can omit it The single-job GET query maps directly to CompletedJob/QueuedJob via FromRow but does not select is_retry, which errored with "no column found". Only the list endpoint populates the marker; #[sqlx(default)] lets every other query omit the column and default to None. * feat(backend): select is_retry in single-job GET too for consistency The list endpoint already exposes the marker; populate it on the single-job GET (both completed and queued variants) as well so a run loaded directly reflects its retry status. #[sqlx(default)] stays as a safety net for any other query. * feat(backend): reap orphaned native_retry_attempt markers via periodic sweep The marker has no FK to v2_job (to keep the hot bulk retention delete cheap), so direct job deletions (workspace/job delete, schedule clearing) would leave marker rows orphaned. Rather than add explicit cleanup to every v2_job delete site (which must then be remembered for every future path), reap orphans in the periodic delete_expired_items pass: DELETE FROM native_retry_attempt WHERE NOT EXISTS (the job). The table is sparse so the anti-join drives off it and probes v2_job by PK — cheap. Retention still sweeps markers inline (keeps the table small so this stays cheap); a transient orphan is harmless (nothing reads is_retry for a gone job). * fix(frontend): include flow handlers in retry chain handler row (codex) Schedule on_failure/on_recovery/on_success handlers can be flow paths (flow/...), whose handler job is a flow, not a script. The chain fetched children with jobKinds:'script', hiding flow handlers. Drop the kind filter — retry attempts are still selected by is_retry and handlers by created_by, so both kinds surface. * fix(backend): carry concurrency/debouncing settings into native retries maybe_enqueue re-pushed the next attempt with ConcurrencySettings/DebouncingSettings ::default(), dropping the script/pipeline concurrency settings the failed job carried in its runnable_settings_handle. A retry of a concurrency-limited script then inserted no concurrency_key and ran unbounded. Resolve both from the same handle (cached) and pass them in the payload, which push forwards to the materialized retry. Adds a regression test asserting the retry's handle resolves to the concurrency settings. * fix(backend): carry concurrency/debounce into scheduled-retry root + document retry-helper auth (codex) P1a (schedule.rs): the scheduled-retry materialization fetched the script's concurrency/debounce settings but passed ConcurrencySettings/DebouncingSettings ::default() into the SingleStepFlow payload, so the root attempt's handle held only the retry policy and the whole chain ran unbounded. Pass the fetched settings. Regression test asserts the root handle resolves to retry + concurrency. P1b (jobs.rs): document maybe_enqueue_native_script_retry's authorization contract — it is pub only for the integration test; the sole production caller is the worker completion path passing a DB-derived, already-authorized MiniCompletedJob. * docs(backend): attach native-retry auth contract to the function itself (codex) The doc block was merged with eval_retry_if's doc and bound to that function, leaving maybe_enqueue_native_script_retry undocumented. Split them: eval_retry_if keeps its own doc; the native-retry + authorization contract now sits directly above maybe_enqueue_native_script_retry. * docs(backend): regenerate served openapi-deref with is_retry + fix stale comments (codex) - Regenerate openapi-deref.{yaml,json} (served from lib.rs): they were stale since 1.734.0 and lacked is_retry on QueuedJob/CompletedJob, so clients reading the served spec couldn't see the field. Now current at 1.739.0. - schedule.rs: a retry_if gate is evaluated at failure time and fails closed without quickjs (no retry); it does not fall back to a flow path. - windmill-types jobs.rs: is_retry is selected by both the list and single-job GET endpoints (not list-only). * docs(backend): fix remaining stale retry_if/quickjs comments (codex) The retry_if block and the push materialization comments claimed push keeps retry_if on a flow path / the worker always has quickjs. The code always materializes native retry and the no-quickjs eval_retry_if path fails closed — correct the comments to that constraint. * docs(backend): fix stale quickjs-fallback + schedule-handler-restriction comments (codex) - Cargo.toml quickjs feature: without quickjs a retry_if gate cannot be evaluated and the job does not retry (no one-step-flow fallback). - jobs.rs handler-defer comment: apply_schedule_handlers resolves per-occurrence failure/recovery status across the retry chain, so the old 'restricted to schedules whose handlers don't need per-occurrence counting' claim is dropped. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
cd42c6ca18 |
fix: decrypt secret variables via external backend in common resolvers (#9784)
`get_variable_or_self`, `get_variable_or_self_as`, `get_secret_value_as_admin` (and `transform_json_unchecked`'s `$var:` branch) in windmill-common always ran the raw `variable.value` through `decrypt()`. With an external secret backend (HashiCorp Vault / Azure Key Vault / AWS Secrets Manager) configured, that column holds a `$vault:`/`$azure_kv:`/`$aws_sm:` marker rather than base64 ciphertext, so base64 decoding failed with `Invalid byte 36, offset 0` (the `$`). This broke GitHub App git sync (git_sync_ee.rs) and any other consumer of these resolvers when an external backend is active. Move backend resolution (`get_secret_backend`, `get_secret_value`, `is_*_stored_value`, caching) into `windmill-common::secret_backend::resolver` so the low-level variable resolvers can route external markers through the configured backend's `get_secret()` instead of `decrypt()`. The windmill-store and windmill-api `secret_backend_ext` modules now re-export these from windmill-common (single source of truth / single backend cache) and keep only their write-side helpers. No `_ee.rs` files change. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
920f5688ca |
chore(main): release 1.739.0 (#9746)
* chore(main): release 1.739.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
 Guilhem
|
42c5e7a3fc |
feat: scope AI sessions per workspace root with lifecycle reconcile (#9734)
* feat: scope AI sessions per workspace family with lifecycle reconcile Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: centralize session reconcile trigger + extract pure lifecycle decision Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * perf: remove unused workspace family index * refactor: scope sessions by workspace root id, drop family_id column Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sessions): preserve user-archived sessions when archiving their workspace archiveSessionsForWorkspace tagged every session archivedByWorkspace, including ones the user had already archived by hand, so a later workspace unarchive auto-restored them. Skip already-archived sessions so only workspace-archived ones are tagged, matching decideSessionLifecycle. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: archived-session banner with unarchive, suppress workspace-gone banner while archived Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: re-root sub-fork sessions on reconcile when an ancestor is deleted Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: group AI sessions by workspace family with show-all-workspaces filter Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: revert unrelated AIProviderPicker cosmetic changes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: hide per-session unarchive when workspace is gone, show move/discard instead Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: GC attached files on lifecycle delete + reconcile on sidebar fork delete Addresses Codex review: deleteSessionsForWorkspace/reconcile delete now GC linked files (deleteItemsForSession), matching deleteSession; sidebar deleteFork now reconciles so surviving child forks re-root off the deleted ancestor. Also de-flaked post-rehydrate reads in the IndexedDB tests via vi.waitFor. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: don't strand user if post-delete reconcile throws; refresh stale warmSessions comment Addresses auto-review P2s: wrap reconcileAfterWorkspaceChange in deleteFork so the parent switch + navigation always runs even on reconcile failure; correct the warmSessions comment which no longer holds under 'Show all workspaces'. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: don't fail/strand fork archive+delete when client session cleanup throws Addresses cubic P1/P2 on forks/compare: the workspace archive/delete is authoritative; wrap the best-effort session cleanup + reconcile so a local IndexedDB failure neither falsely reports failure nor blocks navigation away from the gone fork. Mirrors the SidebarContent fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: drop drafting-history aside from reconcileAfterWorkspaceChange comment Addresses auto-review P2: keep the refresh-before-reconcile invariant, drop the 'which they did inconsistently' narration per AGENTS.md (comments record constraints, not drafting history). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: clean up sessions on fork-id reuse + make all workspace-mutation cleanup best-effort Addresses Codex P1s: (1) CreateWorkspaceInner 'permanently delete existing fork' (id-reuse) now drops local sessions for that id so they don't resurface on the recreated fork; (2) workspace_settings archive/delete and SidebarContent child-delete loop + main delete now treat post-mutation session cleanup as best-effort, so a local IndexedDB failure can't strand the user or abort remaining deletes (matching the compare-page fix). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: make fork-reuse session cleanup fire-and-forget (non-blocking) Addresses cubic P2: don't await the best-effort cleanup so a slow IndexedDB op can't block the delete/reuse flow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: drop previous user's transient drafts on user change Addresses Pi P1: hydrateSessions preserved transient (unsent) drafts across user changes, so user A's draft + its pending fork/workspace state bled into user B's list and got reused by createSession. onUserChange now drops transients when the email changes; reconcile (intra-user) still preserves them. Regression test added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f5828780fd |
fix(backend): resolve folder_labels search_path on non-public (PG_SCHEMA) schemas (#9758)
* fix(backend): strip search_path=public from folder_labels migrations for non-public schema The folder-labels migrations (20260610151334_folder_labels, 20260614075900_dedup_folder_labels) define `folder_labels(...)` with `SET search_path = public` in their `CREATE FUNCTION` bodies. When Windmill runs in a non-public schema (PG_SCHEMA), PostgreSQL validates the function body against the `public` schema, where the `folder` table lacks the new `labels` column, failing with `column "labels" does not exist`. Add both migrations to OVERRIDDEN_MIGRATIONS, stripping the `SET search_path = public` clause so the function inherits the current search_path (which resolves the correct schema). Same regression and fix pattern as PR #5400. Fixes WIN-2093 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): pin folder_labels search_path FROM CURRENT instead of stripping it Keep the SECURITY DEFINER injection hardening while resolving the correct schema on non-public (PG_SCHEMA) installs: FROM CURRENT snapshots the migration connection's search_path at function creation time (public on normal installs, the custom schema otherwise) instead of dropping the pin and inheriting the caller's search_path at call time. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(backend): repair migration to re-pin folder_labels search_path on applied instances Instances that already applied the folder-labels migrations with the hardcoded SET search_path = public have a folder_labels function pinned to public. On a non-public (PG_SCHEMA) schema that reads the wrong folder table at runtime; the OVERRIDDEN_MIGRATIONS fix only helps instances that have not applied them yet. Add a CREATE OR REPLACE ... SET search_path FROM CURRENT migration that re-pins the function to the migration connection's schema. No-op on public installs (re-pins to public) and idempotent on already-correct ones. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
043c2c05b7 |
fix: forbid superadmin job tokens from global user and token management (#9715)
* fix: forbid superadmin job tokens from global user and token management Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: extend superadmin job token guard to offboard and export routes Apply forbid_superadmin_job_token to offboard_global_user and export_global_users, the remaining global user-management routes that were gated only by require_super_admin. Offboarding can delete a user along with their tokens, password, invites and instance-group membership, and export returns every user's password_hash, so both must be unreachable by a superadmin job token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
9e4cf139b1 |
chore(main): release 1.738.0 (#9735)
* chore(main): release 1.738.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
723a65920f |
chore(main): release 1.737.0 (#9728)
* chore(main): release 1.737.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
hugocasa
|
2879cbb65a |
feat(apps): opt-in sandbox isolation for published & raw apps (alpha) (#9420)
* feat(apps): sandbox published & raw apps with a scoped embed token Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: point ee-repo-ref at embed-token EE commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): allow top-navigation from the sandboxed app iframe Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): share app localStorage across apps via the embedder Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): publisher disable-sandbox option with per-version viewer consent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(sqlx): cache for disable-sandbox queries Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump ee-repo-ref to disable-sandbox EE commit Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): always sandbox the served raw-app wrapper + viewer fixes The raw-app wrapper served by get_raw_app_data now always carries `CSP: sandbox`. The publisher "disable sandbox isolation" opt-out is applied entirely on the viewer side, which (after per-version consent) builds its own same-origin blob wrapper — so the backend-served document stays isolated regardless of how it is reached, never via a relaxed real-origin URL. Also: - CORS on the global /apps_u mount so the opaque viewer can load custom-path public apps cross-origin. - Reject runnable-bridge messages unconditionally until the iframe is bound. - Relay the viewer's in-app hash up to the embedder address bar so deep links stay shareable (hash only; embedder keeps its own pathname). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): render public raw apps single-iframe (drop embed token) Public raw apps now render directly on the real origin with a single opaque bundle iframe and the page credential, instead of the opaque viewer + scoped-token indirection. The author bundle stays isolated in its own opaque iframe (CSP-sandboxed); low-code apps, whose code runs in the viewer frame, keep the opaque viewer + scoped token. embed_token now reports raw_app and skips minting a token for raw apps; the access check still gates visibility. Also set disable_sandbox: None in the remaining Policy constructors so the full feature build (all_sqlx_features, enterprise, license) compiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: bump ee-repo-ref to single-iframe raw-app EE commit Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): grandfather existing apps as legacy-unsandboxed + authed-only consent Existing apps are stamped by migration as `legacy_unsandboxed` so they keep running same-origin on upgrade — no breakage and no consent prompt. New apps are sandboxed by default; re-deploying an app clears the flag. The publisher `disable_sandbox` consent prompt is now shown only to authenticated viewers — an anonymous viewer has no session to expose, so the prompt was meaningless friction. embed_token reports `legacy_unsandboxed` and `authed`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: bump ee-repo-ref to legacy-unsandboxed EE commit Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): deploy-time migration prompt for legacy-unsandboxed apps On the first re-deploy of a grandfathered (legacy-unsandboxed) app, the publisher must explicitly choose: enable sandbox isolation (the flag is cleared → the app becomes sandboxed) or keep running without isolation (→ disable_sandbox, with per-version viewer consent). updatePolicy() no longer carries the legacy flag through a deploy, so the choice is what sticks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(apps): disable the sandbox-isolation toggle until the app is deployed The Deploy-drawer "Disable sandbox isolation" toggle called setPublishState() — which updates the app by path — even before the app was first deployed, when the path is empty, throwing an error. Guard it with disabled={!savedApp}, matching the adjacent visibility toggle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(apps): sandbox the in-workspace low-code app viewer in an opaque iframe Extend the opaque-origin iframe isolation to the logged-in /apps/get viewer. /apps/get becomes an embedder that keeps the workspace chrome + Edit button and renders the app inside a cookieless, chrome-less /app_embed viewer route, handed a scoped embed token minted from the member's session. The app frame runs in an opaque origin (no allow-same-origin), so it cannot reach the member's session cookie or window.parent. - apps.rs: get_app_embed_token_for_path (authed, by-path, scope + RLS gated); mint_app_embed_token grants a path-scoped apps:read:{path} so the viewer can load its own app definition and no other - lib.rs: CORS on /apps (bearer-token only, no cookies) for the opaque viewer's by-path reads - new /app_embed/[workspace]/[...path] viewer route (private analog of /public) - PublicAppFrame: viewerUrl prop to point the opaque iframe at the viewer route Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): unify in-workspace app viewers on the shared sandboxed path Route every in-workspace app display (low-code and raw) through the same PublicAppFrame -> PublicApp machinery as the public viewer, so the sandbox / legacy-unsandboxed / disable-sandbox-consent behavior is identical on every page. - new InWorkspaceAppViewer renders both app types via PublicAppFrame; /apps/get and /apps_raw/get become thin wrappers over it - /apps_raw/get previously rendered RawAppPreview directly (always isolated, with no legacy-grandfathering or consent handling); now consistent with the rest - retire the legacy same-origin raw viewer /apps/get_raw/[version] and re-point the apps-list row to /apps_raw/get; remove the dead /apps_raw/[ws]/[version] route - load the raw bundle secret in the shared viewer (getAppByPath doesn't return it) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): address PR review feedback (scope + policy hardening, nits) - require handler-level apps:read on list_apps / list_search_apps so a scoped embed token cannot read app definitions through the list endpoints. The route layer treats apps:run as satisfying read; the handler check (which does not) closes the gap. - treat legacy_unsandboxed as backend-owned: strip any client-provided value in create/update so it can only be set by the grandfather migration, not the API. - document mint_app_embed_token's caller-verifies-access contract. - use Button's declared onClick prop for the consent action (was onclick, which fell into the rest-spread and bypassed the component's click handling). - test: lock that the embed scopes cannot satisfy domain-level apps:read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): document embed-token endpoints in openapi + fix doc nit Second-round review nits: - add the three app embed-token endpoints (apps/embed_token/p/{path}, apps_u/embed_token/{secret}, and the EE apps_u/embed_token_by_custom_path) plus the EmbedTokenResponse schema to openapi.yaml; note .html on get_data - mint_app_embed_token doc: "Both" -> "All" (it lists three call sites) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): bound embed-token scopes to the caller's own The embed-token mint now enforces ensure_scopes_within_caller, so the minted scope set is always within the calling credential's own scopes (a no-op for regular unscoped sessions). Adds a unit test locking the boundary and documents the contract on mint_app_embed_token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): raw-app ctx in external embeds + page credential in direct render - RawAppPreview: engage the storage relay only in opaque frames (probe Web Storage instead of just window.parent), so a public raw app embedded in an external iframe hydrates ctx/storage directly; add a relay-timeout fallback so an unresponsive parent can never stall the ctx handshake. - PublicAppFrame: in direct render, expose the page's own bearer credential through the AuthToken context (JWT public URLs), matching the previous route behavior; opaque-viewer mode keeps the embed token. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): sandbox isolation UI polish + COI embed support for raw apps - Deploy drawer: move the sandbox toggle out of "Public URL" into its own "Sandbox isolation" section (the setting applies to every viewing surface, not just the public URL), with positive phrasing, visible helper text, and state-aware alerts (warning when disabled, info for pre-isolation apps). Toggling it now toasts its own message instead of the login-mode one. - Extract the deploy-time migration prompt into a shared LegacySandboxMigrationModal built on the common Modal component, and wire it into the raw app editor header too (it previously had no prompt, so re-deploying a pre-isolation raw app silently changed behavior). updateRawAppPolicy now also drops the backend-owned legacy flag, matching the low-code updatePolicy. - Viewer consent prompt: use the common ConfirmationModal and show the app path (new appPath prop) instead of the route pathname, falling back to "this app" when the path isn't known yet. - COI embeds: propagate the wm_coep opt-in to the raw-app wrapper document and have the backend assert COEP require-corp on it when the flag is present — required for the bundle iframe to load when the public app page is embedded inside a cross-origin-isolated page. Previously this only worked in dev because the Vite proxy injects the header; the production response lacked it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): app navigation parity across sandboxed and direct viewers - Navbar component: same-app items relay query + hash to the embedder page (which mirrors them onto the root URL, keeping its own pathname and transport params), app items navigate the top page through a validated wm_embed_navigate relay instead of the cookieless viewer iframe, and external items keep opening a new tab. Selected-item detection now recognizes the /app_embed viewer route and ignores transport params. - Frontend-script `goto` and button `onSuccess: gotoUrl`: same-window navigation goes through a shared appNavigateSameWindow helper that relays to the embedder inside the opaque viewer (same-origin paths SPA-navigate, http(s) URLs do a full load, other schemes rejected) and keeps plain window.location everywhere else. - /apps/get and /apps_raw/get: key the viewer by workspace/path so in-route navigation fully remounts it — previously the URL changed but the app (and in sandbox mode its path-scoped token) did not follow. - wm_embed/wm_embedder_origin added to the reserved query params so they no longer leak into the app's ctx.query. - Raw apps: drop the sandbox attribute entirely for the unsandboxed (grandfathered/consented) blob path, matching the pre-isolation viewer exactly — the attribute added no isolation there and sandboxed popups (e.g. OAuth flows). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): preserve grandfathered policy across updates + in-workspace viewer parity Round of compatibility hardening so pre-existing apps behave exactly as before on every surface: - `legacy_unsandboxed` is now preserved across app updates unless the payload explicitly clears it (`false`, sent by the editor's migration prompt and the sandbox toggle). Unrelated update paths — CLI / git-sync redeploys, publish-mode toggles, cross-workspace promotion — no longer silently drop the grandfathering. Clients still can never SET the flag. - The embed-token endpoints (secret, path, EE custom-path) read only the sandbox-decision policy fields, leniently, and no longer mint a token for raw / legacy / disable_sandbox renders: the token is only consumed by the sandboxed low-code render, and minting for the others wrote a useless token row per view and could fail the render for scope-restricted callers. - In-workspace viewer parity with the pre-sandbox `/apps/get`: new `inWorkspace` mode on PublicApp (no "Powered by Windmill" badge / user overlay, no HTML-result approval gate, column flex wrapper, `hideRefreshBar` honored again), and the page's query/hash are forwarded into the opaque viewer so `ctx.query` / `ctx.hash` reach the app. - Raw apps: `window.ctx` is always `{ctx, workspace}` again (anonymous viewers of pre-existing bundles rely on `ctx.workspace`), and the runnable bridge's job-id scoping now applies only to sandboxed renders (`gateJobIds`) — an unsandboxed bundle holds the same credential as the bridge, so gating there only broke pre-existing apps polling persisted or runnable-returned job ids. - Document `disable_sandbox` / `legacy_unsandboxed` in the openapi Policy schema; add a unit test for the lenient policy read. - bump ee-repo-ref to the matching EE commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): keep share-link viewer credentials out of the isolated app context The JWT path segment of authenticated share URLs is an embedder-side credential, consumed only to mint the scoped embed token. Two transport channels still copied it into the isolated frame where app-authored code runs: - the opaque viewer iframe src defaulted to window.location.href — the public and custom-path routes now pass a sanitized viewerUrl (JWT segment stripped, query/hash preserved, captured once so the hash relay does not reload the iframe); - document.referrer on the same-origin iframe navigation carried the full embedder URL — both app iframes now set referrerpolicy="no-referrer" (sandboxed renders only for the raw bundle iframe, keeping exact legacy parity; nothing reads the referrer). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(frontend): drop unused import inherited from main merge `slide` import in AssistantMessage.svelte (from #9539) turns `npm run check` red on this branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): redirect the removed raw-app viewer path to the unified viewer The old same-origin raw-app viewer route (/apps/get_raw/{version}/{path}) was removed in favor of the sandboxed unified viewer. Re-add a thin client route at the old path that redirects stale bookmarks to /apps_raw/get/{path}, preserving query + hash (the pinned version is dropped — the unified viewer shows latest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): narrow embed-token scopes and base consent on browser session - Embed token: resource access is metadata-only (list/type/exists) via a `resources:run` marker — resource values (get/get_value/get_value_interpolated/ list_search) are no longer reachable. Job reads are by-id only: an `app_embed` sentinel blocks the workspace-wide job enumeration/export routes (jobs/list, list_filtered_uuids, queue/list, completed/list, queue/export) while by-id result polling keeps working. - disable_sandbox consent now gates on whether the browser holds any Windmill session (cookie-only whoami) rather than workspace-scoped auth, so a viewer logged into a different workspace is still prompted before a same-origin render. - db-explorer: resolve the MySQL database name server-side (the metadata query already falls back to DATABASE()) instead of reading the resource value client-side; getTablesByResource derives the default db from the schema. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(apps): trim embed-scope and consent comments Reduce duplication — state the resource/job route exclusions and the workspace-session-vs-cookie rationale once at their source and reference them elsewhere; drop contrast/justification phrasing. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): make app sandbox isolation opt-in (alpha) Replace the disable_sandbox + legacy_unsandboxed policy pair and the per-version viewer consent with a single positive `sandbox` opt-in flag. Apps are unsandboxed by default (same-origin, full session — the pre-isolation behavior), so existing apps are unchanged and no migration is needed. Publishers opt an app into isolation from the deploy drawer, flagged alpha. - Policy.sandbox: Option<bool>; EmbedTokenResponse -> {token, expiration, raw_app, sandbox}; mint an embed token only for sandboxed low-code apps. - Drop the legacy-unsandboxed migration and the deploy-time migration prompt; remove the consent modal and the browser-session probe. - Deploy drawer: a single "Sandbox isolation" toggle (alpha), off by default, shared by the low-code and raw editors. - Bump ee-repo-ref to the companion EE commit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed token to its intended user/folder/job routes The embed token's broad read scopes spanned whole domains while the matching routers are CORS-enabled for the opaque app iframe: - users:read / folders:read were domain-wide, so the token could reach users/list, users/list_usage, users/username_to_email/*, folders/list, etc. Restrict to an app_embed-sentinel allowlist: only users/whoami and folders/listnames; deny the rest of those domains. - jobs:read allowed jobs/completed/export, missed by the job denylist. Add it alongside jobs/queue/export. Extend the embed-scope allow/deny test matrix to cover all of these. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): align sandbox comments with the opt-in model The consent prompt, deploy-time migration, and legacy-unsandboxed grandfathering were removed when sandbox isolation became an opt-in policy flag; update the comments that still described them so they match the two-state (default-unsandboxed / opt-in-sandboxed) reality. Comments only, no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed-token job reads to runs the app launched App component jobs are stamped `created_by = the viewer`, so an embed token reads its own runs via the launched-by-viewer fast path. The token then also inherited the viewer's broader job access (share links, folder ACLs, admin RLS), letting user-authored app JS reuse it to read unrelated jobs by id. Stop embed tokens at the fast path: only jobs the viewer launched, never those merely visible to them. Return NotFound so the untrusted app can't probe existence. Regression test: an embed token reads its own launched job but is denied the foreign job (result/logs/getupdate) an admin viewer's normal token can read. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): allowlist embed-token apps/jobs routes + scope run to the app The embed token's apps:run/jobs:read reached more than a running app needs. Replace the job denylist with strict per-domain allowlists on the app_embed sentinel: - Apps: only the app's own definition (apps/get/p/<path>) and the public app-serving endpoints (apps_u/*). Denies workspace app inventory (exists, custom_path_exists, list, list_paths*). - Jobs: only the by-id poll routes the frontend JobLoader uses. Denies job counts and the job_signature/resume_urls capability-minting routes (the by-id reads remain confined to the app's own runs). Drop unqualified apps:run from APP_EMBED_SCOPES; mint apps:run:<path> instead and authorize apps:run:<requested path> first in execute_component, so the token can only run its own app's components, not another app's. Extend the embed-scope route matrix and add a path-scoped run unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(apps): clarify the sandbox toggle vs the on-behalf-of model The deploy-drawer sandbox copy leaned on "session" in a way that collided with the on-behalf-of permissioning right above it. Reword it to say the toggle governs what the app's browser-side code can reach in the viewer's browser — distinct from who its runnables execute as — and rename the label to "Isolate the app from the viewer's browser session". Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): path-scope embed-token S3 download to its own app The apps_u/* allowlist also admitted apps_u/download_s3_file/<path>, whose handler authorized any authenticated caller — so an embed token minted for app A could download app B's S3 files via B's on-behalf policy. Add the same path-scoped guard execute_component uses: download_s3_file_from_app now checks apps:read:<path> first, confining the token to its own app. Other path-taking apps_u routes are already covered (writes lack apps:write; embed_token/p path-checks; public_resource is type-constrained). Extend the path-scoping unit test to cover apps:read (download) alongside apps:run (execute). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): path-scope public-app-by-secret read to the embed token's app The apps_u/* allowlist admitted apps_u/public_app/<secret>, whose handler only checked the viewer's read access — so an embed token minted for app A could read app B's definition by secret (confused deputy via the viewer's identity). get_public_app_by_secret now binds a scoped caller to the resolved app with check_scopes(apps:read:<path>), confining it to its own app; unscoped sessions and anonymous access are unchanged. get_raw_app_data needs no binding (pure secret capability, no caller identity). Document the full set of app-resolving handlers the path-scoped read covers. Bump ee-repo-ref for the companion custom-path fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): preserve pre-sandbox behavior for db-explorer, edit link, jwt Three behavior-parity fixes for non-sandboxed (existing) apps that the sandbox-isolation refactor changed incidentally: - DB-explorer MySQL table picker: when the connection can see multiple non-system schemas, label the default db's tables unprefixed again. The resource-value read was removed globally, so identify the default db from the introspection script's `DATABASE() AS default_db_name` (carried on SQLSchema.defaultDb) instead of guessing "the single schema key". Equivalent to the prior resource.database match; editor-only (table picker). - In-workspace Edit button: restore `?nodraft=true` on both /apps/get and /apps_raw/get, so opening the editor from the viewer loads the deployed version, not a draft. - Custom-path (/a) viewer: restore the "could not authenticate user with jwt token" toast when a path JWT fails to resolve a user, instead of silently falling through. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): confine embed-token S3 downloads to the app's own keys/outputs download_s3_file_from_app authorized any authenticated caller for any S3 key (opt_authed.is_some() bypass). A sandboxed app's embed token carries the viewer's identity, so app-authored JS could fetch arbitrary S3 keys readable by the on-behalf identity, beyond the app's own declared keys or outputs. Route app embed tokens through the same allowlist as anonymous viewers — the app's declared allowed_s3_keys, or files produced by this app's own component runs — instead of the authed bypass. The produced-files check is parameterized by created_by (the embed viewer for a token, else anonymous) so a sandboxed app's own S3 outputs still render while arbitrary keys are denied. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): let embed tokens cancel their own jobs; gate cancel to launcher A sandboxed low-code app supersedes an in-flight component run on re-run by canceling it, but the embed token only had jobs:read, so cancellation silently failed and prior jobs ran to completion. - Permit the by-id jobs_u/queue/cancel POST for app_embed tokens at the route layer (the only write reachable through the existing by-id allowlist). - Gate cancel_job_api: an app_embed token may cancel ONLY jobs it launched (created_by == viewer). cancel_job_api had no other per-job ownership check, so this also confines the token instead of letting it cancel any job by id. - /app_embed now sets workspaceStore so cancellation targets the right workspace instead of an empty/stale one in the cookieless iframe. Add a shared has_app_embed_sentinel helper; cover cancel in the route matrix and the jobs_read_auth integration test (own job cancelable, foreign denied). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): drop get_root_job_id from the embed-token job allowlist Audit of the embed token's reachable job routes: get_root_job (jobs_u/ get_root_job_id) has no access check in its handler at all — it returns any job's root-job id by id — and the app runtime never calls it. Remove it from the by-id allowlist so the embed token can't probe a foreign job's flow lineage; add a denied-route assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(apps): scope sandboxed-app localStorage per app Sandboxed apps shared one localStorage store (one key on the real origin), so an app could read or clobber another app's keys — and, with job ids stashed there, reuse its embed token to read another app's job. Scope the backing store per app. The embed-token endpoints now return the resolved app_path (EmbedTokenResponse; not a new disclosure — the viewer already receives the path when it loads the app). PublicAppFrame (low-code) and RawAppPreview (raw) key their backing store by it: wm_apps_localstorage:<app_path>. Same app shares one store across its public and in-workspace surfaces; different apps are isolated. Unsandboxed apps are unaffected (real same-origin localStorage, as before). Bump ee-repo-ref for the companion custom-path change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): scope embed access checks to embed tokens + key app storage by workspace - Apply the path-scoped read/run checks on the public-by-secret read and the component run path only when the caller is an app embed token, so other caller types keep their prior access. - Key the sandboxed app's backing client storage by workspace + path instead of path alone, and return the resolved workspace from the embed-token endpoints so the custom-path viewer can derive it. - Show a clear message instead of an indefinite loader when the viewer route is opened outside its embedder. Bumps ee-repo-ref to 5b8476b. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(apps): mint embed tokens only from the trusted embedder caller An app embed token must not reach the embed-token mint endpoints; refresh minting stays with the embedder session/JWT. Enforced at the scope route layer and at the mint chokepoint, with a route-matrix regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(apps): support S3 upload and frontend-script S3 download in sandboxed apps Sandboxed apps run with a scoped embed token (no cookie). Let the app's S3 file-input upload and the frontend-script download({s3}) helper work in that context: upload is reachable with apps:run and re-checked per-app at the handler; the script download routes through the app-scoped apps_u endpoint with the embed token instead of the cookie-authed job_helpers path. Default (unsandboxed) apps are unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: update ee-repo-ref to b0cb761bf9852974e571b2978032d310cc998517 This commit updates the EE repository reference after PR #600 was merged in windmill-ee-private. Previous ee-repo-ref: e673c714a4618fdb72353a475f49c748e6016642 New ee-repo-ref: b0cb761bf9852974e571b2978032d310cc998517 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
e82a6a6830 |
chore(main): release 1.736.0 (#9720)
* chore(main): release 1.736.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |