mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-21 08:02:26 +00:00
afe025b886ffbc45dab086d4b167030263fd39b5
6721 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Diego Imbert
|
072f2bf053 |
chore(datatable-migrations): remove unused update_datatable_migrations endpoint
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
f8c216d301 | docs: fix generate_initial migration ordering comment to match code | ||
|
Diego Imbert
|
91021816e7 |
fix(datatable-migrations): serialize run/rollback with a per-db advisory lock
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
2113e87495 |
Merge remote-tracking branch 'origin/main' into change-68b704f7
# Conflicts: # backend/ee-repo-ref.txt # frontend/src/lib/components/CompareWorkspaces.svelte |
||
|
Diego Imbert
|
154f31a054 | clone migrations on fork | ||
 hugocasa
|
5c521d808a |
fix(forks): clone only the current raw-app bundle, via server-side copy (#9899)
* fix(forks): clone only the current raw-app bundle, via server-side copy Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(forks): fall back to get+put when object-store copy is unsupported Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
c3bb639624 |
feat(datatable-migrations): allow non-admins to create/run/revert migrations, gate only opt in/out
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Ruben Fiszel
|
fad5419b9d |
chore(main): release 1.746.0 (#9872)
* chore(main): release 1.746.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
84141add1d |
feat(pipelines): workspace duckdb macro libraries (// macros / // use) (#9890)
* feat(pipelines): parse duckdb macro-library annotations (// macros, // use) * feat(pipelines): duckdb macro registry tables + deploy-path validation and writes * feat(pipelines): inject workspace duckdb macros into consumer jobs at run time * feat(pipelines): surface macro libraries and lib-consumer edges in asset graph api * feat(frontend): macro-library nodes, lib-consumer edges and scaffold in pipeline graph * docs: mark dbt gap #7 (packages/macros) shipped via workspace macro libraries * fix(pipelines): review fixes - char-safe parsing, local macros win, fork clone, trust-model docs * feat(frontend): duckdb macro autocomplete + workspace macro explorer drawer * fix(pipelines): address CI review - use-setup retention, splice past local defs, orphan filter, full consumer rescan, index-keyed strip * fix(pipelines): inject provider library setup for implicitly-called macros too * fix(pipelines): rls-gate macro listing + honor library-level // use transitively * fix(pipelines): weave injected macros around local definitions by bind order * fix(pipelines): injected library setup always runs before user blocks * perf(pipelines): cache macro registry per workspace with notify-event invalidation * perf(pipelines): disable macro registry cache on cloud |
||
|
Ruben Fiszel
|
af01e90b5c |
feat(s3): replace CE 50MB upload cap with 10GiB workspace storage quota (#9874)
* fix(s3_proxy): enforce CE 50MB upload cap on multipart uploads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): replace CE 50MB upload cap with 10GiB workspace storage quota Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): gate CE quota OSS stubs to not(enterprise) to match callers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): delta-aware CE storage quota + guard usage-load retry loop Account for the overwritten object's size in the quota check so valid same-size overwrites near quota are not rejected (Codex review), and stop the storage-usage $effect from re-firing on persistent API errors (Pi review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): count chunked PUTs; revert overreaching volume quota copy Volumes write to workspace storage via a separate worker-side path with its own 50MB-per-file cap that this PR does not change, so revert the drawer copy that claimed they count toward the 10GiB quota (Codex review). Bump ee-repo-ref for the chunked-PUT accounting fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): add SQLx cache for CE usage-bump/quota queries; exclude volumes Regenerate the missing offline SQLx cache for the not(enterprise) bump and remaining-quota queries so the private CE offline build compiles, and bump ee-repo-ref for the volumes/-prefix exclusion from the counted quota (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): always HEAD for CE upload delta so overwrites don't inflate usage Bump ee-repo-ref for the fast-path overwrite-accounting fix (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): reserve volumes/ prefix on CE write surfaces to close quota bypass Reject direct writes to the reserved volume prefix on the app-upload surface and add the OSS stub; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): refuse new multipart work when CE workspace is at quota Bump ee-repo-ref for the multipart-initiate/part quota gate (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(s3): reserve in-flight multipart bytes against CE storage quota Add workspace_multipart_inflight table + grants, SQLx cache for the reservation queries, and bump ee-repo-ref. Bounds abandoned multipart uploads that the list-based recount can't see (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): clear multipart reservation only after a successful complete Add exclude-upload arg to the OSS quota stub/caller and the SQLx cache for the updated remaining-quota query; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(s3): per-part multipart reservation; commit only on part success Per-part workspace_multipart_inflight schema (upload_id, part_id) so retries replace rather than double-count; SQLx cache for the reworked queries; bump ee-repo-ref (Codex review). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * perf(s3): HEAD the multipart overwrite target once per upload, not per part SQLx cache for the stored-credit lookup; bump ee-repo-ref. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore: update ee-repo-ref to bea5a8b5120d6d69cab1ad4611ebe463559bd200 This commit updates the EE repository reference after PR #640 was merged in windmill-ee-private. Previous ee-repo-ref: 6e6ff86f1939cf74736b7d435bf6851416437523 New ee-repo-ref: bea5a8b5120d6d69cab1ad4611ebe463559bd200 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
7c7d7474cc |
feat: support workspace forks on cloud using parent workspace limits (#9864)
* feat: support workspace forks on cloud using parent workspace limits Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: clarify count_paid_seats approximates rather than mirrors billing seats Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: non-admin fork UI, attach cap, and fork-count for cloud forks Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: cloud fork billing cache on rename, usage display, attach cap edge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: fork count in cloud quotas + fork billing points to parent Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate billing/fork caches on fork deletion for id reuse Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: gate fork usage remap on CLOUD_HOSTED, not just the cloud feature Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: note cloud feature vs CLOUD_HOSTED gating in backend guide Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: reserve fork-cap slots for an attach candidate's whole subtree Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate team-plan cache on delete, raise fork depth cap Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: cap fork nesting depth (MAX_FORK_DEPTH, default 5) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: fork count/height robust to cycles and deleted intermediates Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(frontend): reset fork button loading state on creation error Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: invalidate billing cache for attached fork subtree; helper auth docs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
d15033cde6 |
fix: invalidate bun bundle cache on transitive relative-import changes (#9891)
* fix: invalidate bun bundle cache on transitive relative-import changes Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: do not memoize transient fetch errors in bundle-key import cache Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use regular comment on lazy_static block (deny unused_doc_comments) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: align bundle-key import version selection with loader content endpoint Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
hugocasa
|
d9b080f57f |
feat(ai): add Azure AI Foundry as a native AI provider (#9879)
* feat(ai): add Azure AI Foundry as a native AI provider Adds `azure_foundry` as a new AIProvider variant wired through the AI chat (copilot) and AI agent flow steps. Foundry's chat completions API is OpenAI-compatible and uses Azure conventions (api-key header, Azure URL building), so it reuses the existing OpenAI-compatible query builder and proxy path via the shared `is_azure` helper (renamed from `is_azure_openai`). Backend (windmill-ai): - New `AzureFoundry` enum variant (serde `azure_foundry`) - `get_base_url` requires a resource base URL (like Azure OpenAI / Custom) - `is_azure()` covers Azure OpenAI + Foundry (api-key auth, Azure URL) - Added to OpenAI-compatible proxy support and HttpForward proxy mode - New proxy URL unit test Frontend (copilot): - New provider entry, completion config, model-token handling, streamed usage tracking, and reasoning registry (all model-id-gated, so a no-op for Foundry's non-OpenAI catalog) - Treated as a chat-completions provider, not the OpenAI Responses API OpenAPI: - `azure_foundry` added to AIProvider (openapi.yaml) and AIProviderKind (openflow.openapi.yaml); regenerated CLI guidance Note: the `azure_foundry` resource type (base_url + optional api_key) is hub-managed and must be published to the Windmill Hub separately. Fixes WIN-2122 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai): add azure_foundry to copilot flow Zod provider enum The tracked copilot flow schema (openFlowZod.gen.ts and its openFlow.json source) still carried the old AIProvider enum, so validateFlowModules / validateSpecialFlowModule rejected AI-generated flow edits that create or update an aiagent module with provider kind "azure_foundry" before they could be saved. Add the value to both (preserving the generated single-line format) and a regression test over the flow-module validation path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): lead provider list with OpenAI, Anthropic, Google AI Reorder AI_PROVIDERS so the three primary direct providers come first. The AIProviderPicker renders the first three entries as quick-access buttons, so these become the defaults (previously OpenAI, Azure OpenAI, Azure Foundry); Azure OpenAI / Azure Foundry stay adjacent right after. No logic depends on provider order (only per-provider defaultModels[0] is read). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
74a7c843e7 |
refactor: move datatable migration rename/delete cascade into module
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 Alexander Petric
|
789450731b |
fix(triggers): retry transient websocket connect failures before disabling (#9887)
A websocket trigger is permanently disabled (with a critical alert) when a single connect attempt in get_consumer fails. Gateway endpoints fronted by edge proxies (e.g. Discord behind Cloudflare) sporadically answer the upgrade handshake with a transient 502/520, so a long-lived trigger that reconnects frequently eventually catches one and dies until a human re-enables it. Retry transient failures (HTTP 5xx/429 handshake responses and IO errors) up to 5 attempts with exponential backoff before surfacing the error, and report retry progress through the trigger's error status. Permanent-looking errors (bad URL, other 4xx, protocol/TLS mismatch) still disable immediately. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5745dfc6ea | smooth local pipeline dogfooding (#9888) | ||
|
Ruben Fiszel
|
53bbb92953 |
feat(pipeline): backfill a range of partitions from the asset drawer (#9885)
* feat(pipeline): backfill a range of partitions from the asset drawer (ee) * feat(pipeline): cancel in-flight backfill job and show cancelling state * refactor(pipeline): move backfill range logic behind private feature * fix(pipeline): close backfill cancel-launch race and record dispatch intent * docs(openapi): producer_path also covers SDK write-edge producers * chore: update ee-repo-ref to c3852ecb36bd0be1a74c63169e513888f3347850 This commit updates the EE repository reference after PR #641 was merged in windmill-ee-private. Previous ee-repo-ref: 7c1450ef89fbc9e844a121b39cafe0d7235d704b New ee-repo-ref: c3852ecb36bd0be1a74c63169e513888f3347850 Automated by sync-ee-ref workflow. --------- Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
d65f58c388 |
fix: pipeline dogfooding fixes — SCD2 data-test scope, --partition, s3object upload binding (#9875)
* fix: scope SCD2 built-in data tests to current rows Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: add --partition to pipeline run and fix duckdb s3object upload binding Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: note filesystem storage type is dev-only in storage settings Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: use ISO week for weekly partition default in pipeline run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
b92a86b8b3 |
fix: polish pipeline graph view (layout, viewport, minimap, lineage, timestamps) (#9883)
* fix(frontend): keep pipeline graph layered when lineage has cycles Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): fit pipeline graph to visible canvas on initial load Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): style pipeline minimap so it reads as a minimap Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(parser): don't infer s3 reads from bare string-literal mentions in sql Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(duckdb): render temporal values as ISO strings in job results Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(frontend): key pipeline viewport fit on the loaded graph's folder Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(parser): treat list/named read-fn arguments as definitive s3 reads Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
659642e488 | fix(parsers): infer py s3 assets from S3Object constructor and dict forms (#9877) | ||
|
Ruben Fiszel
|
b883adbc00 |
fix(duckdb): auto-declare partition arg for // partitioned scripts (#9878)
* fix(duckdb): auto-declare the partition arg for // partitioned scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): pipeline run --arg to pass plain run args to cascade scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
e77b7523a5 | nit enterprise implies license feature | ||
|
Ruben Fiszel
|
9a24cd2bef |
chore(main): release 1.745.0 (#9858)
* chore(main): release 1.745.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
1a9debb689 |
fix(jobs): give flow dynselect a path and its worker tag, like scripts (#9867)
Fetching options for a `dynselect`/`dynmultiselect` input was inconsistent between deployed scripts and deployed flows: - scripts ran through `push_script_job_by_path_into_queue` — a `script` job with the script's path, tag, lock and codebase resolution; - flows ran their schema dyn-select code as an anonymous `preview` with no path and no tag (always the language default), and reported access failures as a raw `SqlErr: no rows`. Deployed scripts are left exactly as they were (that path already handles tag/lock/codebase/on-behalf-of correctly). The flow branch now: - carries the flow path on the preview job, - reads the flow's `tag` under RLS and routes the job to it (falling back to the language default when unset), matching the script's worker group, and - runs `check_tag_available_for_workspace` on that tag — the same gate a normal flow run and the script path apply — so a caller who can read the flow but is not allowed to use its (custom/scoped) worker tag is rejected consistently. The flow's tag read runs on every request, so it also serves as the per-request access check, replacing the raw error with a clean `NotAuthorized` / `NotFound`. Entrypoint-name validation now covers all branches (it is interpolated into the generated wrapper). Inline is unchanged: a `preview` with no path on the language default, blocked for operators. Fixes WIN-2118 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
20cd1a02d5 |
feat(forks): partial-visibility deploy + surface hidden items (#9868)
* fix(forks): let partial-visibility users deploy the visible subset The fork Compare & Deploy page hid the deploy button entirely whenever the comparison reported any item not visible to the user (all_ahead/all_behind flags), telling them to hand the deploy to someone with full access. But the non-visible items are already filtered out of the diff list, and the UI already supports deploying an arbitrary subset via per-item selection — so blocking everything was inconsistent and, for stale/phantom rows, blocked on items that don't even exist. Show the deploy footer regardless; the user acts on the visible/selected items (the per-item disabled conditions are unchanged). The hidden-items notice is kept but downgraded to a non-blocking, direction-scoped banner that explains the excluded items instead of removing the action. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(forks): surface hidden-item counts by kind + admin path list WIP: expose items dropped by the visibility filter (hidden_ahead/hidden_behind in the compare response): by-kind counts for everyone, kind+path only for admins. * fix(forks): don't close the deployment request on a partial (hidden-items) deploy Making the deploy button reachable in the partial-visibility case exposed a bug: a clean merge-into-parent deploy unconditionally closed any open fork deployment request as "merged" — marking its comments obsolete and notifying the requester and assignees of a merge — even when hidden ahead changes were excluded from the list and left undeployed. Only close the request as merged when the full ahead set was visible (all_ahead_items_visible); otherwise leave it open (with a toast) so someone with full access can finish it. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
7363d2c217 |
fix(forks): require admin of both sides for the compare visibility guard (#9869)
The blast-radius guard added in #9866 forced `all_ahead_items_visible` true for any fork/target admin. But `filter_visible_diffs` keeps a modified/conflict row (one that exists in the source AND the fork) only when the caller can see it on both sides, so an ahead diff can be dropped for a source-side visibility gap even when the caller is a fork admin. Forcing the flag on fork-admin alone then wrongly reported "all ahead items visible", letting the UI enable deployment from an incomplete comparison. Gate the guard on admin of BOTH the source and the fork (superadmin satisfies both), which is what actually guarantees full visibility of every item on every side. Adds a regression test where a fork admin who is only a plain member of the parent (no access to the item's folder) must still get `all_ahead_items_visible = false`, plus the superadmin sanity path. Also restores the SQLx offline cache entry for the phantom-trigger test INSERT that #9866 landed without (CI/`SQLX_OFFLINE=true` builds failed on it), and adds entries for the new test's all-literal queries. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
6a6f12960e |
fix(forks): reset diff tally on trigger delete + guard compare visibility for admins (#9866)
Deleting a trigger left a stale `workspace_diff` row: `delete_trigger` (the generic TriggerCrud handler) was the only delete path that never called `handle_deployment_metadata`, unlike every other kind. Because `compare_workspaces` trusts a cached `has_changes=true` row for non-script/flow kinds and the visibility filter then drops it (the trigger no longer exists), a deleted trigger became a phantom "ahead" item that flipped `all_ahead_items_visible` to false — hiding the deploy button and showing a "changes not visible to your user" warning that even a superadmin could not resolve (`reset_diff_tally` doesn't clear a `has_changes=true` row either). - delete_trigger now re-tallies via handle_deployment_metadata, so the next compare re-evaluates and corrects/removes the row (matches resource/variable/ folder/schedule deletes). - compare_workspaces forces the visibility flags true per side for anyone who sees that side in full: target/fork admin (or superadmin) for ahead items, source/parent admin (or superadmin) for behind items. The flag is a pure visibility guarantee — the deploy itself is authorized separately — so for such users a dropped diff is provably a phantom, never a permission gap. - Add a regression test asserting a phantom trigger diff row no longer blocks a superadmin while still (conservatively) warning a partial-context user. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
d7be355290 | sqlx nits | ||
|
Ruben Fiszel
|
efb62c5e99 |
enforce CE workspace limit when unarchiving (#9865)
* fix(workspaces): enforce CE workspace limit when unarchiving Unarchiving a workspace re-activates a soft-deleted (deleted = true) workspace, effectively bringing it back to the active set. On CE this bypassed the 2-workspace cap that create_workspace enforces, letting a user exceed the limit by archiving and re-unarchiving. Run the same _check_nb_of_workspaces guard before flipping deleted back to false. The workspace being restored is still deleted = true at that point, so it is correctly excluded from the count. Fixes WIN-2119 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(workspaces): cap CE archived workspaces at 1 Complements the unarchive-limit fix: without a cap on archived workspaces, a CE user could stockpile many soft-deleted workspaces (each of which still occupies its workspace id and can later be unarchived). Refuse a new archive on CE when an archived workspace already exists, mirroring the create/unarchive workspace-count guards. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
76a9523009 |
feat: use derived username instead of email for non-member superadmins (#9857)
* feat: use derived username instead of email for non-member superadmins Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: address review - drop redundant username cache, guard whoami membership by email Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor: use explicit non_member boolean instead of role string for superadmin banner Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve email from password table for non-member superadmin permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin drafts via shared username->email resolver Adds resolve_username_to_email (usr, then super_admin password fallback for both derived-username and email modes) and uses it in get_email_from_permissioned_as and the drafts get/list endpoints, so a non-member superadmin's drafts resolve and no email leaks into the drafts payload. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: superadmin-not-in-workspace schedule uses derived username as permissioned_as Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: resolve non-member superadmin identity in draft owner-circles, username_to_email, and home filter Applies the password-fallback username resolution to the script/flow/app/draft owner-circle subqueries and the username_to_email endpoint (was an admins-workspace 'username == email' hack), and switches the home items-list user-folder filter to the non_member flag instead of the now-broken username-contains-@ heuristic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: backfill non-member superadmin favorites from email to derived username Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: propagate DB errors in username resolution instead of leaking email (CI review) Addresses cubic-dev-ai P2: get_instance_username_or_fallback_to_email now returns Result and only falls back to the email for a genuine 'no derived username'; a query error propagates so callers fail closed rather than leaking the raw email as the acting username. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: clarify non-member superadmin popover (username used + admin permissions) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: keep username_to_email endpoint member-only to not disclose non-member superadmin email (CI review) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: forbid disabling automate_username_creation once usernames assigned (CI review) Makes the setting effectively one-way once instance-wide usernames exist, so the global-uniqueness invariant that keeps stored u/<username> identities (schedules/triggers/drafts/superadmin ownership) unambiguous can never be dropped back to workspace-local uniqueness. Re-saving false on an already-disabled instance stays a no-op. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
383c70523b |
fix(folders): allow dots and at-signs in folder owner validation (#9856)
Folder owners use the format u/<username>, and usernames are frequently email addresses containing `.` and `@`. The folder creation path bypasses validate_owner(), so these owners get inserted successfully, but add_owner and remove_owner both call validate_owner() and rejected any later modification of email-style owners. Extend the character allowlist to accept `.` and `@` (and update the error message). SQL injection risk is already mitigated by the bind-parameter queries introduced alongside this validation. Fixes WIN-2116 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
cfcc0b9453 |
chore(main): release 1.744.0 (#9839)
* chore(main): release 1.744.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
Ruben Fiszel
|
74f579e6d9 |
feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) (#9840)
* feat(pipeline): local development for data pipelines (CLI --local + pipeline dev preview) Add the local edit→preview→run loop for data pipelines (folders of `// pipeline` scripts), the analog of `wmill dev` / `wmill app dev`, usable from a code editor or an agentic loop — without deploying. No backend changes: full body inference comes from the same wasm the frontend uses (windmill-parser-wasm-asset), which returns assets + pipeline annotations in one call; local runs reuse runScriptPreview with _wmill_skip_asset_dispatch. - localGraph.ts: wasm-backed working-tree → asset-graph builder (the enabler) - pipeline show/run --local; new pipeline docs (PIPELINE.md/AGENTS.md) subcommand - pipeline dev watcher + /pipeline_dev page (PipelineDevView) rendering the same PipelineGraphEditor from the pushed local graph, run via preview - cascadeRun.ts: reusable run primitives extracted from the route page - regenerated CLI agent docs See docs/pipeline-local-dev.md for the full design, test steps, and handoff TODOs. The live `pipeline dev` browser preview is implemented but not yet stack-verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): improve local dev preview (run, activity, responsive) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): dev-preview args, multi-root run, ws auto-reconnect Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): connect managed-materialize producer in local dev graph The CLI pinned windmill-parser-wasm-asset ^1.728.1, which predates managed-materialize support (added in 1.733.1); the frontend already pins 1.740.0. The CLI's wasm therefore never emitted `// materialize`, so the producer had no output edge and showed disconnected from its `// on` consumers. Bump the CLI to 1.740.0 (matching the frontend) and translate the parsed materialize target into the producer's write edge + materialize_target, mirroring frontend resolveGraph.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): harden local-dev CLI (bare-.sql crash, defaultTs, docs clobber) Review fixes, complementary to the dev-preview/materialize/multi-root work already on the branch (none overlap those commits): - localGraph: a bare `.sql` (no dialect) made inferContentTypeFromFilePath throw and abort the whole graph build — and wedge `pipeline dev` at startup. Skip the unclassifiable file instead. Also map `bunnative` → parse_assets_ts and add ruby/rlang/nu/powershell to the `#`-comment fallback. - show/run/docs/dev: thread the resolved `wmill.yaml` defaultTs into the graph builder so `.ts` infers under the workspace's runtime (bun vs deno) instead of always bun — `opts.defaultTs` was always undefined (no such CLI flag). - dev: wrap the startup graph build so a half-written file can't abort the watcher. - docs: don't clobber a user-authored AGENTS.md/CLAUDE.md — only (over)write the pointer when absent or already a generated `@PIPELINE.md` pointer. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): bind dev WS to loopback + local-graph regression tests - pipeline dev WS broadcast the folder's full script source (scripts[].content + temp_script_refs) unauthenticated on 0.0.0.0:3201 — bind 127.0.0.1 so it's not LAN-reachable (webview localhost + SSH/devbox port-forward still work). - Add regression tests for the just-landed local-graph fixes: bare .sql is skipped (was a build/dev-startup crash), defaultTs threads into .ts runtime inference (bun vs deno), and #-comment languages (ruby) use the # annotation fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --frontend flag for pipeline dev page origin wmill pipeline dev opens <remote>/pipeline_dev, but that route only exists in this build's frontend, so it 404s against a remote whose deployed frontend predates it. --frontend <origin> points the page at a locally-run frontend (REMOTE=<remote> npm run dev) while the API/token still target the remote — enabling the live preview against a real backend before the PR is deployed. No behavior change when omitted. Regenerated CLI agent docs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): WS session token + details-pane live-reload refresh Addresses CI review (Codex/Pi/Claude): - dev WS: a browser tab could open ws://localhost:<port>/ws and receive the folder's full source (browsers don't enforce same-origin on WS, loopback bind alone doesn't help). Gate the upgrade on an unguessable per-session token carried in the dev-page URL (verifyClient → 401 without it). Verified: no-token/bad-token connections get 401 with no bundle. - details pane: scriptRes keyed on [workspace, selection, draftScript] didn't re-run on a pipeline dev live-reload (same selection), so the open pane showed stale source. Thread a localScriptsVersion (the pushed bundle) into the key. Verified: editing a selected node's file updates the pane source without reselect. - docs/pipeline-local-dev.md: refresh the stale 'not yet exercised' status + done TODOs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): emit volume: annotation assets in local dev graph Addresses CI review (Codex P1 / Pi P1): the wasm body parser doesn't surface `// volume: <name>` annotations — the frontend (infer.ts:parseVolumeAnnotations) and backend (asset_inference.rs) parse them separately and merge as rw volume assets. localGraph didn't, so a `# volume: cache` producer had no write edge and showed disconnected from its `// on volume://cache` consumer (and pipeline run --local wouldn't schedule downstream). Mirror the leading-comment-block scan (SQL excluded, matching both reference parsers) and merge into inferScriptAssets. Regression test added; verified producer -> volume://cache -> consumer connects. Also (Codex P2): docs/pipeline-local-dev.md manual browser URL omitted the new ws_token param — without it the WS upgrade is rejected and the page sits disconnected. Doc now says to copy the URL the CLI prints (carries wm_token + ws_token) and recommends --frontend. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): runAll excludes event roots + review polish Addresses CI review (Codex P1, Claude P2/P3): - pipeline run runAll: derive the whole-pipeline selection from validStarts + descendants instead of all runnables, so an unqualified 'pipeline run <folder>' no longer fires event-trigger roots (kafka/mqtt/…) with empty args/side effects. Verified: a kafka root is excluded from the plan. - cascadeRun.ts runBoundedCascade: use buildLineageDownstreamMap (read-aware) so a pure-reader runs after its producer, and return cyclic — parity with the route page's bounded run (the file is meant to be THE shared correct primitive). - PipelineGraphEditor: storedRightPaneSize starts at 0 so the orientation-aware default (55% stacked / 40% side-by-side) actually applies on first open. - localGraph fallbackParse (go/bash): scan only the leading comment header (no body-comment phantom triggers) and strip key=value options from the asset URI; regression test added. - docs: reject '..' in the folder arg (it writes files under f/<folder>). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): route local previews to the // tag worker Addresses CI review P1: the local graph/bundle dropped the parsed `// tag`, so a node annotated `// tag gpu` ran on the default worker in both `pipeline run --local` and `/pipeline_dev`, while the deployed pipeline routes it to that worker tag. Carry the tag through LocalScript / the pushed bundle / LocalScriptContent and pass it to runScriptPreview at all three launch sites. Verified: a duckdb node tagged `bash` produces a job tagged `bash`; regression test added. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): add asset partitions/schemas routes to OpenAPI, use generated client The ducklake asset panels (PartitionStatusGrid, SchemaHistoryPanel) hit /assets/partitions and /assets/asset_schemas via raw fetch with cookie-only auth, because those backend routes were never added to openapi.yaml so the generated client had no methods for them. On /pipeline_dev (token-via-URL, no session cookie) the raw fetches 401'd. Add both GET routes + MaterializedPartition/AssetSchemaVersion schemas to openapi.yaml and call them through AssetService, which injects the bearer token, types, and cancellation automatically. Verified: Partitions + Schema tabs load in /pipeline_dev. (backfill stays a raw fetch — it's an EE-only route not in the OSS spec — with the token added inline.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(cli): regenerate bun.lock for windmill-parser-wasm-asset package.json / package-lock.json carry windmill-parser-wasm-asset@1.740.0 but the tracked bun.lock (the CLI installs/builds/tests via bun) was stale, so fresh bun installs would resolve a different graph than the committed lock. Regenerated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): show asset producer + its runs in the dev-preview panel Selecting a ducklake/asset node in /pipeline_dev showed 'No producer for this asset' because selectionProducers wasn't passed (it's derived from the deployed graph on the route page, absent here). Compute it from the local graph's w/rw write-edges (incl. the // materialize target) and pass it through, mirroring the route page — so the panel shows the producing script and its (preview) runs, including data-test failures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): carry annotation metadata onto local-graph runnables The local graph emitted only path/usage_kind/in_pipeline/materialize_target per runnable, so /pipeline_dev and pipeline show --local weren't the same surface as the deployed graph for annotated scripts — missing the badges/lineage the shared canvas renders. Map the wasm-parsed partition_kind, freshness, tag, retry, data_tests, column_lineage, and materialize_strategy (derived append/merge/replace) onto each runnable, mirroring the deployed AssetGraphRunnableNode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude event handlers that are lineage descendants from runAll The runAll guarantee ('never fires an event handler with empty args') only held for event ROOTS — validStarts excludes them, but runAll then unions in descendants(dag, start), so a kafka/mqtt/... handler that also reads an upstream pipeline asset (a lineage descendant of a valid start) still landed in the plan. Add eventTriggerScripts() and subtract it from the selection after the descendant union. +unit test. Also: docs/pipeline-local-dev.md recipe used 'pipeline docs demo_pipeline' without --local (default queries the deployed graph → hits the empty hint); add --local. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): whole-pipeline run cuts at event handlers (drop their downstream too) The prior runAll fix subtracted event handlers from the selection but left their downstream: for manual_root → asset_x → kafka_handler → asset_y → consumer, deleting only kafka_handler left consumer selected, and topoOrder then ran it as a root with missing/stale event-derived inputs. Replace the descendant-union+delete with reachableCutting(dag, validStarts, eventHandlers): traverse from valid starts but treat event handlers as cut points, so a node reachable ONLY through an event handler is dropped while one reachable via a non-event path stays. +unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): recover // tag in the go/bash annotation fallback The wasm path carries out.tag, but the go/bash fallback (and the wasm-error degradation path) only recovered pipeline + on, so a // tag gpu on a bash/go node — or a temporarily-unparseable ts/py/sql node — silently routed the local preview to the default worker while the deployed pipeline routes to the tag. Scan for // tag in fallbackParse too. +test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): extract shared assetProducers helper The 'who writes this asset' write-edge derivation was copied verbatim in PipelineDevView and the pipeline route page — two copies that would drift. Extract assetProducers(graph, selection) into graphTraversal.ts and use it from both, keeping the dev view and route page in lockstep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): only overwrite AGENTS.md/CLAUDE.md when it's the exact generated pointer Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): wire local-dev runs into the selected-node runs pane Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): exclude data_upload/webhook entrypoints from auto CLI runs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): --upload binds an object to a data_upload/webhook entry point Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): add "Run + downstream" to the dev preview detail form Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): cut non-autorun triggers on all run paths; multi-binding --upload Address CI review: apply the data_upload/webhook/event barrier cut to the single-root and bounded (--from/--to) paths, not just whole-pipeline; accumulate repeatable --upload bindings per script (were overwritten); scope dev upload keys by script+param to avoid basename clobbering; drop <script> from help text. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reseed dev run form when a local edit changes the script's args The read-only pane is keyed on script.path only, so in /pipeline_dev the selected node re-resolves on every WS bundle without remounting; PipelineScriptView cloned script.schema once, so adding/removing args left the run form on a stale schema (could run with missing inputs). Extract PipelineRunForm (owns the SchemaForm clone) and key it on the serialized schema: a real arg change reseeds the form, an unchanged re-resolve keeps in-progress input. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): don't cut a scheduled/manual root that also has a non-autorun trigger Address Codex P1: the barrier set subtracted only --upload-bound scripts, so a script with both `// on schedule` and `// on data_upload` resolved as the start yet was also a barrier — reachableCutting skipped it, giving an empty run plan. Subtract all valid starts (schedule/manual roots + bound handlers) from barriers: a legitimately-scheduled root runs on its schedule path even if it also carries a caller-input trigger; pure input-only roots stay cut. Adds a regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): deployed non-autorun enrichment, s3:// storage, --to cut accounting, tag regex Address CI review (Codex P1/P1/P2, Pi P2): - Deployed `pipeline run` recovers marker-only data_upload/webhook/email triggers from script bodies (like the `show` path) so input-only entrypoints are cut instead of auto-run empty on the deployed graph. - `--upload s3://<storage>/<key>` keeps the named storage (authority) instead of folding it into the key, matching the S3Object round-trip convention. - Bounded `--to` targets cut by a barrier are reported in droppedEnds (+warning), not reachableEnds. - fallbackParse `// tag` matches a single token (\S+), rejecting multi-word prose. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): header-only deployed marker scan, fail-closed enrichment, default-storage s3 keys Address CI review (Codex P2, cubic P1/P1/P2): - Deployed marker recovery scans the LEADING comment header only (shared recoverHeaderMarkers helper, reused by the show enrichment too) so a body comment `// on data_upload` can't inject a phantom trigger and over-cut. - Deployed run enrichment fails CLOSED: a script-body fetch error aborts the run instead of silently letting an input-only entrypoint run with empty args. - Revert `--upload s3://` to default-storage whole-path keys (matching pipeline `s3://` asset-URI semantics); named-storage authority-splitting broke nested default keys like `s3://raw/2026/events.csv`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): reject trailing content on fallback native markers; trim s3:/// key Address CI review (Codex P2, cubic P3): - fallbackParse now requires a native marker (`// on data_upload`) to stand alone; a line with trailing content (`// on data_upload f/foo`, `# on kafka topic`) is rejected, matching the canonical parser and keeping local/deployed parity. - s3UriKey trims a leading slash so the canonical empty-authority default form `s3:///key` doesn't leak a leading slash into the object key. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): persist dev WS token per-port so reconnect survives a CLI restart Address Codex P2: the /pipeline_dev auto-reconnect reuses the ws_token from the page URL, but `pipeline dev` minted a fresh random token each start, so a restart on the same port left the open page rejected by verifyClient forever. Persist the token per-port under the user-private config dir (0600) and reuse it on restart, so an already-open page reconnects — matching the reconnect behavior's intent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): scope persisted dev WS token by workspace+folder+port Address cubic P2: keying the persisted token by port alone let a stale browser tab from a previous folder's session on the same port reconnect and receive a different folder's source. Scope the token file by workspace+folder+port so a same-session restart still reconnects, but a different folder on the same port gets a distinct token that rejects stale cross-folder tabs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): caller args can't override skip-dispatch guard; hash the dev token key Address CI review (Codex P1, cubic P2): - makeLaunch / CLI run build args with `_wmill_skip_asset_dispatch` LAST (and drop any caller-supplied copy) so a run-form/`--upload` arg can't re-enable backend asset dispatch while the client orchestrates the cascade (double-run / running deployed subscribers from a local preview). Adds a cascadeRun guard test. - Dev WS token file key is a sha256 of NUL-delimited workspace+folder+port, so different folders (`a/b` vs `a_b`) can't collide onto the same token file. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): canonical s3://storage/key --upload parsing; scope dev token by remote+root Address Codex P1/P1: - Restore canonical S3Object URI parsing for `--upload` s3 sources, matching the frontend's `parseS3Object` (`s3://<storage>/<key>`, empty authority ⇒ default, `s3:///key`/`s3:///nested/key` for the default store). `s3://secondary/k.csv` → `{ s3: "k.csv", storage: "secondary" }` so a named-storage object is read from the right store. (This is the canonical convention; the default-storage nested key is served by the `s3:///` form.) - Scope the persisted dev WS token by remote+workspace+root+folder+port (was workspace+folder+port), so two profiles on different remotes (or local checkouts) with the same workspace/folder/port don't share a token — a stale tab can't reconnect across a workspace/remote boundary. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
1c46f899ca |
fix(mcp): stop double-escaping string query params in build_query_string (#9855)
MCP tool arguments were converted to URL query values via `value.to_string()`
+ `trim_matches('"')`. For string values containing JSON (e.g. the `args`/`result`
filters on job listing, `args` on schedule listing), `to_string()` JSON-encodes the
string and escapes inner quotes with backslashes; stripping the outer quotes leaves
`{\"k\":\"v\"}`, which the backend's `serde_json::from_str` then fails to parse,
falling back to `FALSE` and returning zero results.
Use `value.as_str()` to emit the raw string content for `Value::String`, falling
back to `value.to_string()` for non-string types (numbers, booleans). Adds
regression tests covering JSON-string, non-string, and plain-string params.
Fixes WIN-2114
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Ruben Fiszel
|
b52972d0de |
fix: validate workspace name length (max 50 chars) on create and fork (#9854)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
293647de4c |
fix: grant workspace_diff, materialized_partition, debounce_stale_data to windmill roles (#9853)
Same grant gap already fixed for notify_event (20260619091631), script_trigger (20260619112847), and dispatch_event: tables created after the one-time GRANT ALL in 20250205131523 rely on ALTER DEFAULT PRIVILEGES, which only covers objects created by the role that set them. On deployments whose migration runner is a different role, these tables end up ungranted, and writes that run under the RLS role (a transaction opened via user_db.begin(&authed) -> SET LOCAL ROLE windmill_user/windmill_admin) fail with "permission denied for table <name>". Audited every table created after 20250205131523: these three are the only ones with a confirmed write on a user_db transaction that lacked a grant: - workspace_diff: UPDATE in set_ws_specific (workspaces.rs) - materialized_partition: INSERT via record_materialization (assets API); sibling materialized_asset_schema was already granted - debounce_stale_data: DELETE in resume_suspended_trigger_jobs (global_handler.rs) GRANT is idempotent so re-application is a no-op. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
5a661279a3 |
feat(pipelines): add managed SCD2 history materialize strategy (#9850)
* feat(pipelines): add managed SCD2 history materialize strategy `// materialize ducklake://... key=<col> history [track=...]` (alias: `scd2`) upgrades the keyed merge to SCD type 2: diff the current snapshot against live rows, close changed versions (valid_to/is_current) and open new ones in one transaction, keeping full history. Adds a consumer-convenience <dim>_current view; effective-dated joins via native ASOF JOIN >= valid_from. Managed, so // data_test and schema capture work (unlike manual mode). Non-partitioned v1, soft-delete on absence. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(pipelines): document scd2 track= spacing, reserved _current suffix, schema-freeze Addresses non-blocking CI-review nits on the new SCD2 public surface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): null-safe scd2 key matching + create _current view inside txn Addresses CI review: (1) Codex P1 — NULL natural keys were flagged as changed but silently dropped because `key IN (...)` never matches NULL; close/open now match with `IS NOT DISTINCT FROM` via correlated EXISTS. (2) cubic P2 — the `<dim>_current` view was created after COMMIT and CREATE VIEW advances the DuckLake snapshot, so the summary recorded the view's snapshot instead of the data write; the view is now created inside the write transaction. Validated both against a real DuckLake (NULL key materialized; one snapshot per run). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): create scd2 _current view with IF NOT EXISTS to keep no-change runs no-op Addresses CI review (Codex P2): CREATE OR REPLACE VIEW advances the DuckLake snapshot every run, so an unchanged rerun still minted/recorded a snapshot. The view definition is static, so IF NOT EXISTS creates it once (folded into the first data-write snapshot) and is a true no-op thereafter — verified an unchanged rerun keeps max(snapshot_id) constant. Also softens the reserved-name collision: IF NOT EXISTS skips silently instead of erroring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipelines): add scd2 deletes=close (hard-delete-close) Opt-in `deletes=close` closes the current version of a key that disappears from the snapshot (dbt's hard_deletes=close); default stays soft-delete. Codegen adds a vanished-key temp set (current keys EXCEPT snapshot keys) + a second null-safe close UPDATE with no reopen; a reappearing key opens a fresh version (validity gap = correct SCD2). Wired through both parsers with parity fixtures/tests, worker derivation, unit + codegen tests, and docs. Verified end-to-end against a real DuckLake incl. delete-close + reactivation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): align materialize deploy precedence warning with runtime (scd2>append>merge) The deploy-time conflict warning only knew append>key, so warned 'append wins' while the runtime (duckdb_executor) runs SCD2 (history wins). Warn for history+append (history wins, append ignored) before the append+key case, mirroring the runtime strategy precedence. (Pi review P2.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): register scd2 _current view as a produced asset for cascade dispatch The docs present the companion <dim>_current view as a subscribable produced asset (// on ducklake://.../<dim>_current), but deploy registered only the base table as a write asset, so a subscriber on the view would never be dispatched (the cascade fans out from deploy-time asset rows). Register <dim>_current as a produced write asset when scd2 so those subscribers fire. (Codex review P1.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipelines): don't register _current asset for manual+history (no view created) Manual mode short-circuits before the scd2 codegen, so no <dim>_current view is created; gate the produced-asset registration on !manual so a contradictory // materialize manual ... history doesn't leave a false write edge dispatching subscribers on a nonexistent view. (Codex review P2.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
f05b50d29a |
fix: grant dispatch_event table to windmill roles (#9852)
The dispatch_event table (migration 20260523055641) was created relying on ALTER DEFAULT PRIVILEGES to reach windmill_user/windmill_admin. Those default privileges only apply to objects created by the role that set them (20250205131523), so deployments whose migration runner is a different role leave dispatch_event ungranted. Direct writes then run as the invoking role and fail with "permission denied for table dispatch_event" -- notably the DELETE in delete_jobs (windmill-common/src/jobs.rs) that reaps a job's side rows on schedule disable, and the dispatcher insert in asset_dispatch.rs. Grant explicitly, same fix as notify_event (20260619091631) and script_trigger (20260619112847). GRANT is idempotent so re-application (squash, or an operator who already granted manually) is a no-op. Fixes WIN-2112 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
68bf0daf58 |
feat(ansible): support repo-provided ansible.cfg in delegate_to_git_repo (#9851)
* feat(ansible): support repo-provided ansible.cfg in delegate_to_git_repo Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ansible): accept colon delimiter and collections_paths alias in cfg parser Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
bca356555f |
test(audit): de-flake S3 export end-to-end test under parallel tests (#9849)
* [ee] test(audit): de-flake S3 export end-to-end test under parallel tests Bumps the EE ref to pull in the companion fix for the flaky `audit_export_end_to_end` test (`ee::audit_s3_export`). Postgres XIDs and the snapshot xmin are cluster-wide, so under `--test-threads` a neighbor test's in-flight transaction can hold the global xmin between this test's row xids, deferring a committed row to a later export tick (`id 7 must be exported: got [3,4,5,6]`). The EE change models successive ticks (drain until exported) and waits for pre-existing rows to settle before anchors that must exclude them. Test-only; no production code changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to bdd4ba4dfd05c8ca7db365e530812dc914517d7f This commit updates the EE repository reference after PR #639 was merged in windmill-ee-private. Previous ee-repo-ref: 70c4c61257bda9263c158ef0ac58eb3aa9c55fa8 New ee-repo-ref: bdd4ba4dfd05c8ca7db365e530812dc914517d7f Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
bf6be967fa |
fix: honor verify-ca/verify-full sslmode for postgres connections (#9835)
* fix: enforce tls verification for postgres verify-ca/verify-full sslmode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: make PG_ACCEPT_INVALID_CERTS value-based and keep cache key well-formed Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: grandfather existing postgres resources via per-resource trust_cert flag Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: scope pg trust_cert migration to resource data, drop schema patch Hub resource type sync (windmill cache-rt + startup SYNC_CACHED_RT) only touches the admins workspace and is opt-in, so the schema is left to the hub; the migration just grandfathers existing resource values so the upgrade is non-breaking. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: replace pg trust_cert with verify-*-scoped accept_invalid_certs, drop migration Per-resource accept_invalid_certs (default false for new resources) replaces the trust_cert flag and grandfather migration. It only applies to verify-ca/verify-full; unset falls back to legacy behavior (verify only when a root cert is present) so existing and git-synced resources are not broken on upgrade. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: warn in job logs when a verify-* postgres resource skips cert verification Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
hugocasa
|
83f3d7f910 |
feat(licensing): enforce offline license seat cap (#9845)
* [ee] feat(licensing): enforce offline license seat cap Companion to windmill-ee-private. Aligns the offline-license seat count with the billing model and adds real-time enforcement when usage exceeds the cap. OSS side carries the ee_oss stubs, the reactivation cap-check call site, the regenerated SQLx cache, and the EE ref bump. - Exclude instance-disabled users (password.disabled) and service accounts from the seat count. Deactivating a user now frees a seat. - Service accounts no longer consume seats (no check at creation). - Hard-block reactivation when it would exceed the cap. - Invalidate the license (halting jobs) when seat usage exceeds the cap, mirroring CU-cap enforcement; recovers when usage drops back under or a higher-cap key is loaded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [ee] fix(licensing): bump EE ref for reactivation seat-check fixes Points to the EE companion commit that fixes reactivation double-counting and preserves the original seat alert tag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [ee] fix(licensing): reactivation seat delta includes pending invites Bumps the EE ref and drops the now-orphaned usr-only cache entry; the reactivation check reuses the existing usr ∪ workspace_invite query. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * [ee] test(licensing): bump EE ref for offline seat-cap tests Adds #[sqlx::test] coverage for the offline seat counting and cap-check logic; EE-only (runtime queries, no cache change). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: update ee-repo-ref to f814c3f75308c1ef1e4526d8d0eeb360ce16abe4 This commit updates the EE repository reference after PR #637 was merged in windmill-ee-private. Previous ee-repo-ref: b2622e3afc2fe1fe3e2ec978ca46cf9decf91b82 New ee-repo-ref: f814c3f75308c1ef1e4526d8d0eeb360ce16abe4 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben Fiszel
|
6b79bddd42 |
fix(s3_proxy): preserve URL-encoding on Hive-partition proxy writes (#9848)
* [ee] fix(s3_proxy): preserve URL-encoding on forward re-sign for Hive-partition keys Bump ee-repo-ref to pull the EE fix for SigV4 SignatureDoesNotMatch on DuckLake Hive-partition writes through the S3 proxy. The forward re-sign leg rebuilt the upstream URI from the decoded object key (literal `=`) instead of the still-encoded request path (`%3D`), diverging from how S3/minio canonicalizes the key. Companion EE commit c6b110f. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 1a98119b0b8b8548601983c9e5ab091150f0b180 This commit updates the EE repository reference after PR #638 was merged in windmill-ee-private. Previous ee-repo-ref: c6b110fd3b3591a5c3f09952c388c42bd5766188 New ee-repo-ref: 1a98119b0b8b8548601983c9e5ab091150f0b180 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
hugocasa
|
b4b0c6a93e |
feat: add dev workspaces paired with a lockable prod workspace (#9793)
* feat: add dev workspaces paired with a lockable prod workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: gate dev-workspace prod-lock on admin and prevent attach cycles Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: redirect locked-prod edits into the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make dev-workspace settings tab available on CE (was EE-gated) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: lock prod against forking too and funnel edits to the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: open dev item page on edit and tailor dev-workspace lock messages Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: prevent nested dev workspaces and hide dev option when one exists Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: drop the redundant already-has-dev hint on the fork form Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: badge dev workspaces and sort them ahead of forks in the tree/switcher Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: label dev workspaces as 'Dev workspace of X' instead of 'Fork of X' Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: label edit as 'Edit in <dev>', cover editor headers, auto-expand dev in tree Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: split prod lock into separate block-deploy and prevent-forking toggles Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: make resources/variables workspace-specific from compare page Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: steer AI-chat sessions to the dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: refine session fork options and lock guidance for dev/prod Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: session picker reads prod's real rules, default to current ws Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: copy members into forks and clarify dev-workspace root labeling Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: place the workspace id field under the fork name Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address dev-workspace review findings and harden fork detection Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: regenerate sqlx offline cache Restores entries dropped during the origin/main merge and adds the dev-workspace queries (is_dev_workspace, ws_specific, has_parent). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address second-round dev-workspace review findings Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: address Pi and Codex review findings on dev-workspace endpoints Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: gate locked-dev git-branch fork on admin and validate ws_specific path Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: clear prod dev-lock when deleting an attached dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: consolidate dev-workspace migration and scope all-group join to attach Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: restore dev-workspace CHECK into consolidated migration and scope all-group join Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: drop copy_members from the dev-workspace attach path Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: dev-workspace lifecycle/auth fixes from Codex review round Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: explicit create-in-other for workspace-specific items Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make create-in-other strictly create-only (never overwrite target) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: return 403 (not 401) for dev-workspace permission denials Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: allow attaching a same-family fork as a dev workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: emphasize the go-to-dev action in the no-direct-deploy alert Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: seed a resource's linked variables when creating it in the other workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: judge workspace deploy/fork locks against the user's identity in that workspace Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * style: clarify create-in help text in workspace-specific panel Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: admin-gate dev-workspace creation and harden lock/seed edges Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: preserve a staged fork's source on picker create-mode re-entry Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: clear dev flag on archive and check dev existence server-side Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: make create-in-other atomically create-only via direct create Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: create-only resource insert, ws-specific list scopes, archive lock guard Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: reserve the dev_workspace_lock protection-rule name from the public API Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: reattach create_protection_rule doc comment to its function Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor: make dev-archive pairing teardown atomic with the archive Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: follow deploy_to on root rename; show dev pairing to non-member prod admins Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: copy creator metadata on fork; invalidate fork routing cache on rename Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: accept g/ paths in set_ws_specific; gate copy_members to dev workspaces Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
83ed011e26 |
feat(object-store): make GCS service account key optional for Workload Identity (#9842)
build_gcs_client always called `.with_service_account_key(...)`, so an
absent key (the settings UI stores "no key" as the empty JSON object `{}`)
was handed to the builder and failed to parse instead of falling through
to the object_store crate's InstanceCredentialProvider. Skip the call when
the key is blank so GCS uses the instance's ambient credentials (GKE
Workload Identity / the GCP metadata server).
"Blank" (empty/whitespace/`{}`/`null`) is centralized in a shared
`gcs_service_account_key_is_blank` predicate so the build path and the
non-super-admin connectivity-test SSRF guard (`validate_object_storage_test`)
agree on what counts as "no key" — otherwise a blank key would bypass the
guard yet still trigger the ambient-credential fallback, letting an
untrusted caller probe arbitrary buckets with the server's instance role.
Also clarify the settings UI hint that the key may be left empty for
ambient credentials, and add regression tests for the blank-key build path
and the guard.
Fixes WIN-2110
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
Diego Imbert
|
cdfd7b3d61 | Refactor + handle datatable setting delete/rename | ||
|
Ruben Fiszel
|
a9ffdb996b |
chore(main): release 1.743.0 (#9837)
* chore(main): release 1.743.0 * Apply automatic changes --------- Co-authored-by: rubenfiszel <275584+rubenfiszel@users.noreply.github.com> |
||
|
hugocasa
|
9b65161c64 |
fix(gcp): require token verification for authenticated push delivery (#9834)
* fix(gcp): require token verification for authenticated push delivery Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to 38e87caeca6a1dce9e4f3fa029ac36dffb30f1b2 This commit updates the EE repository reference after PR #636 was merged in windmill-ee-private. Previous ee-repo-ref: 8c63d487c486002baf09c77ab937fd77a91765eb New ee-repo-ref: 38e87caeca6a1dce9e4f3fa029ac36dffb30f1b2 Automated by sync-ee-ref workflow. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben Fiszel
|
c91027824b |
feat(pipeline): AI-chat data-pipeline editor (route + in-session) + home surfacing (#9805)
* feat(pipeline): AI chat tools to build pipeline nodes with diff/approval Add a data-pipeline AI chat experience modeled on the flow editor and surfaced through the dev-gated global chat (no new chat panel). The /pipeline editor registers PipelineAIChatHelpers on the AIChatManager; while it is open the global mode layers pipeline tools, a pipeline prompt section, and the helpers on top of the full global tool set (behavior is unchanged when no pipeline editor is open). New tools (frontend/src/lib/components/copilot/chat/pipeline/core.ts): - get_pipeline_graph / read_pipeline_node — read the live graph and bodies - build_pipeline_node / edit_pipeline_node — stage changes as AI-pending drafts - remove_pipeline_node — drop a staged proposal - test_pipeline_node — preview-run a node (requires confirmation) Tools never deploy: they stage drafts flagged aiPending, rendered on the canvas with an accent ring and reviewed via Accept all / Reject all (the flow editor's GlobalReviewButtons). Accept commits the drafts; Reject reverts to a pre-AI snapshot, preserving earlier accepted drafts. Auto-accept is gated on the chat autonomy mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): teach the global/session chat to author data pipelines Without an open /pipeline editor the session chat had no pipeline concept, so "create a data pipeline" loaded flow instructions and built a flow. Add a first-class pipeline authoring path: - system_prompts/base/pipeline-base.md — what a data pipeline is (a DAG of annotated scripts wired by storage assets, NOT a flow) and how to author the // pipeline / // on / // materialize annotations; wired through generate.py as getPipelinePrompt() (regenerated prompts.ts/index.ts). - global/core.ts — new get_instructions subject "pipeline", and a global-prompt rule disambiguating data pipelines from flows so the model routes correctly. - ai_evals/cases/global.yaml — two global cases (single node, two-node chain) asserting pipeline-annotated script drafts and forbidding write_flow, guarding the pipeline-vs-flow conflation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): show & build pipelines in the AI session preview Add a 'pipeline' session preview target so the session AI can show the data-pipeline graph for a folder and build nodes in-pane: - open_preview now accepts kind="pipeline" (path = folder); SessionTarget / EDITOR_TARGET_KINDS widen accordingly. The slot/codec load model stays flow|script|raw_app — pipeline bypasses it with its own fetch/draft state. - New PipelineEditorView.svelte mounts in the session pane: fetches the folder graph, overlays AI drafts, renders AssetGraphCanvas + the Accept/Reject review buttons, and registers PipelineAIChatHelpers on the *session-scoped* manager (via getAiChatManager) so build_pipeline_node / edit_pipeline_node + the diff/approval work inside the session too. - System prompt nudges the model to open the pipeline preview and use the staging tools while building. Verified end-to-end with a real model: the session AI called open_preview, the graph mounted in the side panel, then build_pipeline_node staged a node on the session canvas with its schedule trigger and ducklake output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): share the AI editor logic between route page and session Consolidate the duplicated data-pipeline AI logic onto a single shared layer so the route editor and the in-session preview behave identically and the session gains the full code editor. - New pipelineAiHelpers.ts: createPipelineAiHelpers(deps) owns the propose/edit/ remove/accept/reject/test staging + the per-turn snapshot bookkeeping that powers Reject. Callers inject accessors for their own draft Map and graph. - Route page (/pipeline/[folder]) drops its ~250-line inline AI-helper block and wires the shared factory via deps (folder/workspace/graph/drafts + focus, ensureEditable, run-started). Its shell — persistence, navigation guard, activity, cascade, trigger drawers — is untouched. - Session PipelineEditorView uses the same factory and now renders the real AssetGraphDetailsPane (code editor + live overlays + test), so a node built in a session opens with its source, matching the route editor. Verified: route page hydrates/renders drafts unchanged; in a session the AI opened the pipeline preview, built a node, and its code showed in the details pane. check:fast clean, 197 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): externalize editor state into PipelineEditorState (step 1) Introduce PipelineEditorState — the data-pipeline analogue of the flow editor's flowStore. It owns the draft Map, the live editor overlays, and the selection, with callback-safe methods (handleDraftPersist / handleAnnotationsChange / … ), so a single editor can be rendered by both the route page and the session. This commit lands the store and points the in-session PipelineEditorView at it (no behaviour change — the session already had these inline). Next steps move the route page onto the store and a shared <PipelineGraphEditor>. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): point the route editor at PipelineEditorState (step 1) Move the route page's draft Map, live editor overlays, selection, and the draft-persist / live-change handlers onto the shared PipelineEditorState (`pe`), referencing them as `pe.*` in place. No behaviour change — persistence, graph resolution, run dispatch, AI staging, and deploy all stay on the page and now read/write the externalized state. This is the data-pipeline analogue of the flow editor's flowStore: the route page and the in-session preview now share one source of editor truth, setting up the shared <PipelineGraphEditor> in the next steps. Verified: the page hydrates its DB draft, renders the overlay graph, the toolbar counts (Save all (N)) track pe.drafts, and selecting a node opens it in the details pane. check:fast clean, 84 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the route editor via shared PipelineGraphEditor (step 2) Extract the canvas + details-pane editor body into PipelineGraphEditor.svelte, the data-pipeline analogue of FlowBuilder. The route page now delegates its Splitpanes block to it, passing the externalized PipelineEditorState plus its run/cascade/trigger/deploy callbacks; the component owns pane sizing, selection/details-open derivation, and the canvas+details rendering. Root-caused the earlier ts2769 "$props() No overload" to a prop named `state` colliding with the `$state` rune (`let x = $state(...)` parsed as a store auto-subscription on the prop) — the prop is now `editor`. Net: the route page sheds ~310 lines of template/state; behaviour preserved. Verified: the page hydrates its DB draft, renders the graph, opens the draft in the details pane (live code editor + Test), pane sizing works. check:fast clean, 24 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): move draft autosave into PipelineGraphEditor (step 3) Fold the per-user `data_pipeline` DraftService bundle autosave (hydrate + debounced persist + localStorage crash mirror) into PipelineGraphEditor, gated by a `persistDrafts` prop — FlowBuilder's parameterized-autosave shape. The route page passes `persistDrafts` + `folder` and reads `editor.loadedFromDbDraft` for its AutosaveIndicator; the in-session preview will leave persistence off. Also restores the `untrack(...)` wrapping on the pane-sizing $effect (dropped when the editor body was extracted in step 2). Without it the Pane `bind:size` feedback loops the effect and pegs the main thread when the details pane is closed — a latent hang in the step-2 commit. check:fast clean, 24 pipeline tests pass. Note: browser revalidation was not possible this session (the Playwright MCP browser was reset); the autosave is a verbatim port and the untrack fix is the original working form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the session preview via shared PipelineGraphEditor (step 4) Point the in-session PipelineEditorView at the shared PipelineGraphEditor instead of its own inline canvas + details pane. The session now renders the exact same editor body as the route page — gaining the full details/code pane — while opting out of persistence (persistDrafts=false) and the run/cascade/trigger/bounded affordances (their callbacks are omitted, so those controls hide). Building nodes + the Accept/Reject diff still work via the AI helpers. Also fixes issues surfaced by a full `svelte-check` while wiring this up: - PipelineGraphEditor: edit mode opened the details pane unconditionally (a step-2 regression); restored the route's "open only on selection/draft" behaviour. - Route page passed an `isOperator` prop the component doesn't accept (step-2; caught only by full check, not check:fast). - SessionItemNotFound: narrow its `kind` to exclude `pipeline` (pipeline targets never slot-load, so they can't 404 through it) — closes the SessionTarget-widen fallout. - PipelineEditorView: cast the resolveGraph base to AssetGraphResponse. Full `svelte-check` now clean across all pipeline/session files; 137 unit tests pass. (Browser revalidation still pending — Playwright MCP was unavailable this session; see the smoke-test note on the PR.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): stop an infinite microtask loop when persisting a no-output draft handleDraftPersist short-circuits when the open draft's content + inferred writes are unchanged. The writes check compared `d.outputAssets?.length === writes.length`, but a no-output draft has `outputAssets: undefined` (so `?.length` is `undefined`) while the details pane infers an empty `writes: []` (length 0). `undefined === 0` is false, so it never short-circuited: every persist re-wrote the drafts Map with an equivalent object, which gave `activeDraft.script` a new identity → the pane re-emitted its overlays → the graph re-derived → persist fired again. A self- sustaining microtask loop that pegged the renderer and froze the tab on any pipeline carrying a no-output draft (e.g. hydrating one from the saved data_pipeline draft on load). It hangs rather than throwing effect_update_depth_ exceeded because it cycles across microtasks, not within one reactive flush. Fix: coalesce the undefined length to 0 so "no outputs" compares equal to an empty inferred-writes list. Adds pipelineEditorState.test.ts covering the idempotency (fails without the fix) plus the change/no-change cases. Root-caused by instrumenting the reactive churn: every iteration reassigned drafts/liveContent/liveBodyAssets/liveAnnotations/displayGraph with identical values — pure reference churn off the drafts re-write. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): make the agent open the pipeline editor before building nodes In a session, the GLOBAL system prompt only *advised* opening the pipeline preview ("show its graph with open_preview ... prefer those tools once it is open"), so the agent routinely skipped it: on a plain "build a data pipeline" request it reached for write_script and staged plain script drafts, and the canvas editor never opened. build_pipeline_node / edit_pipeline_node are only registered once the preview is open, so skipping open_preview also loses the canvas-staged Accept/Reject diff-approval flow entirely. Make the guidance imperative: open_preview(kind="pipeline", path=<folder>) is the FIRST step before creating any node (an empty or not-yet-created folder is fine — create_folder first if needed), and pipeline nodes go through build_pipeline_node / edit_pipeline_node, never write_script. This also clears the agent's "the folder might not exist" hesitation that pushed it toward write_script. Verified live (same plain prompt, before/after): before it used write_script with no editor; after, the agent opens the editor first and stages a canvas-highlighted node with Accept all / Reject all. The guidance is gated on previewTools (session-only), so it doesn't affect the non-preview global eval cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): preserve in-session pipeline drafts across editor hide/show The session preview's PipelineEditorState lived in the PipelineEditorView component with persistDrafts=false. Hiding the editor sets editorVisible=false, which makes `hasEditor` false and the `{#if hasEditor}` block unmount the view — discarding its component-local store. Showing it again remounted a fresh, empty one, so the pipeline the AI had built in the session vanished. Move the PipelineEditorState onto the per-session SessionRuntime (like the flow / script / raw_app editors, which already host their state there and take {runtime}), so it survives the pane unmount on hide and across session switches. The runtime is keyed by session id and only dropped on session deletion. Because the instance is now reused, guard against a retarget to a different folder: PipelineEditorView resets the state when `path` changes to a new folder (a same-folder remount keeps the drafts). Adds `folder` + `reset()` to the store. Verified: build a node in a session → Close editor → Show editor → the staged node, its wiring, the details-pane code, and Accept/Reject all re-appear. Full svelte-check clean; 139 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline-ai): clearer diff + persistent review banner on the canvas The AI review affordance had two problems on the pipeline canvas: - The floating Accept-all / Reject-all bar sat bottom-center, where it collided with the minimap once the canvas narrowed on node selection — reading as "the buttons vanished when I select a node". - Every staged draft rendered with the same blue ring, so it wasn't clear what the review would actually change (a plain manual draft looked the same as an AI proposal). Replace the floating bar with a top-left review banner (z-30, clear of the controls and minimap) that stays put regardless of selection and spells out the pending counts. Color the diff per node: a proposal that adds a node that isn't deployed rings green with a "new" chip; one that edits an already-deployed node rings amber with an "edited" chip. Plain manual drafts keep the neutral gray dashed border, so only the green/amber nodes read as part of the Accept/Reject set. aiPendingKind is resolved in resolveGraph (deployed runnable present → modified, else added) and forwarded through the canvas to the node. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): persist in-session pipeline proposals across reload/switch Staged AI proposals lived only in the per-session runtime's in-memory PipelineEditorState (persistDrafts=false), so a page reload — and an LRU-evicted runtime on session switch — dropped them, leaving the canvas and the Accept/Reject review empty even though the chat still showed the nodes as staged. Enable the same per-folder DB-draft persistence the route page uses for the in-session editor. To keep hide/show cheap and race-free, hydration is now gated per editor instance (PipelineEditorState.hydratedFromDb) rather than per component mount: the runtime-hosted instance hydrates ONCE when fresh (reload / evicted runtime) and then keeps its in-memory drafts across the editor pane unmounting on hide — re-reading the DB on every remount would race a not-yet-flushed autosave and drop a just-staged draft. A folder retarget resets the flag so the new folder re-hydrates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): make Reject all work for rehydrated proposals rejectAll only reverted paths tracked in the in-memory aiSnapshots map, which is rebuilt empty on each editor mount. After a reload (or session switch into a fresh runtime) the proposals are restored from the persisted draft but have no snapshot, so Reject all was a no-op on exactly the nodes it should discard. Sweep any still-pending draft without a snapshot and discard it (revertPath with no snapshot deletes the path; for an edit of a deployed node that correctly falls back to the deployed body). Adds unit coverage for accept/reject including the no-snapshot case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep proposals visible while the graph reloads on switch The session editor pane is LRU-capped (MAX_WARM_EDITORS), so returning to a session whose pane was evicted remounts PipelineEditorView with a fresh graphRes resource (loading=true, current=undefined). The deployed-graph loading spinner gated the whole canvas, so the staged proposals and the Accept/Reject review banner vanished until the re-fetch resolved — read as "the proposal disappears when I switch sessions". Only show the loading/error placeholder when there are no drafts to display. When the runtime already holds staged drafts, render the editor immediately: resolveGraph overlays them on an empty base so the proposals + banner stay visible, and the deployed nodes fill in when the fetch completes. Verified with a 4s-delayed graph fetch — proposals render through the load with no spinner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): apply AI node edits directly as drafts, no approve/reject The canvas-level Accept all / Reject all review (aiPending proposals, the green/amber diff ring + "new"/"edited" chips, and the review banner) didn't fit the pipeline editor. Match the flow/script editor instead: build/edit apply directly as ordinary unsaved drafts on the canvas, which the user then deploys — there is no separate approval step. Removed across the surface: - aiPending / aiPendingKind on the runnable node + resolveGraph seeding + canvas forwarding; AI-built nodes now render with the existing plain unsaved-draft dashed styling. - the review banner, count derivations, and hasAiPending/onAccept/onReject props from PipelineGraphEditor and both consumers (route page + session view). - acceptAll/rejectAll/hasPending and the per-turn snapshot bookkeeping from the shared helpers; removeProposedNode now just discards the unsaved draft at a path (undo a build). acceptAllProposals/rejectAllProposals/ hasPendingProposals dropped from the PipelineAIChatHelpers interface and the manager's auto-accept hook. - accept/reject language from the tool descriptions, return messages, and the system-prompt section. Tests updated; pipeline + AssetGraph suites pass (142). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts-diff): support data_pipeline diffs + fix blank empty-summary row Two issues in the session "Drafts" diff drawer (DraftDiffDrawer): - Clicking a `data_pipeline` bundle row threw "Draft diff not supported for kind data_pipeline" (utils_draft_deploy.ts) — there was no handler for the kind, so it fell to the OVERLAY_GETTERS lookup and errored. The bundle has no deployed counterpart (each node deploys individually as a script), so diff it node-by-node: surface each node's draft body keyed by path, folding in the deployed body as the "before" when a node edits a deployed script. - A draft row whose summary is an empty string (e.g. the app draft) rendered with no title at all: WorkspaceItemRow's single-line branch used `summary ?? secondary`, and `??` doesn't treat '' as absent, so it showed the empty summary instead of the path. Use `||` so an empty summary falls back to the path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(drafts-diff): explode data_pipeline bundle into per-node subitems A data_pipeline draft is a bundle of node-script drafts, so a single row diffed the whole thing as one blob. Explode it in DraftDiffDrawer into one script row per node, nested under the bundle's `…/data_pipeline` folder so they read as the pipeline's subitems — each with its own path and a proper script Content/Metadata code diff. The node's draft body is the "after"; its deployed body (when the node is already deployed) is the "before", so edits show as line diffs and new nodes as added. A single bundle row (via the getDraftDiffValues data_pipeline fallback) is kept only for the case where the bundle can't be read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): simplify — drop vestigial approve/reject scaffolding & redundant field Review pass over the PR, removing complexity left from the approve/reject removal and the shared-component refactor (all behavior-preserving): - Inline the `acceptPendingEdits` pass-through into `acceptPendingFlowEdits` and revert the now-inert `autoAcceptEditsAvailable` GLOBAL+pipeline widening (pipeline edits are direct drafts — nothing to auto-accept). - Fix the global system prompt: pipeline tools "apply directly as unsaved drafts (no accept/reject)", not "proposals the user Accepts or Rejects". - Collapse the redundant `outputAsset` (singular) into `outputAssets`, removing a whole resolveGraph fallback tier; simplify propose/editNode. - Drop the single-field `PipelineAiHelpersHandle` wrapper (callers just destructured `{ helpers }`); inline the misleading `isoNow()` helper. - Remove the now-unreachable `data_pipeline` branch in getDraftDiffValues (the drafts drawer explodes bundles per-node; an unreadable bundle is skipped) and the "Step N consolidation" drafting narration. - Un-export internal-only types; reuse `storageKey`; refresh stale comments that still referenced proposals / the review banner / diff-approval. svelte-check clean; 141 unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): tooltip clarifying the Create/Save button deploys The accent button in the asset-graph details pane ("Create" for a new script, "Save" for an existing one) is really a deploy, but had no tooltip explaining that. Add a title — "Deploy this new script to the workspace" / "Deploy your changes to this script" — keeping the create-vs-update label distinction while making clear both deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): document the `materialize` annotation in the pipeline prompt The model invented "materialize run" because the prompt only mentioned `// materialize <uri>` in passing. Spell out what it is in both the in-app pipeline prompt (getPipelinePromptSection) and the base prompt (pipeline-base.md, regenerated): a MANAGED output where the runtime writes the table around a single SELECT (no manual CREATE/INSERT); replace (default) vs `append` vs `key=<col>` strategies; `manual` to opt out (track-only); and its pairing with `// partitioned …` (runs once per partition, `{partition}` token substituted at run time). Explicitly: materialize is an output declaration, not a command — there is no "materialize run". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): trigger drawers in the AI session preview Bring the route page's native-trigger affordances to the in-session pipeline editor by reusing the shared <PipelineTriggerEditors> (no duplication of the drawer UI). Clicking a "Schedule · Missing — no trigger row" node (or edit/delete on an attached trigger, webhook, data-upload) now opens the same drawers the full editor uses, instead of doing nothing. Draft nodes get the same "save the script first" guard (a trigger row needs a deployed script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): run buttons + live run state in the AI session preview Wire the per-node Run button and live run-state badges into the in-session pipeline editor, reusing the shared folder-scoped job poll (useActiveRunnableIds) the route page uses — node badges, the event log, and the zero-latency "running" hint all come from it. The session runs one node at a time (preview for an unsaved draft, the deployed version otherwise), skipping the route page's cascade/deploy-queue machinery the AI-session UX doesn't need. Verified: a node's Run button dispatches a job and the badge updates live from the poll. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): label the node deploy button "Deploy" (was Create/Save) Users read "Create" and asked whether it deploys. It does — and the main script editor's DeployButton already says "Deploy", so this is the consistent term. Use "Deploy" for both the new-script and existing-script cases; the new-vs-changes nuance stays in the button's tooltip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(home): surface data pipelines as units, including bundle-phase drafts Treat a data pipeline as one home entry instead of scattering its member scripts: - The home "Pipeline · f/<folder>" entry now also covers bundle-phase pipelines — a folder that so far only exists as a `data_pipeline` draft — not just deployed ones, so a pipeline shows up the moment its first node is drafted (union listPipelineFolders + data_pipeline draft folders). - Pipeline-member scripts (`auto_kind='pipeline'`) are filtered out of the individual scripts list; they're represented by their pipeline's entry. - Tree view injects pipeline folders so they (and their "Pipeline" entry) still appear when their only scripts are hidden members or they have none deployed yet. Verified in both list and tree view: app_groups (deployed member folded) and a draft-only nyc_transit both show as pipelines; the member script no longer lists individually. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(scripts): compute auto_kind for draft-only pipeline nodes A never-deployed pipeline node (a script draft starting with `// pipeline`) had no script row, so list_scripts synthesized it with `auto_kind: None` — and the home page therefore couldn't tell it was a pipeline member, listing it individually instead of folding it into its pipeline. Parse the draft content the same way the create path does (`parse_pipeline_annotations(...).in_pipeline`) and set `auto_kind = "pipeline"` on the synthesized draft-only row, so draft nodes fold into their pipeline like deployed members. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(search): hide pipeline-member scripts from global search The Ctrl+k global search listed pipeline-member scripts (`auto_kind='pipeline'`) individually. Filter them out — they're reached through their pipeline, matching the home page. Deployed members carry auto_kind from the script row; draft-only members now do too (computed from draft content in list_scripts), so both are excluded here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address PR review findings Session run dispatch (the one real bug): - runNode now passes `_wmill_skip_asset_dispatch: true` for a single-node run of a deployed node unless the user chose "run + downstream" (cascade) — previously a single Run could fan out to downstream deployed scripts via the backend asset dispatcher and fire side-effecting production runs. - onRunProducer guards `kind === 'script'`; onTestStateChange only clears the run hint for the script the pane finished (not a different in-flight node); clear the hint on folder retarget; gate the background poll on isActiveSession so hidden warm panes don't poll; note the PipelineTriggerEditors workspace coupling. Home page pipeline surfacing: - Fold pipeline-member folders into `pipelineFolders` (captured in loadScripts) so a members-only / draft-only-`// pipeline` folder still shows its pipeline entry instead of vanishing; and don't render the empty-state when only pipelines remain (they aren't part of the text filter). - Insert injected tree folders in name order instead of prepending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): make clear `// materialize` is DuckDB + DuckLake only The model put `// materialize` on a python3 node, which deploy rejects ("only supported for DuckDB scripts"). The prompt only implied SQL ("write the body as a single SELECT") without stating the hard constraint. Spell it out in both the in-app prompt and pipeline-base.md: `// materialize` is DuckDB-only and its target must be a DuckLake table; for python3/bun/postgresql nodes, write the output via the SDK instead and let it be inferred — reach for duckdb when a node should materialize a DuckLake table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): fix stale comment — session now wires run + trigger affordances Addresses review: the comment still claimed the session 'opts out of the run/cascade/trigger/bounded affordances', but run buttons + trigger drawers were wired in. Describe the current state (wires run + triggers; omits only cascade/bounded/add-script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address Codex review — test_pipeline_node dispatch + tree search - [P1] testNode (the test_pipeline_node tool) ran a deployed node via runScriptByPath without `_wmill_skip_asset_dispatch`, so previewing one node could fan out to downstream deployed subscribers and run side-effecting scripts. Add the skip flag (test is always single-node) + a regression test. - [P2] Home tree view injected pipeline folders — and rendered their Pipeline row — even during a text search, surfacing unrelated pipelines. Gate both the TreeViewRoot injection and TreeView's hasPipeline on `!isSearching`, matching the list view which hides pipeline rows on a query. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep the pipeline prompt after update_user_instructions rebuildGlobalSystemMessage (called by the update_user_instructions tool) rebuilt only the base Global prompt, dropping the pipeline-editor section that configureGlobalMode appends. So after the chat remembered an instruction, the next GLOBAL turn lost the active /pipeline/<folder> context + direct-draft/ materialize guidance while pipeline tools stayed registered. Re-append the pipeline section here when a pipeline editor is registered. Addresses Codex review [P2]. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(home): gate pipeline entries by kind/archived/owner filters Codex review [P2]: pipeline rows/folders rendered independently of the item filters, so a pipeline still showed under the Flows/Apps tabs, in the archived view, and outside a selected owner. Add `visiblePipelineFolders` applying the same gates the items get (kind ∈ {all, script}, not archived, owner-prefix match) and route the list rows, tree injection, and empty-state check through it. Pipelines are always `f/<folder>`, so the user-folder toggle and kind=script keep including them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address review — route folder-switch state, AI node guards, diff identity claude[bot] [P1]: the route page's in-app folder switcher navigates same-route (no remount), but nothing reset PipelineEditorState — so folder A's drafts displayed under B and autosave persisted them into B's bundle, and B never hydrated. Reset pe on folder change (mirror the session retarget), and guard the shared hydrateDrafts against a stale folder result landing after a retarget. codex/claude [P2]: build_pipeline_node (proposeNode) only checked drafts.has — now rejects a path outside the open folder and one colliding with an existing deployed node (model should edit_pipeline_node). + 3 regression tests. codex/claude [P2]: exploded pipeline-node diff rows shared `script/<path>` with a standalone script draft at the same path, colliding in the {#each} key + value cache. Add an explicit unique `key` (the distinct bundle-nested path) on DiffRow; pipeline nodes set/look up by it while `path` stays the real edit target. claude [P2]: session AI test_pipeline_node now arms the live run badge (onRunStarted), matching the route page. nit: pipelineAiHelpers.test uses afterEach(restoreAllMocks) instead of an unreachable inline mockRestore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): harden AI node mutations + close home label-filter / rename gaps Codex [P1] (AI mutations trust model paths) — fully scoped now: - editNode validates the open folder too (proposeNode already did), via a shared assertInFolder; an edit_pipeline_node for f/other/* no longer persists an unrelated script into the current folder's data_pipeline bundle. - both build_pipeline_node and edit_pipeline_node now require the `// pipeline` annotation (assertPipelineAnnotation) so a staged draft is definitionally a pipeline member, not a silently-non-member script. + tests. (proposeNode's folder + deployed-collision guards landed in the prior commit.) Codex [P2] home label filter — visiblePipelineFolders ignored labelFilter, so a label selection still showed every pipeline (and the empty-state fell through to render pipeline rows). Pipelines carry no labels, so a label filter hides them. Codex [P2] session rename — PipelineEditorView now wires onScriptRenamed (repoint selection + refetch), matching the route page; a persisted-script rename no longer leaves the canvas on the old path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): language-specific comment prefix for annotations Codex [P2]: the tool schema and prompt told the model to write `// pipeline` / `// on` / `// materialize` regardless of language, and pipeline-base.md grouped SQL with `#`. A `//` (or `#`) annotation line is invalid in a DuckDB/Postgres node — it passes the frontend parser (which strips `//`/`--`/`#`) but is a SQL syntax error at deploy/run. Make the guidance language-specific everywhere: `--` for SQL (duckdb/postgresql), `#` for python3/bash, `//` for bun/TS — the `//` in examples is the TS form to translate. Regenerated the prompt outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): re-scope Global prompt on folder switch + language-aware base prompt Codex [P2] x2: - The route page resets editor state on an in-app folder switch, but the Global chat's system message kept the old `/pipeline/<folder>` scope (the helper methods read the reactive folder, but the prompt string is only rebuilt on Global-mode reconfigure). Rebuild it on folder change so the next turn targets the new folder. - The pre-editor base Global prompt (seen before open_preview/get_instructions) still showed TS-only `// pipeline` / `// on`. Make it language-aware (`--` SQL, `#` Python/Bash, `//` TS) so the model can't draft invalid DuckDB/Postgres nodes before the pipeline tools are registered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): authoritative new-node probe + SQL-correct eval checklist Codex [P2] x2: - build_pipeline_node's collision check relied on the resolved graph, which can be empty while the session preview races open_preview (a build could shadow a deployed node before the graph loads) and only covered pipeline runnables, not a non-pipeline script at the same path. Add an authoritative backend probe (ScriptService.getScriptByPath): any deployed script at the path → reject with "use edit_pipeline_node". + regression test (empty graph, deployed script). - The DuckLake eval judgeChecklist required the exact `// pipeline` annotation, which would penalize the now-correct `-- pipeline` SQL output (or reward invalid DuckDB syntax). Make both cases syntax-aware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): rebuild Global prompt on session preview folder retarget Codex [P2]: open_preview(kind="pipeline", path="B") can retarget an existing pipeline preview from folder A to B without remounting. The retarget effect resets editor state and the helper methods read the new path, but the registration effect only depends on isActiveSession, so the Global system message stayed scoped to /pipeline/A. Mirror the route-page fix: rebuild the global system message on retarget (gated on isActiveSession — only the active session's helpers are registered; a hidden session reconfigures when it next becomes active). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): edit_pipeline_node preserves deployed script metadata Codex [P1]: editNode kept only the deployed script's language and staged a fresh makePipelineScript draft with empty hash/summary/description/tag/schema/settings. Deploying that edit from the pane (auto_parent) would update the script while wiping its metadata, and the route "Save all" path (no parent_hash) could hit the backend path-conflict branch on the occupied path. Base the draft on the existing draft's / deployed script object and replace ONLY content (+ inferred output assets), preserving hash and metadata. + regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Diego Imbert
|
da04ffdcc0 |
Merge remote-tracking branch 'origin/main' into change-68b704f7
# Conflicts: # backend/ee-repo-ref.txt # backend/windmill-api-workspaces/src/workspaces.rs |