mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-10-13 08:02:47 +00:00
python-instance-baseline
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
   
|
3dae553576 |
feat: pipeline editor: assets-only view, Fix actions and draft triggers (#11440)
* wip: one sidebar for operators and developers, with a detach mode Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: snap-detach the operator sidebar, and join the handle to the picker Dragging the sidebar into the left snap zone detaches it for an operator, who has no icon-only rail to collapse into; the width the drag started from is what gets persisted, so re-attaching restores the sidebar's size. The handle's colour pastille slides onto the card's workspace pastille as the card opens, alongside the workspace name, and both buttons that attach or detach the sidebar carry a tooltip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: attach and detach from the sidebar header, open the card under its handle The button that swaps the sidebar between docked and detached moves to the header, next to the workspace name, so it stays in one place across the swap; the icon-only rail has no room for it and keeps the footer copy. The handle itself becomes a component of its own, carries the workspace icon its picker uses, and opens the card rather than docking — docking is the header button's job. The card is placed under the handle measured at open time instead of at a fixed offset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: one page header across routes, with the workspace picker in it Every route now wears the same bar above its content: the sidebar toggle, the workspace picker, and whatever the route registers — an item's breadcrumb and summary, or a section name — with the route's own buttons at the far end. The bar's row is level with the sidebar's own top row, which the workspace ⇄ sessions switch now owns, so the two read as one band. A route fills the bar by mounting PageHeaderContent, which registers a getter rather than a snapshot: the layout calls it while rendering, so a path or summary the page edits reaches the bar without an effect keeping two copies in step. Nothing registers yet; the routes follow. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: detail pages fill the page header, which now spans above the sidebar A script or flow detail page no longer draws a bar of its own: its path becomes the header's breadcrumb, its summary keeps the rename-and-labels popover next to it, and its badges, menu and buttons ride along as the header's actions. The band runs the full width with the sidebar hanging below it. A snippet rendered in the header initialises its components under the header, not under the page that wrote it, so anything the page's tree provides is lost — the detail page's buttons read TriggerContext and threw. ContextBridge carries the entries a registration names back around the render. The floating handle is gone with it: the header's own toggle is the one control, hovering it peeks the sidebar in and clicking pins it. The workspace picker in the header is a denser variant of the sidebar's, and the card's outside-click catcher starts at the card rather than the viewport top, so it no longer swallows clicks meant for the toggle. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: one breadcrumb in the page header, and the banners that used to sit under it The header band now carries a single breadcrumb — workspace, fork, then the item's path — built from one component, so every segment has the same size, weight and icon size, and slashes separate them. The item's kind icon rides with its summary, after a dot, rather than with the path. The fork and drafts banners move into the band as badges plus a subtle Review button: ahead/behind counts instead of a bare change total, and the per-kind breakdown stays on the compare page. The home hero's CLI, MCP and Hub links move into an ellipsis menu, alongside the toggle that pins the session chat to the page. The flow editor and the app viewer hand their bar to the band too, and a route can drop the band's bottom border when its own content draws one — the border stays in the box with a transparent colour, so contents never shift a pixel across a navigation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the runs page's own header row moves into the page header The title, the queue counters, the kind and status filters, the schedule and future-job switches, the search field and the timeframe picker all ride the band now, and the page starts at the chart — one row of vertical space back. The two toggle groups appear from 1500px: below that the bar cannot hold them next to the search field, and the same filtering is reachable from the search field's own kind and status chips. The search field is what gives width back, and the breadcrumb yields last, so a page's own name stays whole while the workspace and fork names truncate. The queue counters keep their icon form at every width, since the bar has room for two counters and never for the two labelled blocks. Their counts hang outside their icons, so the group carries the margins that keep a badge off its neighbour, and the suspended counter answers a hover at zero rather than sitting there as a dead icon. The band is 44px rather than 40, and a page can hand the breadcrumb a hint — the documentation tooltip that belongs with the page's name rather than with its buttons. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: no border under the page header, and the peek card closes on the way out The band draws no bottom edge and no page asks it to, so `barBorder` goes with it — a page that needs the separation draws its own line, as the detail pages now do above the run form and the tab strip. With nothing toggling a border, the band is a flat 44px on every route. The detached sidebar's card is a hover affordance, so it leaves when the pointer does: off the handle and off the card both, on the same debounce a hover menu uses, because the two do not touch and the leave of one fires before the enter of the other. The mobile drawer is opened deliberately and still stays until it is dismissed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: one band, one workspace part, and the pickers that hang off it The band is a single element whose placement is styled rather than branched: a page registering a side-panel inset used to swap it between two `{#if}` arms, destroying the header and rebuilding it a frame later — a blink of no header with the content sliding up into the gap. The workspace and its fork become one breadcrumb part: the name leads home (inert in a session, where the workspace is the session's scope rather than a place to go) and one chevron opens the picker, which already expands a family to reach its forks. The picker's panel hangs from the part's left edge, opens with the active family expanded, and closes before the switch runs — the switch renames the workspace, which moves the trigger the panel hangs from, and floating-ui was chasing it across the viewport. The item's path is a plain label rather than a row of pickers: same separators and weight, and a click copies it with its own confirmation. The kind icon goes, the page below says what kind it is. Around them: a "prod" badge for a root workspace, the session's acting fork beside the session's name rather than in the breadcrumb, the edit-in-dev affordance as a button in the header, the drafts count as a sentence, and a fade when the workspace changes under a page that stays mounted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep a page's own controls when the workspace navigation is hidden `menuHidden` means "this is an embed, don't show the workspace's navigation" — but Runs, the detail pages and the flow editor now keep their controls in the band and nowhere else, so hiding it took the filters, the timeframe and the page's buttons with it. A session's preview iframe opens exactly that way. The band now renders for an embed too, without its breadcrumb, so the page keeps its controls and the embed still gains no way to navigate the host's workspace. The flow editor's autosave indicator moves into the header's actions with the rest of its bar: saving state, the autosave toggle and other people's drafts were left behind in the branch the full-page editor no longer takes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: the header carries what it promised, and nothing it does not The registry's item described pickers the breadcrumb no longer has: nine fields — the editability flags, the change callbacks, the navigation hook — that nothing read, while the flow editor passed eight of them and quietly lost its inline rename. The type now says what the bar does: a path, and either a summary or the page's own control for one. The flow editor renames through that control, as the detail page already did. `barPlacement` went the same way: no page registered 'inline', so the layout's guard never fired. The sessions page registered an inset of 0 whenever its preview pane never mounted — the "no session selected" and opted-out arms — and 0 reads as "a page owns the right edge", floating the band over the page's first line. It registers an inset only once the pane has a width. The band's sidebar handle docks through `setDetached` like the card's own toggle, so docking still suppresses the rail's entry animation and closes the card; a summary without its breadcrumb no longer leads with a stray separator in an embed; a page's hint sits with its name whichever way the page named itself; and `Menu.closeInstantly` and `EditorHeader`'s `inline`/`hideSummary` go, their callers having been replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: a deployed app gets the viewport, and the header waits in the corner An app is the page, not a page inside Windmill's chrome, so the band leaves the flow and floats above the viewport until the handle in the app's corner calls it down. The handle is faded until the pointer nears it, hovering peeks the band over the app, and a click pins it — one flag for every deployed app, in this browser, because someone who wants the breadcrumb over a dashboard wants it over the next one too. Peeking overlays rather than pushes: the app is an iframe, and taking 44px off its box on every pass of the pointer would re-lay-out whatever it draws. Pinned, the band is in flow like any other route's and the app starts under it. The sidebar is untouched — docked stays docked, detached stays a peek — since the pin is about the header alone. A page says it wants the viewport with one registry field, so the layout keeps asking the registry rather than knowing about apps. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: one corner control for the chrome an app page hides Hovering the button overlaid on a deployed app brings back whatever that page is hiding: the band always, and the sidebar card too when the sidebar is detached. The card hangs from this button rather than from the header's own handle, which is off-screen with the band. The handle in the band no longer attaches the sidebar — it reveals it and nothing more. A control that hides the thing it sits on cannot also be the way to bring it back, so attaching stays with the toggle in the sidebar's own footer, where detaching happens. The band's two buttons go in the order the things nest: the sidebar's handle, then the header's pin. The overlaid button stands where the pin will be once pinned — same 28px box, and a slot further along when the sidebar's handle takes the first — so pinning moves nothing. It wears the panel pair the sidebar's toggle uses: solid while the band is away, dashed while it is shown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the access point sits in the corner, and the header covers it The button is in the page's top-left corner — 8px in, or just past the rail when the sidebar is docked — rather than shifted along to where the header's pin would land. Hovering brings the header down over it: an access point that the regular view covers once it arrives, so there is nothing to line up and nothing to move. Because the header lands on the button, the pointer leaves it the instant it arrives. The band holds both itself and the sidebar card open while the pointer is anywhere on it, or the card would retract under a header that stayed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: on an app page the floating button owns both, and they move together The band drops its sidebar handle there: the floating control in the corner reveals the sidebar along with the band, and two handles for one gesture is one too many. The two are then linked — the pointer resting on either holds both, and leaving either collapses both — so they behave as the single thing the one button revealed. The card meets the header in a line: it hangs from the header's bottom edge rather than from whichever control opened it, which had it starting a couple of pixels high. With no header to sit under — an embed — it still hangs from the control that reached for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the revealed sidebar is a panel under the header, not a card over the page No rounded corners, no inset from the window: it starts at the header's bottom edge and runs to the left and bottom edges, with one border down its right side and a shadow to lift it off the page. A card floating in from nowhere read as a separate thing; this reads as the sidebar the header sits on top of. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only an app page gets the flush panel; elsewhere the card still floats Over an app the sidebar reads as the panel the header sits on — flush left and bottom, square, one border down its right side. On every other page it is what it was: a card inset from the window with rounded corners and a border all round. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: an app opens with the header already away Opening an app showed the header and then slid it out: the page claimed the viewport on mount, and the band animated from where it had been rendering. The slide belongs to the hover, not to arriving, so it is armed a frame after the claim — the first hide is a cut. The claim moves to the route, which mounts before the viewer waiting on the workspace to resolve. A cold load now paints the band away from the first frame instead of showing a header that leaves again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: an app's Edit sits with its name At the far end of the bar it was a journey across the window from the thing it edits — on a page whose header is only there while the pointer holds it, a journey with a time limit. It goes beside the app's path instead. The slot it uses is the one Runs' documentation tooltip already had, renamed from `hint` to `afterName` and documented for both: what belongs to the page's name, whether that is a mark or a control that acts on what the name points at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: developers collapse, operators detach One control each, rather than three for everyone. A developer switches the sidebar between icon-only and full width and never detaches it; an operator, who has no icon-only rail to collapse to, hides it instead — by the button in its footer or by the same drag that collapses it for a developer. The way back survives either way: a detached sidebar keeps its "Show sidebar" button for anyone, so a developer who detached before this was so is not left with a sidebar they cannot return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: configure pipeline triggers and outputs as drafts in the editor Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: one card for pipeline nodes, and the pipeline pages in the page header Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: drop a removed node's trigger drafts, and confirm triggers when leaving Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: show pipeline assets red when unused or awaited but never produced Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: route long pipeline edges around nodes in the target column, and flag scripts waiting on unproduced assets Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: a page is as tall as it is given, not as tall as the screen The band takes 44px off the top of every route, so a page sized to the viewport overflows by exactly that much: h-screen inside a column that already lost the band's height. These pages measure the box they are in instead. CenteredModal gains `inPage` for the five pages that render it inside the layout rather than over it — the same modal, centred in what is left of the page rather than in the window. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the band narrows, wraps, and draws its own edge The band publishes its width, so the pages filling it can answer it. Two tiers: under 1000px the breadcrumb keeps the workspace disc and its env badge and drops the name, and under PHONE_BAR (800) the trail stops yielding at 10rem — that floor is what makes the bar wrap, so a page's buttons take the line below rather than squeezing the name to an ellipsis. The band also draws the line between itself and the page, and only while something is scrolled under it. One capture listener on the window sees every scroller, since most pages scroll inside their own box and never move the layout's. An inset shadow, not a border: a border would move every page down a pixel the moment the line appeared. DetailPageLayout and InWorkspaceAppViewer drew that first line themselves; two lines would stack, so theirs go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the script, app, runs and audit pages fill the band Each of these kept a header of its own above the page. They register with the band instead: the breadcrumb names what the page is on, and the page's buttons sit at the far end of the one row. They answer the band's width rather than the window's, since the buttons share that row with the breadcrumb. The script editor folds Diff and Settings into the menu it already shows when compact, the app editor drops the panel buttons, Awareness and Export and goes icon-only on Deploy and the mode switch, the flow editor menus its diff, and runs hides its counters. Runs also declares `actionsFlexible`: its search is the one control here that can give width back, and a box that refused to shrink would make it take its maximum instead. The run detail page loses its ActionRow, and the audit log its big title — the breadcrumb says where you are, and the page keeps its tooltip. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: one sidebar column, shown as a rail or as a panel The rail and the drawer held two copies of the same column — the same sections, in the same order, in two pieces of markup, so a change to one silently skipped the other. Both now render `sidebarColumn`, and the shells keep only what differs: the rail its width, resize handle and collapse button, the panel its outside-click catcher and its transform. A phone gets that panel rather than a modal. There was a width test deciding between two presentations; the card under the band's handle works at any width, so it goes. What is left of the width test is the choice between a rail and a panel, plus a `(hover: hover)` test — the old `innerWidth >= 768` was standing in for "has a pointer", which is what decides whether the panel closes when the pointer leaves it. With the card everywhere, the backdrop, the `aria-modal` (nothing dims and nothing traps focus) and the drawer's close cross all go: the band's handle opens the panel and a tap outside closes it. The burger row goes with them. Entering panel width now collapses the rail transiently, so coming back to a wide window does not land on a rail the page was resized away from. The band also stops easing its right edge. That transition belongs to the rail, which moves the band's left; its right follows a splitter the page owns, and easing it left the band lying over a session's side panel — a blank strip across that panel's own header for 200ms after every drag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: the home composer flies to the menu that restores it Removing the composer puts it away rather than throws it out, and the way back is one item in the band's menu — easy to miss on a page the composer has just vacated. A mark flies from the control that removed it to that menu, and the menu answers with one ring as the mark dissolves, so where it went is witnessed rather than explained. The cross and the menu item now read "Remove session chat", one name for one thing whichever a reader reaches for. Reduced motion skips both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: give pipeline asset nodes the trigger nodes' background again Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: taller pipeline nodes, with the asset + back on the node's edge Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: judge dbt relations by provenance, keep dynamic_skip on reopen, refresh saved scripts' titles Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: label the pipeline + buttons: add data source, add downstream step on hover Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: name a schedule node's cadence, like "Every day at 12:00" Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep the downstream-step pill open while its menu is, without flicker Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: room between pipeline node chips and the node's edge Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: refuse renaming a pipeline draft out of the pipeline's folder Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: title pipeline schedule nodes with the schedule's summary Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * revert: keep operators out of AI sessions That change is its own PR; this one is about the navigation. `prefersSessionHandoff` takes the role again and every entry point passes it, the sessions page refuses an operator arriving by URL, and the layout leaves them on the docked chat. Two clauses are new rather than restored, because the nav rework is what gives an operator a sidebar at all: the Workspace/AI Sessions switch and session mode are both hidden from them, since the page behind them declines. `askAiShown` gains the operator case for the same reason — they keep the docked pane whatever the beta gate says, so the sidebar has to offer a way into it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: assets-only view of a pipeline, with producer and trigger chips Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: default schedule handlers, unique trigger draft paths, zoned cadence, valid script names Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: setup banner without icon, button actions, plainer activity rows Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: assets-only trigger chip under the title, editable and selectable Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: await schedule defaults before confirming, refuse colliding trigger renames Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: size assets-only cards to their content Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: pipeline draft sync status beside the folder name in edit mode Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: autosave edits to a deployed pipeline script from the first keystroke Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: show schedule handlers once their defaults load, draft suffix on static chips Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep the pipeline folder while the page tears down on leaving Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: discard the open pane's autosaved edits with Discard all Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: pipeline node context menus, add-at-cursor, and assets-only delete Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: borderless pipeline cards on the main surface, like flow steps Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: remember which pipelines are in the assets-only view Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: edit an asset's producing script from its pane and multi-script chip Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: more room between pipeline layers for the bottom + button Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: background on the trigger chip only when it opens an editor Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: assets-only delete in both modes, why it is blocked, and a Show scripts toggle Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: remove the script before its trigger, archive it for non-admins Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: scope assets-only memory by workspace, no delete for operators, flow reason Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: fly in the pipeline node menus Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: size the Show drafts toggle like the other header buttons Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: a No asset node for scripts that build nothing, in the assets-only view Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: describe a pipeline edge's relationship on hover Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: name assets-only edges by what is built from what Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: open an asset's upstream trigger from its pane header Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: reopen the edit-mode pane when coming back from View Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * style: no red border on errored pipeline nodes Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: Fix actions on misconfigured pipeline nodes, and trigger toggles on edges Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: folded errors and edge toggles in assets-only view, preset writer wizard Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: flag pipeline scripts nothing starts, with schedule/webhook/remove fixes Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: an unwritten subscription is fixed on its script only, by removing it Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: hide subscribed-only unwritten assets, review fixes for the edit pane and edge actions Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: an unwritten subscription is the subscriber's error, not the asset's Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: a script only started by reads of unwritten assets is flagged too Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: no add-downstream + on assets nothing in the workspace writes Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: Save all treats a draft identical to the deployed script as saved Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: remove-from-View opens its dialog, error chip fits, no link to an undrawn asset Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: drag a script's dot onto an asset to subscribe it; unbuilt asset opens its script Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: toggle to hide reads that don't trigger runs in the pipeline graph Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: drag an asset's + onto a script or the canvas; tell reactive assets-only edges apart Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: a programmatic on/mute line joins the existing ones in the header Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: address review — live-version save check, bounded writer lookups, script-only trigger writes, dbt sources, hidden pane Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: review — compare every deployed field, passive-read filter only with its toggle, drag cancel, asset escape hatch Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: keep every control the band took over reachable Four controls the page-header conversion could put out of reach, each found by a review pass and reproduced in the browser: - The flow editor's preview buttons. `topbarWidth` is bound only in the branch a full-page editor does not render, so `compactTopbar` was permanently false there; below the 800px phone bar `phoneTopbar` dropped the buttons inline and the graph overlay never took them, leaving FlowPreviewButtons mounted nowhere and step testing, whole-flow testing and "Test flow & record" no-ops through optional calls. It now measures the header the way ScriptBuilder and AppEditorHeader do, and the inline and overlay conditions are exact complements. - The pin on a deployed app. The band lands over the corner handle that called it down, so the pointer that arrives there hits the band. The band carries the pin itself now, on the same pixels, in place of the sidebar handle it hides. - The detached sidebar's own menus. They portal to `body`, so moving into one left the card and closed it 200ms later, sliding it out from under the menu being read. The card holds while the pointer is in a menu and takes the close from the leave of that. - A deployed app's name for a read-only viewer. The breadcrumb was registered behind write access, so without it the trail fell back to the section name "Apps"; and the Edit button rode `afterName`, which the band drops for an embed along with the trail. The name is registered for whoever opened the app, and what sits after it survives `navHidden` as the page's own control. navPageFor gets a table test: it answers both the sidebar links and the breadcrumb label, and its longest-prefix and first-segment rules are what made the "Apps" fallback look like a correct answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: drop what the operator revert and this refactor left behind The operator tour's last step named AI sessions among the pages the sidebar opens, which operators have no way to reach. The nav handle's comment named the AI chat layout's menu button as a second way into the card; that button and its prop are gone, and the corner control on a page that hides the band is what stands in for the handle now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * style: prettier the three files this branch left unformatted `prettier --check` fails on all three at the previous head. Whitespace only, apart from one Button tag that fits on a line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: assets-only nodes name their script in a header tab instead of a side chip Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: no loading spinner in the pane while a clicked script loads Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: no subscribe drag dot in view mode Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: opening a draft no longer autosaves; header tab styling, selected edge, taller cards Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * revert: clicking an unbuilt asset opens the asset again, not its script Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: header tab sets card width, edit icons on script header and trigger chip, no pane header shift on open Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: no trash in the pane header; card width ignores the run chip so a click never re-lays the graph Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: taller trigger chip; node menu and guard badge on the card corner, below the header Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: give a page without the band's chrome what the band was carrying Second review round, three reachable defects plus one decision: - A public raw app was 150px tall. The bundle's iframe is `h-full`, and dropping its `min-h-screen` left nothing to resolve against on the share, embed and custom-path routes — none of which has a layout above them. Measured: 150px in a 900px viewport. The floor goes back for those routes and stays off inside the workspace, where the layout hands a height down and `min-h-screen` would overflow it by the band. - The band retracted out from under its own open menu on a deployed app. Last round taught the sidebar card to hold while the pointer is in a portaled menu and left its sibling alone, but the band opens menus too — the workspace disc, the fork family, every path level. One helper now, used by both closers. - `bandPeeked` outlived the page it belonged to. Clicking a control in the band navigates away, and the next page may not be one the band floats over, so the hover handlers that would clear it are gone — leaving the band already down when a full-bleed page came back. `beforeNavigate` clears it. - An embed has no band to carry a control, so the app's Edit floats over the canvas there, where it sat before the band existed. Registering it as the band's `afterName` under `navHidden` instead would have cost the app 44px of the iframe for a row holding one button. Reading "is the workspace navigation hidden" now has one definition (`isMenuHidden`), since a page has to answer it too: the flag is sticky inside an iframe, so the raw query param is the wrong thing to read. Also inset the band by the rail only when there is a rail — an embed reserved its width and crowded its own controls into a compact layout early. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: a path that names nothing yet has no breadcrumb leaf `splitItemPath` documents that it gives up on a path still being typed, but `f/demo/` is three parts and returned a leaf with an empty name — a blank segment in the trail. The editor binds the path live, so the breadcrumb sees every keystroke. Tested here rather than `navPageFor`'s sort: no two `NAV_PAGES` paths nest, so the longest-prefix case that test claimed to pin has only one candidate. Renamed it to what it actually checks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * style: prettier the four remaining files this branch left unformatted All four fail `prettier --check` on the branch and pass at the merge base. `raw_apps/utils.ts` also fails but already did at the base, so it is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: pipelines open in the assets-only view by default Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: a No asset node per script, banner-only when writes start it; no On asset change chip Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: delete a node's script, trigger, or both from its menu; red header for a script with errors Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: an errored script header stays red when selected Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: coloured run badge border; per-script delete when no combined delete; run button centred on the card Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: Add data source pill sits one small gap above the graph Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: Multiple triggers chip picks the trigger to edit; each folded trigger deletable from the node menu Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: add-step menu labels its path field Path Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: iterate NodeLists with Array.from so svelte-check passes Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: no trigger delete for flow-built assets; Multiple triggers chip shows the one being edited Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix: Archive button for archive-only deletes, single delete doc, inert Multiple triggers label in View Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat: the raw app editor fills the band like the other three It was the one editor route still rendering its own bar under the band, so it showed two header rows and the band named the app "Apps raw" — `navPageFor`'s first-segment fallback, because nothing on the route registered an item. Its top bar now becomes the page header the way the script, flow and low-code app editors' do: the path and summary are the breadcrumb's, the file-sidebar toggle, autosave cloud, presence mark, menu, Diff, Jobs, export, AI and Deploy are the header's actions, and the path is a plain label since this editor renames from its deploy drawer. The actions render under the band, outside the subtree `RawAppEditor` declares the operating workspace in, and `OpenInSessionButton` asks that context who the acting user is — so the workspace travels with them. `operatingWorkspaceContextEntry()` hands it over without exporting the key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep the folders a half-typed path already names The trailing-separator guard gave up on `f/demo/` entirely, which dropped the whole trail to the kind chip and brought `f/demo` back on the first character of the name. The levels the path does name come back now; only the leaf waits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: pin holds the first slot, and the nav table says what it shares Pinning brings the sidebar's handle back, and from behind it the unpin button sat one slot over from where the pin had just been clicked — so a second click in the same place opened the sidebar instead. An operator is detached by default, which makes that the default state of the feature rather than an edge case. `NAV_PAGES` claimed one table stops a page being called one thing in the nav and another in the header, which holds for the seven entries the sidebar's main group is built from and not for the four grouped separately — `/routes` was "Triggers" in the header and "HTTP" in the nav. The comment now says which is which, and that entry is named for the page rather than its group. `sidebarPageAllowed` gets a table test: it decides what an operator sees, and its `=== true` is what a future `Option`-style default would silently change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: the band names the path Deploy will create, and keeps the summary readable A brand-new app is parked at a `draft_<uuid>` placeholder, and the band named that rather than the path its Deploy will create — `newEditedPath` refuses the placeholder for exactly this reason, and the editor's own bar read it. Both app editors register it now. Verified: the trail reads `u/admin/gentle_app` while the URL is still the draft. The band's summary gets the `title` the editor's own bar had, so one the bar truncates can still be read on hover. The wrapper that hid AppExportButton on a phone never hid anything: the component draws no trigger — it is the export drawer, which portals to `body` — so the class toggle was inert in both states and the comment read as if something moved. Dropped from both app editors. Two comments corrected: `NAV_PAGES`' `icon` does reach the breadcrumb for all eleven entries (`NavBreadcrumb` renders `routePage.icon`), so only `operatorKey` goes unread for the four the sidebar groups separately; and the raw editor's phone note described folding that does not happen — the file-sidebar toggle stays in the band, Jobs and the export are already in the menu, and AI is what stands down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: an item the picker files under no kind keeps its name in the band The merge brought main's agents detail page, which reaches DetailPageHeader with no `errorHandlerKind` — and the band's item required one. `WorkspaceItemKindIcon` falls through to the script glyph for anything it does not know, so defaulting would have labelled an agent a script; the kind is optional now and the trail names the item without claiming it is one of the three. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: the file sidebar's toggle leaves the band The band is the workspace's header, not the raw editor's layout controls. ⌘B still folds the file sidebar, and the editor's own bar keeps the button where it draws one — a session pane. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: the scroll edge stops measuring, and the comments describe the code The band's edge listener captures every scroller in the app — Monaco, the audit table, a run's logs — about once a frame while any of them moves, and it took two `getBoundingClientRect()` reads before looking at `scrollTop`. It now answers from `scrollTop` first and only measures when that answer differs from what the band draws, which is the transition rather than the steady state. The rect check still authorizes every change, so a scroller that does not start at the content's top still cannot claim the edge. The two agent routes main just added size themselves to the viewport, which is 44px more than the box the band leaves them. They measure 856 either way — the layout's `min-h-0` column clamps them — so this is the sweep the rest of the branch already had, making the height follow the box rather than depend on a parent's clamp. Six comments described the conversion rather than the code, which AGENTS.md calls out by name. Each now states the constraint a reader could break: why the summary keeps its popover here, why the audit hint is `afterName` and not an action, why a run registers a `section` rather than an item, why the Runs filters are `actionsFlexible`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Revert "fix: the file sidebar's toggle leaves the band" This reverts commit |
||
 hugocasa
|
ede4103b00 |
feat: share AI guidance with the CLI skills and lint flow groups (#11397)
* feat: lint AI agent tool names and flow groups in wmill lint * refactor: assemble chat and CLI AI guidance from one topic table * feat: share flow groups, reuse and pipeline guidance with the CLI skills * feat: share raw app, data table and secret guidance between chat and CLI * docs: document the shared AI guidance source for contributors * fix: keep wmill lint running on flows with malformed collections * fix: reject skill descriptions that are not plain YAML text * fix: tighten fence typos, script base scope and app prompt order * fix: skip tool name checks on agent steps linked to a saved agent * fix: catch any misspelled prompt fence and soften the tool name claim * fix: align cli eval harness with the files and steps wmill init adds * fix: drop cli eval checks that expect unrequested deploy commands * fix: list ansible as mainless and c# Main in script base guidance |
||
|
Ruben FiszelandClaude Opus 5
|
c6e0302d7c |
feat: let // materialize declare a dbt:// warehouse-relation write (#10978)
* feat: let `// materialize` declare a `dbt://` warehouse-relation write `// materialize manual dbt://<warehouse>/<schema>/<name>` lets an ingestion script in any language declare that it writes a warehouse relation, so it and the dbt model reading that relation land on one asset node instead of two disconnected pictures. `manual` is the only mode a warehouse target has — nothing generates warehouse DDL — and the non-`manual` spelling is refused rather than silently degraded. The `<warehouse>` segment is resolved against the workspace's configured warehouses, like a descriptor's `profile.warehouse`. The run records the same `materialized_partition` row a DuckLake target does, from the generic job path rather than an executor: the DuckLake write engine is DuckDB's, this declaration is anyone's. With a non-dbt producer now possible, the blanket deploy-time refusal of `# on dbt://<relation>` narrows to the shape that still cannot fire — every writer of the relation being a dbt script, since a dbt run does not dispatch. "Nothing produces it yet" stays accepted, as for every other asset kind, so deploy order does not matter. A dbt script may not subscribe at all: its graph ingest clears its own `dbt://` trigger rows. The one ordering the deploy cannot catch — a subscription accepted before any producer, then claimed by a dbt project — is named in that project's deploy log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Rw1WrKeRRzyYHjfkuB83ek * fix: address review — preview stamping, stale producer set, public doc Three findings from the local review round: - Record the warehouse write only for a DEPLOYED script job. The annotation is a deploy-time contract (`manual`, three segments, a configured warehouse) checked where write access to the path is also required; honouring it in a preview, hub or inline-flow body let `jobs:run` alone restamp any relation's last writer from a script that never touched it. - Exclude the deploying script's own rows from the producer set. Read committed, they describe the version being replaced, so a script dropping its `// materialize` while adding a subscription counted itself as the producer that would wake it and committed a dormant edge. It could not be that producer anyway — the dispatcher skips self-loops. - `AssetKind::Dbt`'s doc no longer claims dbt is the exclusive producer of a warehouse relation, on both the types and the parser enum. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: review round 1 — dbt-script materialize, set-form rule, doc - Refuse `// materialize` on a dbt script, the producer half of the rule the trigger loop already applies to `// on`: the graph ingest republishes that path's asset rows wholesale, so a declared write is wiped by the deploy that accepted it while its runs keep stamping the relation. - `dormant_dbt_subscriptions` now spells the same predicate its singular sibling does: the producer set has to be non-empty (nothing produces it yet is deploy order, not a dormant edge) and excludes the subscriber's own path (a script never wakes itself). Both divergences are pinned by tests. - The docs no longer claim the dbt deploy log covers a native producer that drops its `// materialize`; it does not, and nothing else reports that case. - An integration test over the deploy contract, since only a real deploy proves the handler feeds `sole_dbt_producer` the canonical key `asset.path` holds — the spelling that has to agree across the materialize target, the `// on` ref and the refusal that joins them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: qualify the any-language claim, and pin the dbt-script refusal `AssetKind::Dbt`'s contract (both enums), the two runtime guides and the deploy comment said a script of any language may declare a `dbt://` write, which the dbt-script refusal added last round contradicts. They now say "any language but dbt's own", with the reason: a project's writes are read from its manifest. The deploy-contract integration test covers that refusal for both annotations. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: teach the pipeline AI guidance the warehouse-relation target The pipeline prompt (both sources, plus the regenerated bundle) told the model `// materialize` is DuckDB-only and rejected on any other target, which now steers users away from the very thing this PR adds. It distinguishes the managed DuckLake write, still DuckDB-only, from the warehouse-relation declaration any language but dbt's own may make. `dbt_manifest.rs`'s module doc carried the same "the only thing that creates one" overclaim the other four sites lost last commit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: draw an explicit dbt:// subscription on the canvas The editor suppressed every `// on dbt://…` overlay, which was right while the deploy refused all of them. It now refuses only a relation dbt alone builds, so the suppression hid the author's own annotation for exactly the case this PR adds — a subscription woken by a native `// materialize manual dbt://…` producer. The deploy stays the gate. Also the two stale claims round 4 named: the live pipeline prompt dropped the dbt-script exception the base prompt carries, and the doc's e2e requirements still said every `dbt://` subscription is refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse `// data_test` beside a `dbt://` materialize target `// data_test` checks are verifier probes the DuckDB executor splices around a managed write. A warehouse relation is written by the script itself, in any language, so nothing would run them — and unlike the DuckLake `manual` case, which at least fails loudly in that executor, a declarer in another language deployed green with its data-quality assertions silently skipped. Covered in the deploy-contract test and documented beside the annotation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: exclude a renamed producer from the sole-dbt producer set The producer set already excluded the deploying script's own path, because its committed rows describe the version being replaced. Under a rename the write sits at the OLD path — still committed, and removed by the same uncommitted transaction — so a producer renamed while it drops its `// materialize` and adds `// on dbt://…` still counted as the producer that would wake it, and committed a dormant edge. The deploy-contract test covers it: without the exclusion the rename deploys 201 instead of being refused. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: take the rename test's parent hash from the create response `format!("{:x}", …)` over the stored i64 drops leading zeros, while `ScriptHash`'s deserializer hex-decodes and demands 8 bytes — so a hash below 2^60 would 422 the request instead of reaching the refusal it asserts on, on roughly one in sixteen spellings of that script body. The create response already carries the zero-padded form, as the rest of the suite uses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: state the concurrent-ingest interleaving honestly `sole_dbt_producer`'s doc claimed the concurrent-deploy race only ever resolves toward refusing. It does when the uncommitted producer is native; when it is the dbt ingest, the check sees an empty producer set and accepts, and if that ingest then commits and runs its warning query before the subscriber's trigger row lands, neither side reports the dormant edge. Not serialized: the two would have to share a per-relation lock, and the ingest takes `script … FOR UPDATE` before its own advisory lock, so a deploy holding relation locks first inverts that order into a cross-subsystem deadlock — a worse failure than the cosmetic edge. Recorded beside the other orphaning the deploy cannot catch, with the bound both share: the next deploy of that project warns. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse a `dbt://` subscription that is not a whole relation `# on dbt://main/analytics` deployed and persisted a trigger row. Every producer spells `<warehouse>/<schema>/<name>` — the manifest ingest derives it from `relation_name`, a `// materialize` target is checked against it — so a partial one is an edge nothing can ever wake, which is what the dbt-only refusal exists to prevent. The shape now has one definition (`is_full_relation_path`) that both halves of the deploy ask, rather than a segment count spelled twice: a subscription and a write that disagreed would refuse and accept the same string. Also rewrites the canvas test's comment as a current constraint per AGENTS.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: hold both halves of the deploy to one `dbt://` relation validator A subscription checked the relation's shape but not its warehouse, so `# on dbt://<unconfigured>/<schema>/<name>` deployed and persisted a trigger row for something no producer can ever write: the write side refuses that exact string, and a dbt project's `profile.warehouse` resolves against the same config, so no later deploy fixes it and the dormant-edge warning cannot report it either. The shape rule and the warehouse rule now live in one `validate_dbt_relation` that both halves call, rather than being spelled per site — the previous two rounds each closed one half of one rule, which is the drift that invites. Also moves the parser test out from between a comment and the test it documents, and names both refusals in the doc's list. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: drop the subscription-only clause from the shared refusal message "so nothing can produce it" reads backwards on the `// materialize` side, which is the producer. The remaining sentence says what is wrong on both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound a `dbt://` relation by the asset-path column in the shared validator `asset.path` is VARCHAR(255) and the manifest ingest drops a relation that outgrows it rather than failing the whole graph, so past the column no producer row can exist on either side. `script_trigger.trigger_ref` is unbounded text, so an overlong subscription deployed and stayed dormant for good; an overlong write reached Postgres and failed the deploy on a `value too long` instead of a message. Both now refuse in the validator the two halves share, against the ingest's own constant. The integration case computes the ref from that constant so it cannot drift back under the bound. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report a warehouse-lookup failure as the failure it is, and correct the boundary `dbt_warehouse_exists` fails three ways — no such warehouse, the query itself, and a setting with no `resource_path` — and all three became a 400 blaming the user's warehouse name. A pool timeout mid-deploy told a retrying sync that a transient server error was a permanent client one. Only `NotFound` is the annotation's fault now. The known-boundary paragraph claimed a flow-runner run still cascades. It does not: it is routed by `flow_step_id`, which `is_eligible_kind` rejects, as `asset_trigger_dispatch.rs` pins. Recording and cascading are decided separately, so the paragraph now names all three routes rather than merging two of them — and the row it omitted, an ordinary flow step, which records and never cascades. E2E item 7 said "deployable" where the rule is "wakeable": with only the dbt project reading the relation the producer set is empty, which deploys fine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct two rationales the last commit got wrong `Error::SqlErr` already maps to 400 in this codebase, so the query case's status was never the thing at stake. What the `NotFound` match earns is that a query failure and a malformed setting stop being described as an unconfigured warehouse name, and that the malformed-setting `InternalErr` reaches its own 500 instead of being flattened. And a flow step is two shapes, not one: a step running a deployed script is a `Script` job that records and never cascades, while a step with an inline body is `FlowScript`, which the recording guard excludes along with previews. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: warn about dormant subscriptions from the run that publishes ownership too A run whose static descriptor finds its profile moved re-ingests the version's graph and republishes path ownership, exactly as a deploy does — so it can be what leaves a subscription accepted while the relation had no producer with dbt as its only one. That path discarded `persist_ingest`'s result and emitted no warning, which also made the doc's enumeration of unreported orphanings wrong. Both ownership-publishing points warn now. An agent worker still cannot: it reaches these tables only through the API and its ingest publishes without reading back, which the doc now says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: an agent run publishes no ownership, and the warning has two callers The agent-worker sentence called it an exception that publishes ownership without warning. It publishes none: `Connection::Http` forces per-run models, and `publishes_ownership()` is the negation of that, so an agent stores a job-pinned snapshot and leaves workspace ownership with the deployed graph — it cannot orphan a subscription at all. `warn_dormant_subscribers`' own doc still named the deploy log as the only place the warning shows, one commit after it gained its second caller. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: stop the managed-write rule from contradicting the dbt:// target The sentence after the warehouse-relation paragraph says `// materialize` means the runtime writes the table for you and the body is a bare SELECT. That is the managed DuckLake rule, written before a `dbt://` target existed, and unqualified it tells the model the opposite of what the paragraph above it just said — a model following the more prominent one emits a SELECT for a warehouse relation, which deploys and then writes nothing. Both prompt sources now scope it, and both name the `// data_test` refusal beside a `dbt://` target, which the badge list advertised without the caveat. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
1f912f4104 |
feat(ai): enable data pipelines in AI sessions with alpha notice (#10273)
* feat(ai): enable data pipelines in AI sessions with alpha notice * test(ai): assert real pipeline instructions returned in session * fix(ai): await pipeline editor registration in open_preview to end build_pipeline_node race * test(ai): drop historical narration in pipeline instruction assertion * docs(ai): clarify S3 asset-URI storage forms and literal-key rule in pipeline prompt * docs(ai): document data_upload S3Object input mechanism in pipeline prompt * test(ai): guard open_preview pipeline tool-registration wait seam * feat(ai): report inferred asset lineage in build/edit_pipeline_node results * test(ai): assert open_preview(pipeline) awaits async handler registration * docs(ai): qualify S3Object as wmill.S3Object in pipeline prompt examples * fix(ai): report failure when a backgrounded session's pipeline tools never register * fix(ai): include output_kind-seeded outputs in build_pipeline_node lineage feedback * fix(ai): report only body-detected lineage, never the output_kind seed placeholder * fix(ai): include materialize target in lineage feedback, condense comments * docs(ai): use default-storage s3:/// in canonical pipeline read example |
||
|
Ruben FiszelandClaude Opus 4.8
|
0819641f3a |
feat(ai): improve data-pipeline building in AI sessions (prompt + evals + e2e) (#10270)
* test(ai_evals): pipeline coverage for AI sessions + editor e2e Add a complex incremental DuckLake pipeline case, harden the two-node case, and encode the declarative pipeline contract (`-- on` triggers, `-- materialize` + bare SELECT) in the pipeline judgeChecklists so the LLM judge stops false-negativing correct nodes. Add a deterministic Playwright e2e that seeds annotated pipeline scripts and asserts the /pipeline/<folder> editor derives the lineage DAG. Fixes WIN-2229 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): steer pipeline chat to DuckDB+materialize and warn on missing storage The pipeline authoring prompt (getPipelinePrompt, used by the global/session chat and the /pipeline editor) was neutral on language choice and said nothing about storage readiness. Default it to duckdb materializing into DuckLake unless the work specifically needs postgres/data-tables or bun/python, and add a storage prerequisites section: a DuckLake pipeline needs workspace object storage + a DuckLake catalog, so warn when none is configured and give role-appropriate next steps (admin: workspace settings; others: ask an admin). Drafting is not blocked. A/B on sonnet (global pipeline cases): no regression, +639 finalContext tokens. Fixes WIN-2229 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai_evals): address review - e2e teardown, async-edge note, merge-mode hedge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai): address review nits - drop phantom list_ducklakes tool ref, trim narration comments Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): add list_ducklakes chat tool for pipeline storage readiness The pipeline counterpart to list_datatables: lists the workspace's configured DuckLake catalogs so the chat can detect the storage prerequisite before building a DuckLake pipeline and warn with role-appropriate next steps when none exists (drafting stays unblocked). Wired into the global tool set and referenced from the pipeline authoring prompt. In the eval run all three pipeline cases called it unprompted with no build regression. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai): fix DuckDB annotation syntax in duckdb-default section (-- not //) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
68debab877 |
feat(triggers): add AMQP (RabbitMQ) trigger via lapin (#10230)
* feat(triggers): add AMQP (RabbitMQ) trigger using the lapin library Fixes WIN-2214 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(triggers): defer AMQP cross-workspace deploy pending utils-internal publish Revert the amqp_trigger additions to the shared windmill-utils-internal TriggerDeployKind and the frontend cross-workspace deploy adapter: the frontend installs the published npm package, which lacks the new kind until a release is cut. AMQP create/edit/delete/list/sync/capture are unaffected (they use local types); only cross-workspace deploy/merge of AMQP triggers waits on the package bump. Also document the at-most-once ack in the consumer loop. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): address AMQP review — at-least-once ack, workspace cascade, contracts - ack AMQP deliveries only after successful dispatch; nack+requeue on failure - add ON DELETE CASCADE workspace FK so amqp_trigger rows are cleaned on workspace deletion (and the listener stops) - fix the /amqp_triggers/test OpenAPI body and add amqp_trigger to WorkspaceDiffRow.kind - register AMQP in the generated workspace trigger tool (create_trigger) - drop banned $bindable defaults on optional props in the config section - add build_uri unit tests (encoding, ports, vhost) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): stop AMQP poison-message loop and reconnect on transient drops Chaos testing against a live RabbitMQ broker showed the previous nack(requeue) + immediate re-poll spun a tight redelivery loop (~1000 critical-error reports/sec) on a poison message, and any connection blip permanently disabled the trigger (lapin has no built-in reconnect). - on dispatch failure: nack+requeue then stop consuming; the listener framework re-lists the trigger after its ping goes stale (~15s), backing redelivery off to that cadence instead of a tight loop (verified: rate dropped from ~1000/s to ~1 per ~26s, message preserved) - on connection/stream error: stop and let the framework reconnect instead of disabling; persistent failures are still disabled via get_consumer (verified: a forced connection close now auto-reconnects and resumes) - finish the AI create-trigger action wiring for AMQP: add amqp to CreatedResourceTriggerKind, the action-card registry, and the drawer registry so the result card renders and its "Open" action works Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP frontend registries and defer merge rows - add amqp to capturableTriggerTypes (so AmqpCapture mounts), the Runs jobTriggerKinds filter, and CLOUD_DISABLED_TRIGGER_TYPES - wire AMQP into global AI chat mode: TRIGGER_KINDS, the request union, writeTriggerSchema, triggerServices, and the draft adapter - stop emitting actionable AMQP fork-comparison rows (revert amqp_trigger from TRIGGER_OR_SCHEDULE_TABLES) since cross-workspace deploy is deferred until windmill-utils-internal is published — avoids a deploy that fails with "Unknown kind: amqp_trigger" - use design-system TextInput instead of raw <input> in the config section Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP session/draft registries and constrain prefetch - add amqp to the session-deploy, draft-compare, preview-router, and copilot workspace-item registries so AMQP drafts/deploys/nav/path resolution work - include amqp_count in the MoveDrawer attached-trigger rename warning - replace the raw prefetch <input> with a design-system TextInput bounded to an integer 1-65535 (backend u16) and block save on invalid values Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): make AMQP disconnect/reconnect consistent with the Kafka trigger lapin, like rdkafka, has no transparent reconnect, so the AMQP listener now mirrors the Kafka trigger's explicit reconnect loop instead of relying on the framework re-list (which disabled the trigger once get_consumer failed on a sustained outage): - get_consumer returns cheaply; consume owns a (re)connect loop that retries with a 30s backoff, reports a critical error every 10 failed attempts, and reports a recovered critical error once it reconnects — never disabling the trigger on a connectivity failure - a consumer/stream error breaks out to reconnect rather than disabling - dispatch failure still nacks+requeues (at-least-once) with a short backoff to avoid a tight poison-message loop, keeping the connection alive Verified against a live RabbitMQ broker: killing the broker keeps the trigger enabled and retrying (attempt N), and restarting it auto-reconnects (logs "reconnected after N attempts") and resumes dispatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): complete AMQP capture registries and constrain prefetch contract - add the 'amqp' case to triggerKindToTriggerType so opening the AMQP editor from a capture button no longer throws "Unknown TriggerKind: amqp" - register AmqpIcon in CaptureTable's icon map and add an AMQP entry to the script/flow CaptureButton menu - bound the OpenAPI prefetch_count to an integer 1-65535 (matches the Rust u16) and regenerate clients/prompts - require a non-empty exchange name when the exchange binding is enabled - build_uri: fall back to "/" on a blank vhost and bracket IPv6 hosts (+ tests) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(triggers): wire AMQP into pipeline graph, git-sync, and preprocessor types - asset_graph: discover attached amqp_trigger rows and emit an AMQP TriggerEdge so AMQP triggers render (and can be opened/deleted) on the data-pipeline canvas - frontend pipeline graph: add amqp to NativeTriggerKind, the add-trigger menu, node presentation, event-trigger set, annotation keywords, and the editor/service registrations - git-sync: add the amqp_trigger include pattern (+ test) so an AMQP git-sync deployment stages only its .amqp_trigger.* file, not an unrelated same-path object - preprocessor starters: add the AMQP event to the generated TS/Python/PHP trigger event types (kind/payload/exchange/routing_key/queue_name/redelivered/ delivery_tag) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): finish AMQP pipeline/parser wiring, prefetch validation, source lists - fix a stray edit that corrupted the pre-existing MqttTriggerEditor import ($lib/... path) in PipelineTriggerEditors.svelte - reject prefetch_count = 0 server-side in validate_config (RabbitMQ treats 0 as unlimited) and defensively skip basic_qos(0) in build_consumer (covers the capture path that bypasses CRUD validation) - recognize `// on amqp` in the canonical parser (TriggerSpec::Amqp) and add amqp to the CLI non-autorun/event-trigger sets so a pipeline cascade never runs an AMQP-only node as a manual root without an event - add amqp to the preprocessor intro lists and both pipeline AI instructions Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(triggers): reject zero AMQP prefetch in all paths and finish guidance lists - extract a shared validate_amqp_options used by both CRUD validate_config and build_consumer, so capture configs (which bypass CRUD validation) also reject prefetch 0 instead of silently connecting with an unlimited buffer (+ unit tests for 0/1/65535/None) - add AMQP to the main script-writing preprocessor-sources prompt and the CLI triggers-skill guidance list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(triggers): de-duplicate AMQP prefetch comment and fix GET response text - keep the zero-prefetch rationale only on the shared validate_amqp_options doc; drop the redundant call-site comments - correct the getAmqpTrigger OpenAPI 200 description ("deleted" -> "retrieved") Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: update ee-repo-ref to aaa6cb89b05b76139252c64f057e53b94d12ac60 This commit updates the EE repository reference after PR #680 was merged in windmill-ee-private. Previous ee-repo-ref: 5da5fd65aca9594b2611837a52e4677b544b0380 New ee-repo-ref: aaa6cb89b05b76139252c64f057e53b94d12ac60 Automated by sync-ee-ref workflow. * chore(migrations): consolidate the four AMQP migrations into one The table and the three enum ADD VALUE statements (trigger_kind, job_trigger_kind, draft_kind) are one atomic feature. ALTER TYPE ... ADD VALUE runs inside the migration transaction on PG >= 14 (Windmill's minimum) since the amqp_trigger table doesn't reference those enum types, so they can share a single migration instead of four. Verified applying cleanly in a single transaction on a fresh DB. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: windmill-internal-app[bot] <windmill-internal-app[bot]@users.noreply.github.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
c91027824b |
feat(pipeline): AI-chat data-pipeline editor (route + in-session) + home surfacing (#9805)
* feat(pipeline): AI chat tools to build pipeline nodes with diff/approval Add a data-pipeline AI chat experience modeled on the flow editor and surfaced through the dev-gated global chat (no new chat panel). The /pipeline editor registers PipelineAIChatHelpers on the AIChatManager; while it is open the global mode layers pipeline tools, a pipeline prompt section, and the helpers on top of the full global tool set (behavior is unchanged when no pipeline editor is open). New tools (frontend/src/lib/components/copilot/chat/pipeline/core.ts): - get_pipeline_graph / read_pipeline_node — read the live graph and bodies - build_pipeline_node / edit_pipeline_node — stage changes as AI-pending drafts - remove_pipeline_node — drop a staged proposal - test_pipeline_node — preview-run a node (requires confirmation) Tools never deploy: they stage drafts flagged aiPending, rendered on the canvas with an accent ring and reviewed via Accept all / Reject all (the flow editor's GlobalReviewButtons). Accept commits the drafts; Reject reverts to a pre-AI snapshot, preserving earlier accepted drafts. Auto-accept is gated on the chat autonomy mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): teach the global/session chat to author data pipelines Without an open /pipeline editor the session chat had no pipeline concept, so "create a data pipeline" loaded flow instructions and built a flow. Add a first-class pipeline authoring path: - system_prompts/base/pipeline-base.md — what a data pipeline is (a DAG of annotated scripts wired by storage assets, NOT a flow) and how to author the // pipeline / // on / // materialize annotations; wired through generate.py as getPipelinePrompt() (regenerated prompts.ts/index.ts). - global/core.ts — new get_instructions subject "pipeline", and a global-prompt rule disambiguating data pipelines from flows so the model routes correctly. - ai_evals/cases/global.yaml — two global cases (single node, two-node chain) asserting pipeline-annotated script drafts and forbidding write_flow, guarding the pipeline-vs-flow conflation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): show & build pipelines in the AI session preview Add a 'pipeline' session preview target so the session AI can show the data-pipeline graph for a folder and build nodes in-pane: - open_preview now accepts kind="pipeline" (path = folder); SessionTarget / EDITOR_TARGET_KINDS widen accordingly. The slot/codec load model stays flow|script|raw_app — pipeline bypasses it with its own fetch/draft state. - New PipelineEditorView.svelte mounts in the session pane: fetches the folder graph, overlays AI drafts, renders AssetGraphCanvas + the Accept/Reject review buttons, and registers PipelineAIChatHelpers on the *session-scoped* manager (via getAiChatManager) so build_pipeline_node / edit_pipeline_node + the diff/approval work inside the session too. - System prompt nudges the model to open the pipeline preview and use the staging tools while building. Verified end-to-end with a real model: the session AI called open_preview, the graph mounted in the side panel, then build_pipeline_node staged a node on the session canvas with its schedule trigger and ducklake output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): share the AI editor logic between route page and session Consolidate the duplicated data-pipeline AI logic onto a single shared layer so the route editor and the in-session preview behave identically and the session gains the full code editor. - New pipelineAiHelpers.ts: createPipelineAiHelpers(deps) owns the propose/edit/ remove/accept/reject/test staging + the per-turn snapshot bookkeeping that powers Reject. Callers inject accessors for their own draft Map and graph. - Route page (/pipeline/[folder]) drops its ~250-line inline AI-helper block and wires the shared factory via deps (folder/workspace/graph/drafts + focus, ensureEditable, run-started). Its shell — persistence, navigation guard, activity, cascade, trigger drawers — is untouched. - Session PipelineEditorView uses the same factory and now renders the real AssetGraphDetailsPane (code editor + live overlays + test), so a node built in a session opens with its source, matching the route editor. Verified: route page hydrates/renders drafts unchanged; in a session the AI opened the pipeline preview, built a node, and its code showed in the details pane. check:fast clean, 197 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): externalize editor state into PipelineEditorState (step 1) Introduce PipelineEditorState — the data-pipeline analogue of the flow editor's flowStore. It owns the draft Map, the live editor overlays, and the selection, with callback-safe methods (handleDraftPersist / handleAnnotationsChange / … ), so a single editor can be rendered by both the route page and the session. This commit lands the store and points the in-session PipelineEditorView at it (no behaviour change — the session already had these inline). Next steps move the route page onto the store and a shared <PipelineGraphEditor>. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): point the route editor at PipelineEditorState (step 1) Move the route page's draft Map, live editor overlays, selection, and the draft-persist / live-change handlers onto the shared PipelineEditorState (`pe`), referencing them as `pe.*` in place. No behaviour change — persistence, graph resolution, run dispatch, AI staging, and deploy all stay on the page and now read/write the externalized state. This is the data-pipeline analogue of the flow editor's flowStore: the route page and the in-session preview now share one source of editor truth, setting up the shared <PipelineGraphEditor> in the next steps. Verified: the page hydrates its DB draft, renders the overlay graph, the toolbar counts (Save all (N)) track pe.drafts, and selecting a node opens it in the details pane. check:fast clean, 84 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the route editor via shared PipelineGraphEditor (step 2) Extract the canvas + details-pane editor body into PipelineGraphEditor.svelte, the data-pipeline analogue of FlowBuilder. The route page now delegates its Splitpanes block to it, passing the externalized PipelineEditorState plus its run/cascade/trigger/deploy callbacks; the component owns pane sizing, selection/details-open derivation, and the canvas+details rendering. Root-caused the earlier ts2769 "$props() No overload" to a prop named `state` colliding with the `$state` rune (`let x = $state(...)` parsed as a store auto-subscription on the prop) — the prop is now `editor`. Net: the route page sheds ~310 lines of template/state; behaviour preserved. Verified: the page hydrates its DB draft, renders the graph, opens the draft in the details pane (live code editor + Test), pane sizing works. check:fast clean, 24 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): move draft autosave into PipelineGraphEditor (step 3) Fold the per-user `data_pipeline` DraftService bundle autosave (hydrate + debounced persist + localStorage crash mirror) into PipelineGraphEditor, gated by a `persistDrafts` prop — FlowBuilder's parameterized-autosave shape. The route page passes `persistDrafts` + `folder` and reads `editor.loadedFromDbDraft` for its AutosaveIndicator; the in-session preview will leave persistence off. Also restores the `untrack(...)` wrapping on the pane-sizing $effect (dropped when the editor body was extracted in step 2). Without it the Pane `bind:size` feedback loops the effect and pegs the main thread when the details pane is closed — a latent hang in the step-2 commit. check:fast clean, 24 pipeline tests pass. Note: browser revalidation was not possible this session (the Playwright MCP browser was reset); the autosave is a verbatim port and the untrack fix is the original working form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the session preview via shared PipelineGraphEditor (step 4) Point the in-session PipelineEditorView at the shared PipelineGraphEditor instead of its own inline canvas + details pane. The session now renders the exact same editor body as the route page — gaining the full details/code pane — while opting out of persistence (persistDrafts=false) and the run/cascade/trigger/bounded affordances (their callbacks are omitted, so those controls hide). Building nodes + the Accept/Reject diff still work via the AI helpers. Also fixes issues surfaced by a full `svelte-check` while wiring this up: - PipelineGraphEditor: edit mode opened the details pane unconditionally (a step-2 regression); restored the route's "open only on selection/draft" behaviour. - Route page passed an `isOperator` prop the component doesn't accept (step-2; caught only by full check, not check:fast). - SessionItemNotFound: narrow its `kind` to exclude `pipeline` (pipeline targets never slot-load, so they can't 404 through it) — closes the SessionTarget-widen fallout. - PipelineEditorView: cast the resolveGraph base to AssetGraphResponse. Full `svelte-check` now clean across all pipeline/session files; 137 unit tests pass. (Browser revalidation still pending — Playwright MCP was unavailable this session; see the smoke-test note on the PR.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): stop an infinite microtask loop when persisting a no-output draft handleDraftPersist short-circuits when the open draft's content + inferred writes are unchanged. The writes check compared `d.outputAssets?.length === writes.length`, but a no-output draft has `outputAssets: undefined` (so `?.length` is `undefined`) while the details pane infers an empty `writes: []` (length 0). `undefined === 0` is false, so it never short-circuited: every persist re-wrote the drafts Map with an equivalent object, which gave `activeDraft.script` a new identity → the pane re-emitted its overlays → the graph re-derived → persist fired again. A self- sustaining microtask loop that pegged the renderer and froze the tab on any pipeline carrying a no-output draft (e.g. hydrating one from the saved data_pipeline draft on load). It hangs rather than throwing effect_update_depth_ exceeded because it cycles across microtasks, not within one reactive flush. Fix: coalesce the undefined length to 0 so "no outputs" compares equal to an empty inferred-writes list. Adds pipelineEditorState.test.ts covering the idempotency (fails without the fix) plus the change/no-change cases. Root-caused by instrumenting the reactive churn: every iteration reassigned drafts/liveContent/liveBodyAssets/liveAnnotations/displayGraph with identical values — pure reference churn off the drafts re-write. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): make the agent open the pipeline editor before building nodes In a session, the GLOBAL system prompt only *advised* opening the pipeline preview ("show its graph with open_preview ... prefer those tools once it is open"), so the agent routinely skipped it: on a plain "build a data pipeline" request it reached for write_script and staged plain script drafts, and the canvas editor never opened. build_pipeline_node / edit_pipeline_node are only registered once the preview is open, so skipping open_preview also loses the canvas-staged Accept/Reject diff-approval flow entirely. Make the guidance imperative: open_preview(kind="pipeline", path=<folder>) is the FIRST step before creating any node (an empty or not-yet-created folder is fine — create_folder first if needed), and pipeline nodes go through build_pipeline_node / edit_pipeline_node, never write_script. This also clears the agent's "the folder might not exist" hesitation that pushed it toward write_script. Verified live (same plain prompt, before/after): before it used write_script with no editor; after, the agent opens the editor first and stages a canvas-highlighted node with Accept all / Reject all. The guidance is gated on previewTools (session-only), so it doesn't affect the non-preview global eval cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): preserve in-session pipeline drafts across editor hide/show The session preview's PipelineEditorState lived in the PipelineEditorView component with persistDrafts=false. Hiding the editor sets editorVisible=false, which makes `hasEditor` false and the `{#if hasEditor}` block unmount the view — discarding its component-local store. Showing it again remounted a fresh, empty one, so the pipeline the AI had built in the session vanished. Move the PipelineEditorState onto the per-session SessionRuntime (like the flow / script / raw_app editors, which already host their state there and take {runtime}), so it survives the pane unmount on hide and across session switches. The runtime is keyed by session id and only dropped on session deletion. Because the instance is now reused, guard against a retarget to a different folder: PipelineEditorView resets the state when `path` changes to a new folder (a same-folder remount keeps the drafts). Adds `folder` + `reset()` to the store. Verified: build a node in a session → Close editor → Show editor → the staged node, its wiring, the details-pane code, and Accept/Reject all re-appear. Full svelte-check clean; 139 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline-ai): clearer diff + persistent review banner on the canvas The AI review affordance had two problems on the pipeline canvas: - The floating Accept-all / Reject-all bar sat bottom-center, where it collided with the minimap once the canvas narrowed on node selection — reading as "the buttons vanished when I select a node". - Every staged draft rendered with the same blue ring, so it wasn't clear what the review would actually change (a plain manual draft looked the same as an AI proposal). Replace the floating bar with a top-left review banner (z-30, clear of the controls and minimap) that stays put regardless of selection and spells out the pending counts. Color the diff per node: a proposal that adds a node that isn't deployed rings green with a "new" chip; one that edits an already-deployed node rings amber with an "edited" chip. Plain manual drafts keep the neutral gray dashed border, so only the green/amber nodes read as part of the Accept/Reject set. aiPendingKind is resolved in resolveGraph (deployed runnable present → modified, else added) and forwarded through the canvas to the node. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): persist in-session pipeline proposals across reload/switch Staged AI proposals lived only in the per-session runtime's in-memory PipelineEditorState (persistDrafts=false), so a page reload — and an LRU-evicted runtime on session switch — dropped them, leaving the canvas and the Accept/Reject review empty even though the chat still showed the nodes as staged. Enable the same per-folder DB-draft persistence the route page uses for the in-session editor. To keep hide/show cheap and race-free, hydration is now gated per editor instance (PipelineEditorState.hydratedFromDb) rather than per component mount: the runtime-hosted instance hydrates ONCE when fresh (reload / evicted runtime) and then keeps its in-memory drafts across the editor pane unmounting on hide — re-reading the DB on every remount would race a not-yet-flushed autosave and drop a just-staged draft. A folder retarget resets the flag so the new folder re-hydrates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): make Reject all work for rehydrated proposals rejectAll only reverted paths tracked in the in-memory aiSnapshots map, which is rebuilt empty on each editor mount. After a reload (or session switch into a fresh runtime) the proposals are restored from the persisted draft but have no snapshot, so Reject all was a no-op on exactly the nodes it should discard. Sweep any still-pending draft without a snapshot and discard it (revertPath with no snapshot deletes the path; for an edit of a deployed node that correctly falls back to the deployed body). Adds unit coverage for accept/reject including the no-snapshot case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep proposals visible while the graph reloads on switch The session editor pane is LRU-capped (MAX_WARM_EDITORS), so returning to a session whose pane was evicted remounts PipelineEditorView with a fresh graphRes resource (loading=true, current=undefined). The deployed-graph loading spinner gated the whole canvas, so the staged proposals and the Accept/Reject review banner vanished until the re-fetch resolved — read as "the proposal disappears when I switch sessions". Only show the loading/error placeholder when there are no drafts to display. When the runtime already holds staged drafts, render the editor immediately: resolveGraph overlays them on an empty base so the proposals + banner stay visible, and the deployed nodes fill in when the fetch completes. Verified with a 4s-delayed graph fetch — proposals render through the load with no spinner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): apply AI node edits directly as drafts, no approve/reject The canvas-level Accept all / Reject all review (aiPending proposals, the green/amber diff ring + "new"/"edited" chips, and the review banner) didn't fit the pipeline editor. Match the flow/script editor instead: build/edit apply directly as ordinary unsaved drafts on the canvas, which the user then deploys — there is no separate approval step. Removed across the surface: - aiPending / aiPendingKind on the runnable node + resolveGraph seeding + canvas forwarding; AI-built nodes now render with the existing plain unsaved-draft dashed styling. - the review banner, count derivations, and hasAiPending/onAccept/onReject props from PipelineGraphEditor and both consumers (route page + session view). - acceptAll/rejectAll/hasPending and the per-turn snapshot bookkeeping from the shared helpers; removeProposedNode now just discards the unsaved draft at a path (undo a build). acceptAllProposals/rejectAllProposals/ hasPendingProposals dropped from the PipelineAIChatHelpers interface and the manager's auto-accept hook. - accept/reject language from the tool descriptions, return messages, and the system-prompt section. Tests updated; pipeline + AssetGraph suites pass (142). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts-diff): support data_pipeline diffs + fix blank empty-summary row Two issues in the session "Drafts" diff drawer (DraftDiffDrawer): - Clicking a `data_pipeline` bundle row threw "Draft diff not supported for kind data_pipeline" (utils_draft_deploy.ts) — there was no handler for the kind, so it fell to the OVERLAY_GETTERS lookup and errored. The bundle has no deployed counterpart (each node deploys individually as a script), so diff it node-by-node: surface each node's draft body keyed by path, folding in the deployed body as the "before" when a node edits a deployed script. - A draft row whose summary is an empty string (e.g. the app draft) rendered with no title at all: WorkspaceItemRow's single-line branch used `summary ?? secondary`, and `??` doesn't treat '' as absent, so it showed the empty summary instead of the path. Use `||` so an empty summary falls back to the path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(drafts-diff): explode data_pipeline bundle into per-node subitems A data_pipeline draft is a bundle of node-script drafts, so a single row diffed the whole thing as one blob. Explode it in DraftDiffDrawer into one script row per node, nested under the bundle's `…/data_pipeline` folder so they read as the pipeline's subitems — each with its own path and a proper script Content/Metadata code diff. The node's draft body is the "after"; its deployed body (when the node is already deployed) is the "before", so edits show as line diffs and new nodes as added. A single bundle row (via the getDraftDiffValues data_pipeline fallback) is kept only for the case where the bundle can't be read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): simplify — drop vestigial approve/reject scaffolding & redundant field Review pass over the PR, removing complexity left from the approve/reject removal and the shared-component refactor (all behavior-preserving): - Inline the `acceptPendingEdits` pass-through into `acceptPendingFlowEdits` and revert the now-inert `autoAcceptEditsAvailable` GLOBAL+pipeline widening (pipeline edits are direct drafts — nothing to auto-accept). - Fix the global system prompt: pipeline tools "apply directly as unsaved drafts (no accept/reject)", not "proposals the user Accepts or Rejects". - Collapse the redundant `outputAsset` (singular) into `outputAssets`, removing a whole resolveGraph fallback tier; simplify propose/editNode. - Drop the single-field `PipelineAiHelpersHandle` wrapper (callers just destructured `{ helpers }`); inline the misleading `isoNow()` helper. - Remove the now-unreachable `data_pipeline` branch in getDraftDiffValues (the drafts drawer explodes bundles per-node; an unreadable bundle is skipped) and the "Step N consolidation" drafting narration. - Un-export internal-only types; reuse `storageKey`; refresh stale comments that still referenced proposals / the review banner / diff-approval. svelte-check clean; 141 unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): tooltip clarifying the Create/Save button deploys The accent button in the asset-graph details pane ("Create" for a new script, "Save" for an existing one) is really a deploy, but had no tooltip explaining that. Add a title — "Deploy this new script to the workspace" / "Deploy your changes to this script" — keeping the create-vs-update label distinction while making clear both deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): document the `materialize` annotation in the pipeline prompt The model invented "materialize run" because the prompt only mentioned `// materialize <uri>` in passing. Spell out what it is in both the in-app pipeline prompt (getPipelinePromptSection) and the base prompt (pipeline-base.md, regenerated): a MANAGED output where the runtime writes the table around a single SELECT (no manual CREATE/INSERT); replace (default) vs `append` vs `key=<col>` strategies; `manual` to opt out (track-only); and its pairing with `// partitioned …` (runs once per partition, `{partition}` token substituted at run time). Explicitly: materialize is an output declaration, not a command — there is no "materialize run". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): trigger drawers in the AI session preview Bring the route page's native-trigger affordances to the in-session pipeline editor by reusing the shared <PipelineTriggerEditors> (no duplication of the drawer UI). Clicking a "Schedule · Missing — no trigger row" node (or edit/delete on an attached trigger, webhook, data-upload) now opens the same drawers the full editor uses, instead of doing nothing. Draft nodes get the same "save the script first" guard (a trigger row needs a deployed script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): run buttons + live run state in the AI session preview Wire the per-node Run button and live run-state badges into the in-session pipeline editor, reusing the shared folder-scoped job poll (useActiveRunnableIds) the route page uses — node badges, the event log, and the zero-latency "running" hint all come from it. The session runs one node at a time (preview for an unsaved draft, the deployed version otherwise), skipping the route page's cascade/deploy-queue machinery the AI-session UX doesn't need. Verified: a node's Run button dispatches a job and the badge updates live from the poll. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): label the node deploy button "Deploy" (was Create/Save) Users read "Create" and asked whether it deploys. It does — and the main script editor's DeployButton already says "Deploy", so this is the consistent term. Use "Deploy" for both the new-script and existing-script cases; the new-vs-changes nuance stays in the button's tooltip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(home): surface data pipelines as units, including bundle-phase drafts Treat a data pipeline as one home entry instead of scattering its member scripts: - The home "Pipeline · f/<folder>" entry now also covers bundle-phase pipelines — a folder that so far only exists as a `data_pipeline` draft — not just deployed ones, so a pipeline shows up the moment its first node is drafted (union listPipelineFolders + data_pipeline draft folders). - Pipeline-member scripts (`auto_kind='pipeline'`) are filtered out of the individual scripts list; they're represented by their pipeline's entry. - Tree view injects pipeline folders so they (and their "Pipeline" entry) still appear when their only scripts are hidden members or they have none deployed yet. Verified in both list and tree view: app_groups (deployed member folded) and a draft-only nyc_transit both show as pipelines; the member script no longer lists individually. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(scripts): compute auto_kind for draft-only pipeline nodes A never-deployed pipeline node (a script draft starting with `// pipeline`) had no script row, so list_scripts synthesized it with `auto_kind: None` — and the home page therefore couldn't tell it was a pipeline member, listing it individually instead of folding it into its pipeline. Parse the draft content the same way the create path does (`parse_pipeline_annotations(...).in_pipeline`) and set `auto_kind = "pipeline"` on the synthesized draft-only row, so draft nodes fold into their pipeline like deployed members. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(search): hide pipeline-member scripts from global search The Ctrl+k global search listed pipeline-member scripts (`auto_kind='pipeline'`) individually. Filter them out — they're reached through their pipeline, matching the home page. Deployed members carry auto_kind from the script row; draft-only members now do too (computed from draft content in list_scripts), so both are excluded here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address PR review findings Session run dispatch (the one real bug): - runNode now passes `_wmill_skip_asset_dispatch: true` for a single-node run of a deployed node unless the user chose "run + downstream" (cascade) — previously a single Run could fan out to downstream deployed scripts via the backend asset dispatcher and fire side-effecting production runs. - onRunProducer guards `kind === 'script'`; onTestStateChange only clears the run hint for the script the pane finished (not a different in-flight node); clear the hint on folder retarget; gate the background poll on isActiveSession so hidden warm panes don't poll; note the PipelineTriggerEditors workspace coupling. Home page pipeline surfacing: - Fold pipeline-member folders into `pipelineFolders` (captured in loadScripts) so a members-only / draft-only-`// pipeline` folder still shows its pipeline entry instead of vanishing; and don't render the empty-state when only pipelines remain (they aren't part of the text filter). - Insert injected tree folders in name order instead of prepending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): make clear `// materialize` is DuckDB + DuckLake only The model put `// materialize` on a python3 node, which deploy rejects ("only supported for DuckDB scripts"). The prompt only implied SQL ("write the body as a single SELECT") without stating the hard constraint. Spell it out in both the in-app prompt and pipeline-base.md: `// materialize` is DuckDB-only and its target must be a DuckLake table; for python3/bun/postgresql nodes, write the output via the SDK instead and let it be inferred — reach for duckdb when a node should materialize a DuckLake table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): fix stale comment — session now wires run + trigger affordances Addresses review: the comment still claimed the session 'opts out of the run/cascade/trigger/bounded affordances', but run buttons + trigger drawers were wired in. Describe the current state (wires run + triggers; omits only cascade/bounded/add-script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address Codex review — test_pipeline_node dispatch + tree search - [P1] testNode (the test_pipeline_node tool) ran a deployed node via runScriptByPath without `_wmill_skip_asset_dispatch`, so previewing one node could fan out to downstream deployed subscribers and run side-effecting scripts. Add the skip flag (test is always single-node) + a regression test. - [P2] Home tree view injected pipeline folders — and rendered their Pipeline row — even during a text search, surfacing unrelated pipelines. Gate both the TreeViewRoot injection and TreeView's hasPipeline on `!isSearching`, matching the list view which hides pipeline rows on a query. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep the pipeline prompt after update_user_instructions rebuildGlobalSystemMessage (called by the update_user_instructions tool) rebuilt only the base Global prompt, dropping the pipeline-editor section that configureGlobalMode appends. So after the chat remembered an instruction, the next GLOBAL turn lost the active /pipeline/<folder> context + direct-draft/ materialize guidance while pipeline tools stayed registered. Re-append the pipeline section here when a pipeline editor is registered. Addresses Codex review [P2]. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(home): gate pipeline entries by kind/archived/owner filters Codex review [P2]: pipeline rows/folders rendered independently of the item filters, so a pipeline still showed under the Flows/Apps tabs, in the archived view, and outside a selected owner. Add `visiblePipelineFolders` applying the same gates the items get (kind ∈ {all, script}, not archived, owner-prefix match) and route the list rows, tree injection, and empty-state check through it. Pipelines are always `f/<folder>`, so the user-folder toggle and kind=script keep including them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address review — route folder-switch state, AI node guards, diff identity claude[bot] [P1]: the route page's in-app folder switcher navigates same-route (no remount), but nothing reset PipelineEditorState — so folder A's drafts displayed under B and autosave persisted them into B's bundle, and B never hydrated. Reset pe on folder change (mirror the session retarget), and guard the shared hydrateDrafts against a stale folder result landing after a retarget. codex/claude [P2]: build_pipeline_node (proposeNode) only checked drafts.has — now rejects a path outside the open folder and one colliding with an existing deployed node (model should edit_pipeline_node). + 3 regression tests. codex/claude [P2]: exploded pipeline-node diff rows shared `script/<path>` with a standalone script draft at the same path, colliding in the {#each} key + value cache. Add an explicit unique `key` (the distinct bundle-nested path) on DiffRow; pipeline nodes set/look up by it while `path` stays the real edit target. claude [P2]: session AI test_pipeline_node now arms the live run badge (onRunStarted), matching the route page. nit: pipelineAiHelpers.test uses afterEach(restoreAllMocks) instead of an unreachable inline mockRestore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): harden AI node mutations + close home label-filter / rename gaps Codex [P1] (AI mutations trust model paths) — fully scoped now: - editNode validates the open folder too (proposeNode already did), via a shared assertInFolder; an edit_pipeline_node for f/other/* no longer persists an unrelated script into the current folder's data_pipeline bundle. - both build_pipeline_node and edit_pipeline_node now require the `// pipeline` annotation (assertPipelineAnnotation) so a staged draft is definitionally a pipeline member, not a silently-non-member script. + tests. (proposeNode's folder + deployed-collision guards landed in the prior commit.) Codex [P2] home label filter — visiblePipelineFolders ignored labelFilter, so a label selection still showed every pipeline (and the empty-state fell through to render pipeline rows). Pipelines carry no labels, so a label filter hides them. Codex [P2] session rename — PipelineEditorView now wires onScriptRenamed (repoint selection + refetch), matching the route page; a persisted-script rename no longer leaves the canvas on the old path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): language-specific comment prefix for annotations Codex [P2]: the tool schema and prompt told the model to write `// pipeline` / `// on` / `// materialize` regardless of language, and pipeline-base.md grouped SQL with `#`. A `//` (or `#`) annotation line is invalid in a DuckDB/Postgres node — it passes the frontend parser (which strips `//`/`--`/`#`) but is a SQL syntax error at deploy/run. Make the guidance language-specific everywhere: `--` for SQL (duckdb/postgresql), `#` for python3/bash, `//` for bun/TS — the `//` in examples is the TS form to translate. Regenerated the prompt outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): re-scope Global prompt on folder switch + language-aware base prompt Codex [P2] x2: - The route page resets editor state on an in-app folder switch, but the Global chat's system message kept the old `/pipeline/<folder>` scope (the helper methods read the reactive folder, but the prompt string is only rebuilt on Global-mode reconfigure). Rebuild it on folder change so the next turn targets the new folder. - The pre-editor base Global prompt (seen before open_preview/get_instructions) still showed TS-only `// pipeline` / `// on`. Make it language-aware (`--` SQL, `#` Python/Bash, `//` TS) so the model can't draft invalid DuckDB/Postgres nodes before the pipeline tools are registered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): authoritative new-node probe + SQL-correct eval checklist Codex [P2] x2: - build_pipeline_node's collision check relied on the resolved graph, which can be empty while the session preview races open_preview (a build could shadow a deployed node before the graph loads) and only covered pipeline runnables, not a non-pipeline script at the same path. Add an authoritative backend probe (ScriptService.getScriptByPath): any deployed script at the path → reject with "use edit_pipeline_node". + regression test (empty graph, deployed script). - The DuckLake eval judgeChecklist required the exact `// pipeline` annotation, which would penalize the now-correct `-- pipeline` SQL output (or reward invalid DuckDB syntax). Make both cases syntax-aware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): rebuild Global prompt on session preview folder retarget Codex [P2]: open_preview(kind="pipeline", path="B") can retarget an existing pipeline preview from folder A to B without remounting. The retarget effect resets editor state and the helper methods read the new path, but the registration effect only depends on isActiveSession, so the Global system message stayed scoped to /pipeline/A. Mirror the route-page fix: rebuild the global system message on retarget (gated on isActiveSession — only the active session's helpers are registered; a hidden session reconfigures when it next becomes active). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): edit_pipeline_node preserves deployed script metadata Codex [P1]: editNode kept only the deployed script's language and staged a fresh makePipelineScript draft with empty hash/summary/description/tag/schema/settings. Deploying that edit from the pane (auto_parent) would update the script while wiping its metadata, and the route "Save all" path (no parent_hash) could hit the backend path-conflict branch on the occupied path. Base the draft on the existing draft's / deployed script object and replace ONLY content (+ inferred output assets), preserving hash and metadata. + regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |