mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-09-06 08:01:35 +00:00
bebd8bef8ba86f0cc016db3827ed1d8ee8c577fc
61
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  GuilhemandClaude Opus 5
|
449b1a6933 |
fix: ground the chat's AI agent provider in the workspace's models (#10774)
* fix: ground the chat's AI agent provider in the workspace's models Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: never reject an AI agent model the catalog could not confirm Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: benchmark AI agent provider grounding in ai_evals Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: only reject an AI agent model an exhaustive listing rules out Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep instance-level AI settings out of the workspace provider catalog Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep untrusted model ids out of the chat's context Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: carry completeness on the model listing itself Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct two comments left behind by the catalog rework Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound the model listing and verify the default against it Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: honour a workspace default a filtered listing names Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: recognise a workspace default past the prompt's model cap Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep an aliasing provider's unlisted model ids permissive Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
574775d50c |
fix: teach the AI the raw-app job bindings, the SDK reference and the draft/deployed split (#10754)
* feat: teach the AI the raw-app job bindings and the draft/deployed split Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: scope the raw-app deploy advice to the referenced item, and stop kind-conversion from stranding fields The draft/deployed guidance added in the previous commit was read as "deploy the app too": the agent asked for both the flow and the app and routed a one-item dependency through the review-and-deploy page. Only the referenced flow or script has to exist deployed — the preview runs the app's draft — so the prompts, the `write_app_runnable` warning and the testing rule now say to offer that one deploy and leave the app a draft. `buildPersistedRunnable` spread the existing runnable when rewriting it, so converting a path runnable to inline left `runType`/`path` behind (and the reverse left `inlineScript`). `isRunnableByName` matches the inline branch first, so an app "wired to a flow" silently ran stale inline code. `test_run_app_runnable` now fills ctx-bound inputs with `$ctx:<prop>` the way RawAppBackgroundRunner does, so a ctx argument no longer arrives missing. The SDK-reference rationale claimed WM_TOKEN may be unset, that a missing base URL falls back to localhost, and that a job token is scoped enough to 403 a hand-rolled REST call. None of the three is true, and it shipped to every write-script prompt; the text now only says the client configures itself. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address review round on the raw-app AI instructions The eval case could pass on the exact answer it exists to reject. Every `requiredMentionsAnyOf` alternative but one was flow-agnostic, so "the app must be deployed" satisfied "must be deployed". All alternatives now name the flow, and a unit test pins that the app-only phrasing fails. `instanceLine` asserted "self-hosted Community Edition" outside the browser, where `isCloudHosted()` reads false and the license store is unset — so every global eval was told that regardless of what it pointed at. It is now emitted only under BROWSER. `assistantExpect.forbiddenMentions` defaulted a missing `assistantText` to "", which passes every entry forever on a mode whose runner does not report it. It now fails with that as the reason. `buildPersistedRunnable` carried `schema` across a retarget, so a path runnable pointed at a new flow kept the previous item's schema and `genWmillTs` typed `backend.<key>(args)` from the wrong inputs. It survives only while kind and path both match. The SDK header claimed "a function that is not listed below does not exist". `windmill-client` also exports the generated services, and the Python client exposes `Windmill.get`/`.post`, so an endpoint without a helper had no legal move. Each language now names its own escape hatch. `getAppInstructions` said the attached reference carries the TypeScript SDK even when `language: "python3"` had swapped in the Python one — on the very sentence telling the model to make that call. The kind-conversion comment claimed a hybrid runnable "silently runs stale inline code". It does not: `isRunnableByName`, `isRunnableByPath`, `convertPersistedToBackendRunnable` and `rawAppPolicy.processRunnable` all dispatch on `type` alone. The leftovers contradict the runnable's kind rather than override it, which is what the comment now says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: address round-2 review nits on the raw-app AI instructions `flow is deployed` was satisfied both by "once the flow is deployed, the button works" and by a hallucinated "done — the flow is deployed", which eval mode makes impossible and the drafts-only judge cannot see. Every alternative now states an outstanding obligation, and two more real phrasings ("will need to be deployed") are accepted so a correct answer is not failed on wording. Condenses the three comment blocks that ran past the four-line limit in AGENTS.md, and drops two claims inside them that no longer hold: the `testRunAppRunnable` doc said it runs a runnable the way the app's own frontend does (it is the editor preview, which a deployed app's stored policy does not match), and `undeployedRunnableTargets` described its argument as the write tool's raw input when the call site passes the persisted runnable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report the real cause when a test run fails, and label the app-runnable card Driving `test_run_app_runnable` in a live session surfaced two defects the API-level check could not see. `executeTestRun` built its failure message from `error.message`, which the generated client leaves as the bare status text while the server's message sits in `body`. A path runnable aimed at an undeployed flow reported "Not Found" instead of "Not found: flow not found at name u/admin/current_time" — dropping the one diagnostic the run exists to produce. `formatToolError`, in the same file and written for exactly this, now does it. This also applies to test_run_script and test_run_flow, which had the same loss. The completion card read "Flow test completed successfully" for an app runnable, because `contextName` doubles as the jobs-tray kind and a path runnable pointing at a flow really does queue a flow job. A `completionName` override now names what ran without changing the kind. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: pin the deploy expectation against wrong answers, not just correct ones `deploying the flow` was satisfied by "done deploying the flow" — a deploy the agent only claims to have made, which eval mode makes impossible and the drafts-only judge cannot see. Replaced with the prospective forms, and dropped the same reading from the workflow variant. Three review rounds each found this same class of hole in the phrasing list, so the list is now exercised against the wrong answers themselves rather than eyeballed: naming the app as what needs deploying, claiming the deploy is already done, claiming to have deployed the flow, and saying nothing about deploying all have to fail, while four real correct phrasings have to pass. The test reads the case out of global.yaml, so a future edit to the alternatives is checked by it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: drop the tense-neutral deploy alternatives and cover completed claims A gerund after a preposition carries no tense, so `before`/`after`/`by deploying the flow` all match a deploy the agent only claims to have made ("after deploying the flow, I clicked the button and it returns the greeting") just as the bare gerund did. All three are gone rather than swapped for whichever reads least badly, and the two completed-deploy phrasings are now negative fixtures. The remaining alternatives are imperative or obligational, which a claim of having already deployed cannot satisfy. Condenses the two comments this list carries: the YAML block to four lines, and the test's rationale to the durable constraint about substring matching. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: encrypt sensitive inputs when test-running an app runnable `test_run_app_runnable` sent `force_viewer_static_fields` but not `force_viewer_sensitive_inputs`, which every other preview path derives from the runnable's `sensitive` user fields. That list is the only thing driving the encryption loop in apps.rs, so testing a runnable with a sensitive input wrote the real value into the job's args in plaintext, readable by anyone with run access to the workspace. Verified against a running EE instance. With the list, `api_key` is stored as `$encrypted:mvqtSRI9…` and the sentinel appears nowhere in the job record; without it, the sentinel is readable in run details. A non-sensitive field is left plaintext either way. The tool claims parity with the editor preview, so it uses that same filter (`type == 'user' && sensitive`) and omits the field entirely when empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
 AlexRV12
|
fd9295a58e |
feat(copilot): let plan mode draw, but never write the plan (#10725)
* fix(copilot): validate the version an approval stamps * feat(copilot): let plan mode write artifacts, but never the plan * feat(copilot): tell plan mode it may keep notes, not rewrite the plan |
||
|
AlexRV12andClaude Opus 5
|
850b028778 |
feat: advertise the pinned artifact version in get_preview_status (#10691)
* test: let global evals seed the session's preview tabs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: advertise the pinned artifact version in get_preview_status Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test: reject ambiguous preview-tab and artifact eval fixtures Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AlexRV12andClaude Opus 5
|
caa189868c |
feat(ai-sessions): add plan mode (#10057)
* feat(sessions): let an opener name the artifact version to show A tab already remembers the version a reader pinned, and re-pointing it keeps that pin. Plan mode needs the two intents that leaves out: a plan card scrolled up the transcript wants the version it proposed, and a plan going up for approval wants the current text with no pin at all. `ArtifactVersionTarget` is those two alongside the existing one: a number, `'latest'`, or omitted. Omitted still cannot double as `'latest'` — every artifact tool re-opens the document it just wrote, so taking that as a request to move would yank a reader out of the version they chose on every edit the agent makes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(copilot): add the plan-mode gate and tag plan-mode-safe tools Plan mode is a read-only posture, so something has to decide which tools it may still run. `Tool.planModeSafe` is that tag, and processToolCall fails closed on it: untagged means mutating means blocked. Deriving it from `requiresConfirmation` was not an option — unconfirmed mutating tools exist, and a posture that leaks one is not a posture. The gate runs twice per call. Before `validateBeforeConfirmation`, so a validator cannot reach out while planning; and again after the confirmation wait, because plan mode can be entered while a mutating tool's card is already pending, and that approval must not carry it through. Arguments are read one field at a time rather than through a parse of the whole call. `change_note` is optional and cosmetic, and a model that sends it as `null` would otherwise fail the object parse and take the plan down with it — the user being told there was no plan to approve, which is false. Also here, because refusing a call well needs them: a validator may now return the row the user reads and the result the model gets separately, a tool may word its own cancellation, and a tool may start work when its card appears rather than when it is approved. The gate is consulted before any of them. `shouldAutoAcceptToolConfirmations` is asked about the tool by name, because skipping the confirmation wait is itself an answer on the user's behalf and one tool must not be answered for. Deciding that without the name would put the exception out of reach of the only path that needs it. The gate stays inert until a chat supplies `isPlanModeActive`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(copilot): give a session one versioned plan document The plan the user agrees to has to survive `/clear`, so it belongs to the session rather than the conversation, and a session holds exactly one. Its id is the session's, so the primary key is the constraint — there is no second row to mint, no index to maintain and no schema change at all. Every write reads the row it is about to replace inside the transaction that replaces it. Read outside, two tabs both see version N, both stamp N+1, and the later write silently drops the earlier one's text and its snapshot; IndexedDB serialises readwrite transactions over a store, so read and write together cannot interleave. Approval takes the same route but patches only the pointer: an approval computed while another tab was revising must not carry this tab's older content back over the newer text. Approval is `approvedVersion`, a pointer at a version, never a flag. Below the current version means the newest text is a proposal the user has not agreed to; absent means nothing here was ever approved. Only exit_plan_mode can leave the pointer behind, since every write outside plan mode carries it forward — an amendment the user's posture already trusts is still the agreed plan. Declining writes nothing at all: the refused proposal stands as the newest version, with the agreed one still in history. Nor can create_artifact confer approval. It asks for no confirmation, so the model writing a plan document is not the user agreeing to one; a plan written there holds the session's slot as a draft until a decision lands on it. That is also why the approved version is exempt from pruning. A plan approved at v1 and then planned against for twenty more rounds would otherwise lose the very version that stands as agreed, and with it the card that opens it, the banner offering it back, and read_artifact at that version. It is excluded from the pruning candidates rather than added on top, so the budget is unchanged and what survives simply stops being contiguous. The write reports whether the database took it. Most callers still degrade like the reads do, but a plan cannot: returning one the database refused would let the user approve and execute against a document that disappears on reload — a refused plan write raises instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(copilot): add plan mode — the posture and its two tools enter_plan_mode asks to hold work; exit_plan_mode hands over a plan and, on approval, gives the posture back to whatever preceded it. Both carry `planModeSafe`, since a posture with no exit is a trap. Only the transition the current posture allows is offered, so there is no tool for leaving a posture the chat is not in. A planning round runs from entering plan mode to the proposal the user decides on. It remembers only the write it made, because nothing it does is undone — and that write is shared between the card's confirmation hook and the tool's `fn`, so the plan is on screen while the user is deciding whether to approve it rather than after. The round is identified by an epoch bumped on *entering*, not by the conversation. A chat rotation mid-approval must still let that approval hand the posture back; a round the user has since left and re-entered must not, or approving the old plan would drop them out of a read-only posture they just chose. Saving a proposal revises the session's plan document and creates one only when there is none — both halves in a single transaction, so a second tab proposing at the same moment revises the row this one wrote rather than racing it. Persistence failures hold the posture. Approval is reported only once both the proposal and the approval pointer are durable, so a plan the database refused cannot unblock mutating tools. The failure is reported from `fn` and no earlier: the write settles while the card is still waiting to be confirmed, and clearing that card from underneath the wait would take away the only control that resolves it. An auto-accepting posture answers for the user through one predicate, asked by every path that answers: the pending-card sweep, the confirmation itself, and the decision to skip the wait at all. enter_plan_mode never qualifies: YOLO means "stop asking and run it", and a call from a tool set snapshotted before the switch must not answer that with a read-only posture — whether its card is already pending or has yet to be registered. Plan mode lives in its own controller with a narrow view of the chat it runs in: it reads that autonomy state and asks for the two changes it can cause, rather than owning any of it. Plan mode is offered only in a session chat, and a session chat is GLOBAL for its whole life. The gate reads that mode, so `changeMode` refuses to move one out of GLOBAL rather than resting the invariant on a picker being hidden. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(copilot): surface plan mode in the chat and the artifact list Plan mode is the only posture that refuses work, so the composer says so before the user types the request it is about to turn down: the mode pill is tinted whole rather than by its icon, and the empty placeholder carries the constraint in words. Teal, not the house green — green is the transcript's success colour a few rows up, and a mode signal in it would read as "this worked" rather than "this is held". A blocked tool renders as its own lean row naming the tool, not as an error: the call did what plan mode says it should, and "why can't it edit" is answered where it is asked. A plan card names the decision — proposed, approved, or not approved — and never the button, since a Stop and a posture switch resolve it too. Its button opens the version that card proposed, so a card far up the transcript still shows the plan it put forward rather than whatever the document has become since. The artifact list and the preview header both label the plan through one badge helper, so the two cannot disagree about what counts as one: a plan the user never approved keeps the plan icon and takes the neutral badge, leaving the teal to mean exactly one thing. In the viewer, an unapproved revision says so in a bar that cannot be scrolled past, with the version the user did agree to one click away. The autonomy picker became a table with one row per posture, so adding one touches a single place instead of four parallel switch statements. A version of a plan is read against the one the user approved, not against the newest: latest is only where the model happened to stop. So the approved version is never stale — its bar is teal and points forward to the draft rather than warning about it — the version in front of it is the draft, and anything behind it is history that is neither and takes no pill at all. The list opens a plan at the approved version for the same reason, which is what lets its pill say `plan` while an unapproved draft sits at the head. One helper answers all of it, so the list and the preview header cannot drift apart on what counts as the plan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ai-evals): exercise plan mode end to end A case a unit test cannot stand in for: it starts in plan mode against the real gate and the real exit_plan_mode, and grades whether the model researches and hands over a usable plan instead of guessing at one. The checklist does not grade what the harness does for the model — exit_plan_mode writes the plan document itself, so "saves the plan as an artifact" would pass on any run where the tool is called at all. The eval store seeds artifacts with history and mirrors the store's own approval rules, so a rename cannot promote a proposal the user turned down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ai-evals): import the plan-mode messages from the module that owns them `PLAN_MODE_MESSAGES` moved to `planModeMessages.ts`; `planMode.ts` imports it without re-exporting. Under vitest, which runs the frontend adapters, the stale import resolved to `undefined` rather than failing to link, so `global-planmode1-hands-over-a-plan` threw on the approval message after the posture had already been dropped and the tool withdrawn. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(copilot): state plan mode's constraint in neutral text The composer's two-tone placeholder becomes a plain "Read-only" beside the autonomy picker, next to where YOLO puts its own warning, and a blocked call's row drops the mode colour. Teal is left marking what the posture is — the badge, the version bars, the pill — rather than every call it refuses. ContextTextarea goes back to main with the accent: `placeholderAccent` had no other consumer, and the aria-label existed only because the accent blanked the native placeholder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(copilot): hold the plan header's verdict until the snapshot lands Opening a plan at the version its reader approved pins a version behind the head, and until that read resolves `shownVersion` is still the head — so the header wore the draft's badge and its orange "not approved" bar over the very case the pin exists to serve, then flipped. The header now says nothing while `restoringPin`, as the body already does. Judging `pinned` instead would print the approved signal over text that is still the draft, trading a true transient signal for a false one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(copilot): refuse a hand-over once plan mode has ended A response can carry two exit_plan_mode calls, and the tool list they run against is snapshotted before the first one restores the posture. The second then found the tool with plan mode already over: under YOLO every confirmation is answered for the user, so it wrote its own summary and stamped the user's approval on a plan no card had shown them. Refused in `validateBeforeConfirmation` rather than in `fn`, since `onConfirmationRequested` writes the document too. The maintenance path is untouched — a plan still gets revised outside the posture with update_artifact, which is what the tool's own description already tells the model to use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 5
|
eb238e3f0b |
fix: stop the AI chat destroying secret variables on edit (#10616)
* fix: stop the AI chat destroying secret variables on edit Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: clear stale staged secret values and state the draft-staging rule Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: condense the pending-secret invariant to its field Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse empty and oauth-managed secret values, keep drawer-staged ones in the draft Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: resolve a variable deploy's secret from one draft snapshot Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: make the variable draft the single source of a staged secret Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: drop stale in-memory secret invariants from comments and the eval Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop null account/expires_at leaking into variable drafts and diffs Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: report when a variable deploy leaves the secret value unchanged Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: scope the variable-value readability claims to the chat Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: correct the secret-draft invariant in the diff masking comment Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: record why a non-secret value is resent on a partial update Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: stop "Load secret value" discarding a staged secret The audit-logged load writes the deployed secret into the draft row the variable drawer shares with the AI chat, so offering it while that row already stages a value silently replaces it — and the deploy that follows carries the old value with no sign the staged one was lost. The gate that hid the action already existed but keyed on `isEncryptedDraftValue`, which only holds once a draft has round-tripped through the server. A value staged in the same tab is still plaintext, so it slipped through. Key on "anything staged" instead; clearing stays explicit via Reset. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: extend the variable draft's empty-value sentinel past secrets Two gaps in the chat's variable write path, both from treating "the draft cannot carry this value" as meaning only "the value is secret". `variableToDraftState` drops the value of an OAuth-managed variable so a refreshed live token is never pinned into a draft, leaving '' behind. The deploy body resent that '' verbatim for a non-secret one, wiping the token the refresh flow owns. The sentinel now covers every value the draft is not allowed to hold, which also removes the divergence from `VariableEditor.save` and the shared deployer. Making a variable secret when it holds no value produced a secret draft staging '', a deploy body with no `value`, and the backend's "cannot change is_secret without updating value too" — the sibling create path already answers that case with guidance, so answer it here too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate the Secret toggle's secret load on the staged value too The toggle calls `onLoadSecret` on every change so an is_secret flip has a value to send, but that load overwrites the shared draft row — the same discard the button gate just closed, reached by a different control. It now loads only when the row stages nothing, which is exactly when the flip needs a value fetched. With a value already staged there is one to send, and it is the one the user or the chat put there. Blocking the load costs the side effect that used to mask a worse bug: for a deployed variable, the load replaced an `$encrypted:` marker with real plaintext before save. Without it, un-securing a marker would store the marker string as the value, since the deploy endpoints only decrypt it while is_secret stays true. So the toggle is disabled outright while a marker is staged — Reset first. That closes the marker case for draft-only variables as well, where no load could ever have masked it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 5
|
ce58b8495c |
feat: expose every runs filter on the open_page chat tool (#10612)
* feat: expose every runs filter on the open_page chat tool Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: reject runs filters the page would silently ignore Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: normalize runs list filters and refuse combinations the page drops Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: validate the full folder-name contract and pin evals to one call Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: refuse queue statuses the concurrency view cannot filter on Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AlexRV12andClaude Opus 5
|
77adf85ccd |
feat: version history for session artifacts (#10574)
* fix: never replace an in-flight indexeddb open, only a settled one Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: keep a version history for session artifacts Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: let the assistant browse an artifact's earlier versions Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat: pick an older artifact version from the preview panel Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ai_evals): cover the change note the assistant writes on each edit Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: bound every indexeddb open, not only one told it is blocked --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 5
|
505705fd3d |
serve getJob in the ai evals benchmark api catalog (#10511)
* fix: serve getJob in the ai evals benchmark api catalog Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: scope the frontend format hook to the frontend dir Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: answer the run-by-path endpoints and mirror the real getJob entry Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: gate the format hook on a repo-root frontend, not the project dir Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ruben Fiszel
|
a3c59c7320 |
perf: serve the rare schedule options on demand instead of inlining them (#10487)
* perf: serve the rare schedule options on demand instead of inlining them * fix: let a real schedule argument win over a duplicate inside advanced * fix: catch nested stripped schedule options and share the schema builder * fix: check the whole schedule request for stripped options, not just advanced * fix: do not point unknown schedule keys at the schema lookup |
||
|
GuilhemandClaude Opus 5
|
ed9fb47f5a |
fix: stop the chat deleting drafts as deployed workspace items (#10476)
* fix: stop the chat deleting drafts as deployed workspace items Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: keep archived scripts deletable and defer malformed args to the schema Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: correct the delete/draft prompt claim and narrow the script probe catch to 404 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 5
|
5feec9b4cd |
fix(ai-chat): make hub script paths readable from the global chat (#10381)
* fix(ai-chat): make hub script paths readable from the global chat Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ai-evals): match hub fixtures on whole words, not substrings Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Fable 5
|
0f62891d43 |
fix(ai): stop teaching nonexistent while-loop iter.value state-carrying (#10345)
* fix(ai): stop teaching nonexistent while-loop iter.value state-carrying Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): scope while-loop results guidance to cross-iteration reads only Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): drop unverified wmill state-helper fallback from while-loop guidance Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): document supported cross-iteration results state in while loops Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): rescope while-loop fast-path rule and add results-carrying example Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
248c8751b1 |
feat(ai-chat): let the session/global chat create email triggers (#10282)
* feat(ai-chat): let AI create email triggers for flows and scripts The AI chat could not identify that it can create email triggers. The `open_page` tool advertised `email` as a trigger kind (routing to /email_triggers), but neither `write_trigger` (global/session chat) nor `create_trigger` (flow chat) accepted `kind: "email"` — their enums stopped at 10 kinds. Seeing email as a navigable page but having no tool to create one, the model concluded email triggers were unsupported and faked them with an HTTP webhook plus an external mail provider. Add `email` to the shared trigger-schema generation (`WORKSPACE_TOOL_*` in system_prompts/generate.py), which feeds both `triggerRequestSchemas` (global) and `createTriggerToolSchema` (flow), then wire the kind through the global chat draft path (TRIGGER_KINDS, draft-kind map, triggerServices/labels, write_trigger config union) and the flow chat `triggerConfigs`. The backend route, EmailTriggerService, `trigger_email` draft kind, and email UI drawers already existed. Add discovery eval guards (`flow-test17`, `global-test29`) that assert the model selects `kind: email` for an implicit "run when an email is received" request. They assert the recorded tool-call kind rather than the resulting draft, so they hold on a CE backend where email trigger routes (smtp+private-gated) are not compiled. A mocked unit test in global/core.test.ts covers the full draft->read->deploy path. Verified: flow guard 5/5, global guard 4/4 (sonnet); frontend unit tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): default email trigger workspaced_local_part before persist The global chat forwards the write_trigger draft unchanged, but workspaced_local_part maps to a NOT NULL column and the model may omit the optional field — deploying such a draft would fail instead of creating the trigger. Default it to false at draft-write time (as flow chat does at create), and assert the deployed request carries it in the unit test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): make email trigger listing and edits robust Two issues from adding the email kind to the global chat: - list_workspace_items iterated every trigger kind and awaited each list unconditionally; email routes 404 without smtp+private (as do EE kinds on CE), so one unavailable kind threw and dropped the entire trigger listing. Skip a kind whose list endpoint is unavailable. - Defaulting workspaced_local_part on the incoming config before the base merge reset an existing trigger's value: editing a workspaced trigger while omitting the optional field overwrote true with false, changing its receiving address. Default on the merged draft instead, so an omitted field keeps the existing value and only genuinely new drafts get false. Adds regression tests for both. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): only skip 404 trigger-list failures, surface the rest The prior fix caught every rejection from a per-kind trigger list, so an auth, 5xx, or network failure produced a successful-but-incomplete listing that could make the chat treat existing triggers as absent. Skip only a 404 (route not compiled in) and re-throw anything else. Adds a propagation test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
e41440b344 |
feat(ai-chat): email triggers in flow/script chat + trigger-intent eval guards (WIN-2228) (#10267)
* test(ai-evals): guard implicit trigger/schedule intent in flow chat Investigation of WIN-2228 (does flow AI chat understand it should create a flow AND its associated triggers): the flow-editor chat already exposes create_schedule and create_trigger (10 kinds), both confirmation-gated, and an A/B eval shows the model already recognizes IMPLICIT trigger intent reliably (12/12 across two new cases on the current prompt) without naming a "schedule" or "trigger". Add two ai_evals flow cases that phrase the trigger intent implicitly, to guard that recognition against future prompt/tool regressions. These are not redundant with the existing explicit cases (flow-test15/16): a trial system prompt addition that spelled out a deployment prerequisite regressed the HTTP case from 6/6 to 2/6 (the model deferred instead of creating the trigger), which these cases caught. No prompt change ships: the addition showed no measured benefit over baseline and the fuller version regressed behavior. Fixes WIN-2228 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): support email triggers in flow/script create_trigger The chat's create_trigger tool exposed 10 trigger kinds but not email, even though the backend supports email triggers and the chat's open-resource drawer was already wired for them (CreatedResourceActionDrawers, the 'email' CreatedResourceTriggerKind). So when asked to make a flow run on incoming email, the model had no email kind and substituted an HTTP trigger it mislabeled as email. Add email as a create_trigger kind (generator + regenerated zod schema + triggerConfigs → EmailTriggerService.createEmailTrigger). Email triggering only works once an instance superadmin has stood up an SMTP server and set the `email_domain` global setting, so guard the create path: read `email_domain` (readable by any authed user; returns null when unset) and, when it is not configured, return role-aware setup guidance instead of a failing create — pointing a superadmin to Instance settings and a regular user to ask a superadmin, both with the docs link. When configured, create the trigger and report the resulting inbound email address. userStore and the email-address helper are lazy-imported so the chat tools module does not drag in the heavy $lib/stores graph at load. Guarded by unit tests for both branches (shared.test.ts) and an ai_evals case (flow-test19); the model now calls create_trigger(kind=email) 3/3 on a natural "run when an email is received" prompt. Fixes WIN-2228 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): address codex review on email trigger + eval guards - [P1] Default `workspaced_local_part` on the email trigger request body before it is sent, not only when formatting the success address. The column is BOOLEAN NOT NULL, so a request omitting it (the model may) was rejected by the backend. Assert the defaulted `false` in the happy-path unit test. - [P2] Tighten the implicit-intent eval guards so they validate the requested configuration, not just tool selection + path prefix: flow-test17 now checks the cron time (07:30) and UTC timezone; flow-test18 checks kind=http, POST method, no auth, and the route path. Cases still pass 9/9 (sonnet). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Ruben FiszelandClaude Opus 4.8
|
0819641f3a |
feat(ai): improve data-pipeline building in AI sessions (prompt + evals + e2e) (#10270)
* test(ai_evals): pipeline coverage for AI sessions + editor e2e Add a complex incremental DuckLake pipeline case, harden the two-node case, and encode the declarative pipeline contract (`-- on` triggers, `-- materialize` + bare SELECT) in the pipeline judgeChecklists so the LLM judge stops false-negativing correct nodes. Add a deterministic Playwright e2e that seeds annotated pipeline scripts and asserts the /pipeline/<folder> editor derives the lineage DAG. Fixes WIN-2229 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): steer pipeline chat to DuckDB+materialize and warn on missing storage The pipeline authoring prompt (getPipelinePrompt, used by the global/session chat and the /pipeline editor) was neutral on language choice and said nothing about storage readiness. Default it to duckdb materializing into DuckLake unless the work specifically needs postgres/data-tables or bun/python, and add a storage prerequisites section: a DuckLake pipeline needs workspace object storage + a DuckLake catalog, so warn when none is configured and give role-appropriate next steps (admin: workspace settings; others: ask an admin). Drafting is not blocked. A/B on sonnet (global pipeline cases): no regression, +639 finalContext tokens. Fixes WIN-2229 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai_evals): address review - e2e teardown, async-edge note, merge-mode hedge Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai): address review nits - drop phantom list_ducklakes tool ref, trim narration comments Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai): add list_ducklakes chat tool for pipeline storage readiness The pipeline counterpart to list_datatables: lists the workspace's configured DuckLake catalogs so the chat can detect the storage prerequisite before building a DuckLake pipeline and warn with role-appropriate next steps when none exists (drafting stays unblocked). Wired into the global tool set and referenced from the pipeline authoring prompt. In the eval run all three pipeline cases called it unprompted with no build regression. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore(ai): fix DuckDB annotation syntax in duckdb-default section (-- not //) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Fable 5
|
1da664fc89 |
open_page opens resource/variable edit drawers when asking user to act (#10258)
* feat: open_page opens resource/variable edit drawers when asking user to act Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: clear handled drawer hash when navigation leaves the target Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: force preview reload when open_page re-targets the tab's current URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
GuilhemandClaude Fable 5
|
9739d5a2c2 |
feat: unified read-only diff chat tool (drafts, fork vs parent, search) (#10211)
* feat(frontend): unified `diff` chat tool with cached snapshot, fork mode, and search One read-only global-chat tool for every comparison: drafts vs deployed (workspace index + per-item unified patches over stable YAML), deployed fork vs parent workspace (against="parent_workspace", sharing the fork banner's compareWorkspaces fetch through a single-flight store), and a literal grep over changed diff lines. Multi-file raw apps split into per-file text patches with folder-style index children and per-file reads. Patches are materialized once into a per-workspace cache keyed on draft created_at / comparison ahead-behind markers and the workspace drafts version, so repeated queries never refetch unchanged content. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai_evals): teach the mock draft backend what the diff tool reads The diff tool reads drafts through the get_draft overlay, the drafts listing's draft_only flag, and per-row created_at change markers — none of which the benchmark mock modelled (fixed timestamp, always draft_only, overlay ignored), so in evals every draft looked absent and the model looped to max turns. Mirror production: monotonic deterministic created_at bumped per upsert, draft_only computed against the deployed stores, and draft/no_deployed overlays on script/flow/app reads (404-shaped not-found). Also drop the diff case's judge items about conversation content the judge never sees — tool usage is already enforced deterministically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): diff tool reads unsaved editor state instead of going stale The pre-diff flush honors the auto-save toggle (a read-only tool must not persist parked edits), which left a gap: with auto-save off — or after a failed save — the persisted draft the diff reads is stale, and a brand-new editor-only draft looks absent. Item reads now detect unflushed parked edits (hasUnsavedDisabledChanges / failed save state) and diff the in-memory editor value directly, bypassing the snapshot cache (it must only hold persisted state) with an explicit unsaved- changes note; index and search modes warn which items' unsaved edits they exclude. Local values are canonicalized onto the persisted draft shape so they never diff noisily against the deployed side. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai): invalidate diff cache the moment any draft write lands The snapshot cache leaned on time windows (5s listing throttle, 15s read reuse) to notice writes it didn't trigger itself — an editor autosave landing between two diff reads could serve the pre-edit patch. The syncer now exposes onAnySaved (fires for landed upserts AND deletes, all keys), and the snapshot subscribes once: a landed write marks exactly that item's patch stale and expires the listing throttle, so the next read refetches regardless of any reuse window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): reject the diff file arg on single-document items Passing file for a script/flow/classic-app diff was silently ignored and returned the whole patch — an explicit error steers the model to call again without it. Also declares the file arg on the item handlers' signatures it was already flowing through. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): surface empty-file additions/deletions in app diffs An empty file appearing or disappearing produces no text patch, so the per-file split dropped it — a draft whose only change was such a file read "unchanged". Presence changes now keep their added/deleted entry (patch '', 0 lines), render as "(empty file)" in summaries, and a file read states the presence change instead of an empty window. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): include classic-app drafts in the diff index and fix ++/-- search itemTypeForKind now maps classic `app` draft rows to the chat app type (mirroring the read path, which already pairs app/raw_app), so their diffs materialize in the index and search instead of reporting "not addressable". Changed-line search is hunk-aware: `---`/`+++` file labels only occur before the first @@ marker, so a changed source line like `++counter` (rendered `+++counter`) now matches instead of being mistaken for a label. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): mask every variable value in chat diffs; compare classic apps value-to-value Variable VALUES never reach a tool result — the chat-wide invariant read_workspace_item enforces, not just for secrets. Draft-mode diffs mask both sides with a placeholder pair that still marks WHETHER the value changed; fork-mode masks at fetch (and still never decrypts); item reads carry an explicit note. The former secret-only flag is now valueMasked. Classic-app drafts hold the bare grid value while the deployed row nests it beside summary/policy — diffed raw, a one-field edit read as a whole-document rewrite. Both sides now reduce to { value } via classicAppDraftValue (pure, unwraps legacy wrapped drafts), which also cleans the CompareDrafts drawer for classic apps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): honest secret-draft reporting, classic-app metadata split, glob-safe file subjects A secret variable's sides are both masked upstream, so an empty patch cannot prove the value is unchanged — such drafts now report "cannot be compared; may differ" (valueUncomparable) instead of "matches deployed", in the index and item reads. Classic-app drafts mirror summary/draft_path into the bare grid while the deployed row keeps summary as a column: sides now reduce to {summary, value} via classicAppDraftParts, applied to both sides, so a summary edit diffs as one and draft-only markers never pollute the grid diff. Raw-app search subjects strip the file key's leading slash so slash-anchored globs like f/x/*.tsx match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): classic-app local edits, comparison-relevant fork fields, conflicts as unflushed The chat app type spans two draft kinds: item mode now flushes and probes both raw_app and classic app keys, and the flush sweep includes classic-app editor cells (kept out of GLOBAL_DRAFT_KINDS so clearGlobalDrafts never clears an open classic editor). Fork projections gain the fields the backend comparison counts that getItemValue drops: flow schema (with a taxonomy-agnostic inline-hash strip) and resource-type description/format_extension/is_fileset. Folder display_name is not exposed by the API's Folder type, so it cannot be projected. A conflicted save leaves its payload parked with state 'none', so index/search now count conflicts among unflushed paths and say so in their warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): staged app renames diff as path; flush classic-app cells at their real keys A staged rename (draft_path) changes where deploy lands an app, so both app kinds now compare `path` on both sides — a rename-only draft diffs instead of reading "matches deployed". classicAppDraftParts returns the staged path separately from the grid. Item mode resolves each draft kind's own storage path and additionally asks the listing which row owns a friendly/renamed path — a renamed classic app's cell lives at its ORIGINAL storage path, which only the listing knows — so pending/failed/auto-save-off edits are flushed and probed at the real keys. The appDiffSides rationale comment is compressed to the repo's four-line limit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): never claim folder parity the API cannot prove folder.display_name exists only as a DB column — no folder endpoint returns it — so an identical projection cannot prove a fork folder matches its parent. Fork index and item reads for folders now say the display name is not exposed and may be what differs, instead of "content matches parent". Exposing the field on getFolder is a backend follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): gap-free patch pagination; forced-fresh comparisons never join older fetches When the char backstop cut inside a patch window, the continuation offset still pointed past the requested window — silently skipping the undelivered lines forever. windowPatch now cuts at the last complete line and continues exactly there (a single over-budget line is delivered truncated and stepped past so pagination always advances). fetchWorkspaceComparison treats an in-flight request as being as old as its start: maxAgeMs now gates joining it, so a freshness-forced post-mutation read (maxAgeMs 0) always issues its own fetch instead of adopting a tally that began before the mutation, and a superseded request can no longer clobber a newer cached result. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): generation-ordered comparison writes; path-only fork reads for enum-less kinds Concurrent comparison requests can share a Date.now() value, letting a superseded request's late result overwrite a newer one and be reused for 30s — cache writes are now ordered by a monotonic request generation. Test pins the same-millisecond race with the newer request resolving first. Fork comparison kinds outside the chat type enum (folder, resource_type, …) were listed and even advertised as readable but no call could reach them: a fork item read without `type` is now a path-only wildcard (ambiguous paths list their kinds and ask for type), messages label entries by their comparison kind, and pending index lines for enum-less kinds advertise the path-only read. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): grid-based wrapper detection, comparison invalidation on mutations, multi-kind wildcard reads, honest hidden-diff summaries The classic-app wrapper heuristic keyed on metadata keys the editor mirrors into every bare grid — a grid with a component named `value` was reduced to that component. `grid` presence is the discriminator: a bare App always has it, a legacy wrapper never does. invalidateWorkspaceDrafts now also drops cached fork comparisons for the workspace, so the FIRST post-deploy fork read cannot reuse a banner-prewarmed pre-deploy tally (the snapshot-baseline check only covered subsequent reads). Wildcard fork reads return a section per matching kind instead of an unactionable "pass type" for kinds the chat type enum cannot name, and the fork index never summarizes ACL-hidden differences as parity — hidden counts stay directional (a conflicted item counts in both). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): address cubic review batch — invalidation scope, races, edge output Comparison cache: invalidation matches either side of the pair (a parent deploy moves its forks' tallies), fences in-flight requests (no new joins, late results rejected via a per-key generation floor), and the map is LRU-capped. Eviction moves from every drafts-version bump to deploy success only — draft saves never move the deployed tally. Fork snapshots also baseline the PARENT's drafts version. Draft materialization carries a stale-generation token so a save landing mid-fetch discards that run's pre-save result instead of repopulating the invalidated entry; a save/delete also expires the fork cache's hasLocalDraft join. onAnySaved listeners are error-isolated (a throwing listener must not mark a committed save failed) and the pagehide keepalive flush notifies them on dispatch. Output edges: folder fork lines drop the empty parenthetical, and a patch-window offset past the end reports itself instead of an impossible range. The eval case pins the diff call's path argument. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai): one fencing primitive per cache instead of per-surface races Review rounds kept finding pairwise races between async producers and invalidation — each patched with its own fence. Replace the class: - diffSnapshot: a per-workspace mutation epoch, bumped by every invalidation. Both reconcilers run a bounded retry loop — joiners re-validate after awaiting, producers refuse to store results whose inputs predate a mutation. Covers in-flight listing adoption and pre-deploy fork tallies in one mechanism. - workspaceComparison: per-WORKSPACE generation floors (either side of a pair). Any request started before an invalidation is fenced from joining and from landing in the cache — including superseded requests the inflight map no longer tracks. Also: delete_workspace_item invalidates comparisons like deploy does (deployed state moved), and empty FILTERED indexes say the filter matched nothing instead of claiming workspace/fork parity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): invalidate comparisons on every direct deploy; keep secret caveat with metadata changes Direct chat deploys (schedule/trigger/resource/variable/app) bypass deployDraftToWorkspace and never evicted cached fork comparisons — the shared deploy tail now invalidates before the fallible draft cleanup. A secret variable whose metadata also changed produced a non-empty patch that silently dropped the value-uncomparable caveat; item reads, the index, and fork sections now keep the caveat alongside the patch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): scope diff caches to the authenticated identity; derive fork freshness from the comparison store An SPA logout/login left workspace-keyed diff caches (per-user drafts, permission-filtered fork patches) readable by the next account — both cache modules now wipe on identity change, with a global generation floor fencing requests started under the previous account. The fork snapshot stamped its own fetchedAt over a comparison that could already be near expiry, compounding the two 30s windows, and survived comparison-store invalidation when draft cleanup failed after a deploy. It now carries the comparison's own fetchedAt/generation and stops reuse the moment the store fences it. Delete-item invalidation moved before the fallible draft cleanup for the same reason. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): fence fork-reconciliation joins across account switches ForkCache lacked the epoch stamp WorkspaceCache carries, so a joiner arriving after an identity change (or any epoch bump landing before it) compared its own post-bump epoch against itself and adopted the old producer's in-flight tally. The cache now records its producer's epoch for the joiner and reuse gates, and an identity change also discards the in-flight reconciliation maps so no cross-identity join exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): surface swallowed fork-side fetch failures; include conflicted editor edits in item diffs The shared getItemValue reads {} for any failed fetch, so a transient API failure on a fork side rendered as a fabricated one-sided diff (or parity when both sides failed). A fork side is only fetched when the comparison lists it as existing, so an empty read now raises and shows as a fetch-error entry. Item reads promised conflicted local edits (the index says so) but the local-override branch only covered autosave-off and failed saves — a conflict silently fell back to the persisted draft. Conflicts now read the in-memory editor value too, with a caveat naming which side is shown either way. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): report failed diff materializations as unsearched instead of silently omitting them A side-fetch failure left an index entry with status 'error' and no patch; diff search skipped it and still presented definitive no-match or complete-count results. Failed entries are now listed in a warning naming what was not searched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Guilhem
|
572d69e5ae |
feat(ai): open the Compare & Deploy page from chat with item preselection (#10232)
* feat(ai): open the Compare & Deploy page from chat with item preselection * fix(ai): label the compare link card outside sessions * fix(ai): scope untracked-chat compare links to explicit items * style: drop narration comment on compare mask precedence * fix(ai): match compare items mask against parked live-draft paths * fix(ai): land maskless-mode compare on the view holding the masked drafts * fix(ai): honor explicit fork mode over the draft-mask heuristic * docs(ai): describe mask-aware compare mode auto-pick * fix(ai): match legacy app fork diffs under their identity mask key |
||
|
|
0593ff7d7d |
feat(ai): add npm package search tool to global chat (#10204)
* feat(ai): add npm package search tool to global chat Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai): encode npm search query before building registry URL Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(ai-evals): add global-mode npm package search case Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
 hugocasa
|
0e04bc6991 |
fix: steer ai chat away from draft-blind api catalog reads and runs (#10202)
* fix: steer ai chat away from draft-blind api catalog reads and runs * fix: support api catalog tools in ai eval harness mock backend * fix: block remaining draft-blind read and list endpoints in api catalog * fix: scope eval fetch stub to handled benchmark api paths * feat: add deployed version read option to read_workspace_item * fix: include input schema in script workspace item reads * feat: add pagination to list_workspace_items * fix: paginate list_workspace_items per item type without cross-type loss * fix: window draft overlay in list_workspace_items by page and limit * refactor: simplify list draft overlay to capped page-1 merge * fix: label server-synthesized draft-only rows as drafts in listings * chore: fix stale eval comment and pin draft_only listing label |
||
|
Guilhem
|
f4308cf033 |
feat(ai): expose get_db_schema tool in global chat (#10207)
* feat(ai): expose get_db_schema tool in global chat * fix(ai): skip cross-workspace editor cache write in global get_db_schema * test(ai): add global eval case for get_db_schema resource lookup |
||
|
|
0ea570570e |
feat(ai-sessions): CRUD markdown artifacts in sessions (#10046)
* feat: add IndexedDB persistence layer for AI-chat artifacts * feat: add reactive store for AI-chat artifacts * feat: add artifact chat tools and wire store lifecycle * feat: add markdown artifact viewer with source toggle * feat: surface session artifacts in the preview panel and chat list * feat: tell the copilot when to use artifacts in the session prompt * test(ai_evals): add artifact case and wire artifact helpers for session context * fix(copilot): keep in-memory artifacts across same-session resyncs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: unify session composer edits/artifacts/jobs into a status line Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: add an artifacts section to the session preview picker Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat: share markdown prose presets and restyle the artifact viewer Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor: unify session status popovers into one keyboard-navigable shell Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: reset first-block top margin in all markdown prose presets Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: open the preview picker on the artifacts branch for an active artifact Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: keep artifact picker scope independent of branch hydration state Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Guilhem Lemouel <guilhemlemouel@gmail.com> |
||
|
GuilhemandClaude Opus 4.8
|
6b01caaf26 |
fix(ai): flow writer builds approval steps as scripts, not identity (#9985)
* fix(ai): flow writer builds approval steps as scripts with getResumeUrls, not identity Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): accept rawscript or script for approval step type The flow-writer prompt allows an approval step to be `type: rawscript` or `type: script`, but the topLevelStepTypes check pinned an exact `rawscript` match, so a valid `type: script` approval would fail deterministically. Let the check accept a list of allowed types. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
GuilhemandClaude Opus 4.8
|
4bb82ad6cd |
feat: open runs/schedules pages from AI chat in session preview tabs (#9976)
* feat(copilot): open runs/schedules pages in session preview tabs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): drop buggy in-place nav, always chip outside a session Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): open_page covers variables/resources/assets/audit-logs/settings, perm-gated Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): open_page adds folders, groups and all trigger kinds (EE-gated) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): close_page tool to close session preview tabs Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): fail-closed on unavailable trigger_kind in open_page handler Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): gate open_page on operator_settings, keep open_preview mention preview-only Gate the open_page page set on the workspace operator_settings for operators (mirrors OperatorMenu) instead of hardcoding runs/assets, with an empty-enum guard. Also move the open_preview cross-reference out of the always-on prompt line into the preview-gated block so it isn't advertised when preview tools are off. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): gate open_page on the session's operating workspace A session chat targets its own (possibly forked) workspace while $workspaceStore stays on the navigation workspace, so operator_settings must be read for the operating workspace, not the global store. Thread it through GlobalToolHelpers so both setSchema (advertised enum) and the handler guard gate on the same workspace; the global side-panel chat still follows the live store. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
056ebdb035 |
fix: read chat drafts via own-draft route so drawer-kind drafts deploy (#9913)
* fix: read chat drafts via own-draft route so drawer-kind drafts deploy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover trigger and resource chat-draft read/deploy regressions Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: cover non-secret variable chat-draft read/deploy regression Completes the drawer-kind matrix from the review notes on #9913: schedule, trigger, and resource already had full write→read→deploy regressions; this adds the variable one (non-secret — the secret flow deploys through the ephemeral in-memory value and is pinned by the existing ephemeral tests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai_evals): mock getOwnDraft so eval draft hydration stays in-memory The frontend eval adapter intercepts DraftService for benchmark workspaces, but only updateDraft/getDraftForUser/listDrafts. Global eval output collection hydrates draft values through getGlobalDraft, which reads via getOwnDraft — so draft-producing global cases fell through to the real generated client instead of the in-memory benchmark store. Adds a getBenchmarkOwnDraft helper (null on miss, mirroring the 200/null route semantics), wires it into the adapter mock, and pins it in mockBackendDrafts.test.ts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Ruben Fiszel <ruben@windmill.dev> |
||
|
Ruben FiszelandClaude Opus 4.8
|
c91027824b |
feat(pipeline): AI-chat data-pipeline editor (route + in-session) + home surfacing (#9805)
* feat(pipeline): AI chat tools to build pipeline nodes with diff/approval Add a data-pipeline AI chat experience modeled on the flow editor and surfaced through the dev-gated global chat (no new chat panel). The /pipeline editor registers PipelineAIChatHelpers on the AIChatManager; while it is open the global mode layers pipeline tools, a pipeline prompt section, and the helpers on top of the full global tool set (behavior is unchanged when no pipeline editor is open). New tools (frontend/src/lib/components/copilot/chat/pipeline/core.ts): - get_pipeline_graph / read_pipeline_node — read the live graph and bodies - build_pipeline_node / edit_pipeline_node — stage changes as AI-pending drafts - remove_pipeline_node — drop a staged proposal - test_pipeline_node — preview-run a node (requires confirmation) Tools never deploy: they stage drafts flagged aiPending, rendered on the canvas with an accent ring and reviewed via Accept all / Reject all (the flow editor's GlobalReviewButtons). Accept commits the drafts; Reject reverts to a pre-AI snapshot, preserving earlier accepted drafts. Auto-accept is gated on the chat autonomy mode. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): teach the global/session chat to author data pipelines Without an open /pipeline editor the session chat had no pipeline concept, so "create a data pipeline" loaded flow instructions and built a flow. Add a first-class pipeline authoring path: - system_prompts/base/pipeline-base.md — what a data pipeline is (a DAG of annotated scripts wired by storage assets, NOT a flow) and how to author the // pipeline / // on / // materialize annotations; wired through generate.py as getPipelinePrompt() (regenerated prompts.ts/index.ts). - global/core.ts — new get_instructions subject "pipeline", and a global-prompt rule disambiguating data pipelines from flows so the model routes correctly. - ai_evals/cases/global.yaml — two global cases (single node, two-node chain) asserting pipeline-annotated script drafts and forbidding write_flow, guarding the pipeline-vs-flow conflation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline): show & build pipelines in the AI session preview Add a 'pipeline' session preview target so the session AI can show the data-pipeline graph for a folder and build nodes in-pane: - open_preview now accepts kind="pipeline" (path = folder); SessionTarget / EDITOR_TARGET_KINDS widen accordingly. The slot/codec load model stays flow|script|raw_app — pipeline bypasses it with its own fetch/draft state. - New PipelineEditorView.svelte mounts in the session pane: fetches the folder graph, overlays AI drafts, renders AssetGraphCanvas + the Accept/Reject review buttons, and registers PipelineAIChatHelpers on the *session-scoped* manager (via getAiChatManager) so build_pipeline_node / edit_pipeline_node + the diff/approval work inside the session too. - System prompt nudges the model to open the pipeline preview and use the staging tools while building. Verified end-to-end with a real model: the session AI called open_preview, the graph mounted in the side panel, then build_pipeline_node staged a node on the session canvas with its schedule trigger and ducklake output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): share the AI editor logic between route page and session Consolidate the duplicated data-pipeline AI logic onto a single shared layer so the route editor and the in-session preview behave identically and the session gains the full code editor. - New pipelineAiHelpers.ts: createPipelineAiHelpers(deps) owns the propose/edit/ remove/accept/reject/test staging + the per-turn snapshot bookkeeping that powers Reject. Callers inject accessors for their own draft Map and graph. - Route page (/pipeline/[folder]) drops its ~250-line inline AI-helper block and wires the shared factory via deps (folder/workspace/graph/drafts + focus, ensureEditable, run-started). Its shell — persistence, navigation guard, activity, cascade, trigger drawers — is untouched. - Session PipelineEditorView uses the same factory and now renders the real AssetGraphDetailsPane (code editor + live overlays + test), so a node built in a session opens with its source, matching the route editor. Verified: route page hydrates/renders drafts unchanged; in a session the AI opened the pipeline preview, built a node, and its code showed in the details pane. check:fast clean, 197 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): externalize editor state into PipelineEditorState (step 1) Introduce PipelineEditorState — the data-pipeline analogue of the flow editor's flowStore. It owns the draft Map, the live editor overlays, and the selection, with callback-safe methods (handleDraftPersist / handleAnnotationsChange / … ), so a single editor can be rendered by both the route page and the session. This commit lands the store and points the in-session PipelineEditorView at it (no behaviour change — the session already had these inline). Next steps move the route page onto the store and a shared <PipelineGraphEditor>. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): point the route editor at PipelineEditorState (step 1) Move the route page's draft Map, live editor overlays, selection, and the draft-persist / live-change handlers onto the shared PipelineEditorState (`pe`), referencing them as `pe.*` in place. No behaviour change — persistence, graph resolution, run dispatch, AI staging, and deploy all stay on the page and now read/write the externalized state. This is the data-pipeline analogue of the flow editor's flowStore: the route page and the in-session preview now share one source of editor truth, setting up the shared <PipelineGraphEditor> in the next steps. Verified: the page hydrates its DB draft, renders the overlay graph, the toolbar counts (Save all (N)) track pe.drafts, and selecting a node opens it in the details pane. check:fast clean, 84 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the route editor via shared PipelineGraphEditor (step 2) Extract the canvas + details-pane editor body into PipelineGraphEditor.svelte, the data-pipeline analogue of FlowBuilder. The route page now delegates its Splitpanes block to it, passing the externalized PipelineEditorState plus its run/cascade/trigger/deploy callbacks; the component owns pane sizing, selection/details-open derivation, and the canvas+details rendering. Root-caused the earlier ts2769 "$props() No overload" to a prop named `state` colliding with the `$state` rune (`let x = $state(...)` parsed as a store auto-subscription on the prop) — the prop is now `editor`. Net: the route page sheds ~310 lines of template/state; behaviour preserved. Verified: the page hydrates its DB draft, renders the graph, opens the draft in the details pane (live code editor + Test), pane sizing works. check:fast clean, 24 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): move draft autosave into PipelineGraphEditor (step 3) Fold the per-user `data_pipeline` DraftService bundle autosave (hydrate + debounced persist + localStorage crash mirror) into PipelineGraphEditor, gated by a `persistDrafts` prop — FlowBuilder's parameterized-autosave shape. The route page passes `persistDrafts` + `folder` and reads `editor.loadedFromDbDraft` for its AutosaveIndicator; the in-session preview will leave persistence off. Also restores the `untrack(...)` wrapping on the pane-sizing $effect (dropped when the editor body was extracted in step 2). Without it the Pane `bind:size` feedback loops the effect and pegs the main thread when the details pane is closed — a latent hang in the step-2 commit. check:fast clean, 24 pipeline tests pass. Note: browser revalidation was not possible this session (the Playwright MCP browser was reset); the autosave is a verbatim port and the untrack fix is the original working form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(pipeline): render the session preview via shared PipelineGraphEditor (step 4) Point the in-session PipelineEditorView at the shared PipelineGraphEditor instead of its own inline canvas + details pane. The session now renders the exact same editor body as the route page — gaining the full details/code pane — while opting out of persistence (persistDrafts=false) and the run/cascade/trigger/bounded affordances (their callbacks are omitted, so those controls hide). Building nodes + the Accept/Reject diff still work via the AI helpers. Also fixes issues surfaced by a full `svelte-check` while wiring this up: - PipelineGraphEditor: edit mode opened the details pane unconditionally (a step-2 regression); restored the route's "open only on selection/draft" behaviour. - Route page passed an `isOperator` prop the component doesn't accept (step-2; caught only by full check, not check:fast). - SessionItemNotFound: narrow its `kind` to exclude `pipeline` (pipeline targets never slot-load, so they can't 404 through it) — closes the SessionTarget-widen fallout. - PipelineEditorView: cast the resolveGraph base to AssetGraphResponse. Full `svelte-check` now clean across all pipeline/session files; 137 unit tests pass. (Browser revalidation still pending — Playwright MCP was unavailable this session; see the smoke-test note on the PR.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): stop an infinite microtask loop when persisting a no-output draft handleDraftPersist short-circuits when the open draft's content + inferred writes are unchanged. The writes check compared `d.outputAssets?.length === writes.length`, but a no-output draft has `outputAssets: undefined` (so `?.length` is `undefined`) while the details pane infers an empty `writes: []` (length 0). `undefined === 0` is false, so it never short-circuited: every persist re-wrote the drafts Map with an equivalent object, which gave `activeDraft.script` a new identity → the pane re-emitted its overlays → the graph re-derived → persist fired again. A self- sustaining microtask loop that pegged the renderer and froze the tab on any pipeline carrying a no-output draft (e.g. hydrating one from the saved data_pipeline draft on load). It hangs rather than throwing effect_update_depth_ exceeded because it cycles across microtasks, not within one reactive flush. Fix: coalesce the undefined length to 0 so "no outputs" compares equal to an empty inferred-writes list. Adds pipelineEditorState.test.ts covering the idempotency (fails without the fix) plus the change/no-change cases. Root-caused by instrumenting the reactive churn: every iteration reassigned drafts/liveContent/liveBodyAssets/liveAnnotations/displayGraph with identical values — pure reference churn off the drafts re-write. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): make the agent open the pipeline editor before building nodes In a session, the GLOBAL system prompt only *advised* opening the pipeline preview ("show its graph with open_preview ... prefer those tools once it is open"), so the agent routinely skipped it: on a plain "build a data pipeline" request it reached for write_script and staged plain script drafts, and the canvas editor never opened. build_pipeline_node / edit_pipeline_node are only registered once the preview is open, so skipping open_preview also loses the canvas-staged Accept/Reject diff-approval flow entirely. Make the guidance imperative: open_preview(kind="pipeline", path=<folder>) is the FIRST step before creating any node (an empty or not-yet-created folder is fine — create_folder first if needed), and pipeline nodes go through build_pipeline_node / edit_pipeline_node, never write_script. This also clears the agent's "the folder might not exist" hesitation that pushed it toward write_script. Verified live (same plain prompt, before/after): before it used write_script with no editor; after, the agent opens the editor first and stages a canvas-highlighted node with Accept all / Reject all. The guidance is gated on previewTools (session-only), so it doesn't affect the non-preview global eval cases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(pipeline): preserve in-session pipeline drafts across editor hide/show The session preview's PipelineEditorState lived in the PipelineEditorView component with persistDrafts=false. Hiding the editor sets editorVisible=false, which makes `hasEditor` false and the `{#if hasEditor}` block unmount the view — discarding its component-local store. Showing it again remounted a fresh, empty one, so the pipeline the AI had built in the session vanished. Move the PipelineEditorState onto the per-session SessionRuntime (like the flow / script / raw_app editors, which already host their state there and take {runtime}), so it survives the pane unmount on hide and across session switches. The runtime is keyed by session id and only dropped on session deletion. Because the instance is now reused, guard against a retarget to a different folder: PipelineEditorView resets the state when `path` changes to a new folder (a same-folder remount keeps the drafts). Adds `folder` + `reset()` to the store. Verified: build a node in a session → Close editor → Show editor → the staged node, its wiring, the details-pane code, and Accept/Reject all re-appear. Full svelte-check clean; 139 pipeline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(pipeline-ai): clearer diff + persistent review banner on the canvas The AI review affordance had two problems on the pipeline canvas: - The floating Accept-all / Reject-all bar sat bottom-center, where it collided with the minimap once the canvas narrowed on node selection — reading as "the buttons vanished when I select a node". - Every staged draft rendered with the same blue ring, so it wasn't clear what the review would actually change (a plain manual draft looked the same as an AI proposal). Replace the floating bar with a top-left review banner (z-30, clear of the controls and minimap) that stays put regardless of selection and spells out the pending counts. Color the diff per node: a proposal that adds a node that isn't deployed rings green with a "new" chip; one that edits an already-deployed node rings amber with an "edited" chip. Plain manual drafts keep the neutral gray dashed border, so only the green/amber nodes read as part of the Accept/Reject set. aiPendingKind is resolved in resolveGraph (deployed runnable present → modified, else added) and forwarded through the canvas to the node. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): persist in-session pipeline proposals across reload/switch Staged AI proposals lived only in the per-session runtime's in-memory PipelineEditorState (persistDrafts=false), so a page reload — and an LRU-evicted runtime on session switch — dropped them, leaving the canvas and the Accept/Reject review empty even though the chat still showed the nodes as staged. Enable the same per-folder DB-draft persistence the route page uses for the in-session editor. To keep hide/show cheap and race-free, hydration is now gated per editor instance (PipelineEditorState.hydratedFromDb) rather than per component mount: the runtime-hosted instance hydrates ONCE when fresh (reload / evicted runtime) and then keeps its in-memory drafts across the editor pane unmounting on hide — re-reading the DB on every remount would race a not-yet-flushed autosave and drop a just-staged draft. A folder retarget resets the flag so the new folder re-hydrates. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): make Reject all work for rehydrated proposals rejectAll only reverted paths tracked in the in-memory aiSnapshots map, which is rebuilt empty on each editor mount. After a reload (or session switch into a fresh runtime) the proposals are restored from the persisted draft but have no snapshot, so Reject all was a no-op on exactly the nodes it should discard. Sweep any still-pending draft without a snapshot and discard it (revertPath with no snapshot deletes the path; for an edit of a deployed node that correctly falls back to the deployed body). Adds unit coverage for accept/reject including the no-snapshot case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep proposals visible while the graph reloads on switch The session editor pane is LRU-capped (MAX_WARM_EDITORS), so returning to a session whose pane was evicted remounts PipelineEditorView with a fresh graphRes resource (loading=true, current=undefined). The deployed-graph loading spinner gated the whole canvas, so the staged proposals and the Accept/Reject review banner vanished until the re-fetch resolved — read as "the proposal disappears when I switch sessions". Only show the loading/error placeholder when there are no drafts to display. When the runtime already holds staged drafts, render the editor immediately: resolveGraph overlays them on an empty base so the proposals + banner stay visible, and the deployed nodes fill in when the fetch completes. Verified with a 4s-delayed graph fetch — proposals render through the load with no spinner. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): apply AI node edits directly as drafts, no approve/reject The canvas-level Accept all / Reject all review (aiPending proposals, the green/amber diff ring + "new"/"edited" chips, and the review banner) didn't fit the pipeline editor. Match the flow/script editor instead: build/edit apply directly as ordinary unsaved drafts on the canvas, which the user then deploys — there is no separate approval step. Removed across the surface: - aiPending / aiPendingKind on the runnable node + resolveGraph seeding + canvas forwarding; AI-built nodes now render with the existing plain unsaved-draft dashed styling. - the review banner, count derivations, and hasAiPending/onAccept/onReject props from PipelineGraphEditor and both consumers (route page + session view). - acceptAll/rejectAll/hasPending and the per-turn snapshot bookkeeping from the shared helpers; removeProposedNode now just discards the unsaved draft at a path (undo a build). acceptAllProposals/rejectAllProposals/ hasPendingProposals dropped from the PipelineAIChatHelpers interface and the manager's auto-accept hook. - accept/reject language from the tool descriptions, return messages, and the system-prompt section. Tests updated; pipeline + AssetGraph suites pass (142). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(drafts-diff): support data_pipeline diffs + fix blank empty-summary row Two issues in the session "Drafts" diff drawer (DraftDiffDrawer): - Clicking a `data_pipeline` bundle row threw "Draft diff not supported for kind data_pipeline" (utils_draft_deploy.ts) — there was no handler for the kind, so it fell to the OVERLAY_GETTERS lookup and errored. The bundle has no deployed counterpart (each node deploys individually as a script), so diff it node-by-node: surface each node's draft body keyed by path, folding in the deployed body as the "before" when a node edits a deployed script. - A draft row whose summary is an empty string (e.g. the app draft) rendered with no title at all: WorkspaceItemRow's single-line branch used `summary ?? secondary`, and `??` doesn't treat '' as absent, so it showed the empty summary instead of the path. Use `||` so an empty summary falls back to the path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(drafts-diff): explode data_pipeline bundle into per-node subitems A data_pipeline draft is a bundle of node-script drafts, so a single row diffed the whole thing as one blob. Explode it in DraftDiffDrawer into one script row per node, nested under the bundle's `…/data_pipeline` folder so they read as the pipeline's subitems — each with its own path and a proper script Content/Metadata code diff. The node's draft body is the "after"; its deployed body (when the node is already deployed) is the "before", so edits show as line diffs and new nodes as added. A single bundle row (via the getDraftDiffValues data_pipeline fallback) is kept only for the case where the bundle can't be read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(pipeline-ai): simplify — drop vestigial approve/reject scaffolding & redundant field Review pass over the PR, removing complexity left from the approve/reject removal and the shared-component refactor (all behavior-preserving): - Inline the `acceptPendingEdits` pass-through into `acceptPendingFlowEdits` and revert the now-inert `autoAcceptEditsAvailable` GLOBAL+pipeline widening (pipeline edits are direct drafts — nothing to auto-accept). - Fix the global system prompt: pipeline tools "apply directly as unsaved drafts (no accept/reject)", not "proposals the user Accepts or Rejects". - Collapse the redundant `outputAsset` (singular) into `outputAssets`, removing a whole resolveGraph fallback tier; simplify propose/editNode. - Drop the single-field `PipelineAiHelpersHandle` wrapper (callers just destructured `{ helpers }`); inline the misleading `isoNow()` helper. - Remove the now-unreachable `data_pipeline` branch in getDraftDiffValues (the drafts drawer explodes bundles per-node; an unreadable bundle is skipped) and the "Step N consolidation" drafting narration. - Un-export internal-only types; reuse `storageKey`; refresh stale comments that still referenced proposals / the review banner / diff-approval. svelte-check clean; 141 unit tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): tooltip clarifying the Create/Save button deploys The accent button in the asset-graph details pane ("Create" for a new script, "Save" for an existing one) is really a deploy, but had no tooltip explaining that. Add a title — "Deploy this new script to the workspace" / "Deploy your changes to this script" — keeping the create-vs-update label distinction while making clear both deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): document the `materialize` annotation in the pipeline prompt The model invented "materialize run" because the prompt only mentioned `// materialize <uri>` in passing. Spell out what it is in both the in-app pipeline prompt (getPipelinePromptSection) and the base prompt (pipeline-base.md, regenerated): a MANAGED output where the runtime writes the table around a single SELECT (no manual CREATE/INSERT); replace (default) vs `append` vs `key=<col>` strategies; `manual` to opt out (track-only); and its pairing with `// partitioned …` (runs once per partition, `{partition}` token substituted at run time). Explicitly: materialize is an output declaration, not a command — there is no "materialize run". Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): trigger drawers in the AI session preview Bring the route page's native-trigger affordances to the in-session pipeline editor by reusing the shared <PipelineTriggerEditors> (no duplication of the drawer UI). Clicking a "Schedule · Missing — no trigger row" node (or edit/delete on an attached trigger, webhook, data-upload) now opens the same drawers the full editor uses, instead of doing nothing. Draft nodes get the same "save the script first" guard (a trigger row needs a deployed script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline-ai): run buttons + live run state in the AI session preview Wire the per-node Run button and live run-state badges into the in-session pipeline editor, reusing the shared folder-scoped job poll (useActiveRunnableIds) the route page uses — node badges, the event log, and the zero-latency "running" hint all come from it. The session runs one node at a time (preview for an unsaved draft, the deployed version otherwise), skipping the route page's cascade/deploy-queue machinery the AI-session UX doesn't need. Verified: a node's Run button dispatches a job and the badge updates live from the poll. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(pipeline): label the node deploy button "Deploy" (was Create/Save) Users read "Create" and asked whether it deploys. It does — and the main script editor's DeployButton already says "Deploy", so this is the consistent term. Use "Deploy" for both the new-script and existing-script cases; the new-vs-changes nuance stays in the button's tooltip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(home): surface data pipelines as units, including bundle-phase drafts Treat a data pipeline as one home entry instead of scattering its member scripts: - The home "Pipeline · f/<folder>" entry now also covers bundle-phase pipelines — a folder that so far only exists as a `data_pipeline` draft — not just deployed ones, so a pipeline shows up the moment its first node is drafted (union listPipelineFolders + data_pipeline draft folders). - Pipeline-member scripts (`auto_kind='pipeline'`) are filtered out of the individual scripts list; they're represented by their pipeline's entry. - Tree view injects pipeline folders so they (and their "Pipeline" entry) still appear when their only scripts are hidden members or they have none deployed yet. Verified in both list and tree view: app_groups (deployed member folded) and a draft-only nyc_transit both show as pipelines; the member script no longer lists individually. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(scripts): compute auto_kind for draft-only pipeline nodes A never-deployed pipeline node (a script draft starting with `// pipeline`) had no script row, so list_scripts synthesized it with `auto_kind: None` — and the home page therefore couldn't tell it was a pipeline member, listing it individually instead of folding it into its pipeline. Parse the draft content the same way the create path does (`parse_pipeline_annotations(...).in_pipeline`) and set `auto_kind = "pipeline"` on the synthesized draft-only row, so draft nodes fold into their pipeline like deployed members. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(search): hide pipeline-member scripts from global search The Ctrl+k global search listed pipeline-member scripts (`auto_kind='pipeline'`) individually. Filter them out — they're reached through their pipeline, matching the home page. Deployed members carry auto_kind from the script row; draft-only members now do too (computed from draft content in list_scripts), so both are excluded here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address PR review findings Session run dispatch (the one real bug): - runNode now passes `_wmill_skip_asset_dispatch: true` for a single-node run of a deployed node unless the user chose "run + downstream" (cascade) — previously a single Run could fan out to downstream deployed scripts via the backend asset dispatcher and fire side-effecting production runs. - onRunProducer guards `kind === 'script'`; onTestStateChange only clears the run hint for the script the pane finished (not a different in-flight node); clear the hint on folder retarget; gate the background poll on isActiveSession so hidden warm panes don't poll; note the PipelineTriggerEditors workspace coupling. Home page pipeline surfacing: - Fold pipeline-member folders into `pipelineFolders` (captured in loadScripts) so a members-only / draft-only-`// pipeline` folder still shows its pipeline entry instead of vanishing; and don't render the empty-state when only pipelines remain (they aren't part of the text filter). - Insert injected tree folders in name order instead of prepending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): make clear `// materialize` is DuckDB + DuckLake only The model put `// materialize` on a python3 node, which deploy rejects ("only supported for DuckDB scripts"). The prompt only implied SQL ("write the body as a single SELECT") without stating the hard constraint. Spell it out in both the in-app prompt and pipeline-base.md: `// materialize` is DuckDB-only and its target must be a DuckLake table; for python3/bun/postgresql nodes, write the output via the SDK instead and let it be inferred — reach for duckdb when a node should materialize a DuckLake table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): fix stale comment — session now wires run + trigger affordances Addresses review: the comment still claimed the session 'opts out of the run/cascade/trigger/bounded affordances', but run buttons + trigger drawers were wired in. Describe the current state (wires run + triggers; omits only cascade/bounded/add-script). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address Codex review — test_pipeline_node dispatch + tree search - [P1] testNode (the test_pipeline_node tool) ran a deployed node via runScriptByPath without `_wmill_skip_asset_dispatch`, so previewing one node could fan out to downstream deployed subscribers and run side-effecting scripts. Add the skip flag (test is always single-node) + a regression test. - [P2] Home tree view injected pipeline folders — and rendered their Pipeline row — even during a text search, surfacing unrelated pipelines. Gate both the TreeViewRoot injection and TreeView's hasPipeline on `!isSearching`, matching the list view which hides pipeline rows on a query. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): keep the pipeline prompt after update_user_instructions rebuildGlobalSystemMessage (called by the update_user_instructions tool) rebuilt only the base Global prompt, dropping the pipeline-editor section that configureGlobalMode appends. So after the chat remembered an instruction, the next GLOBAL turn lost the active /pipeline/<folder> context + direct-draft/ materialize guidance while pipeline tools stayed registered. Re-append the pipeline section here when a pipeline editor is registered. Addresses Codex review [P2]. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(home): gate pipeline entries by kind/archived/owner filters Codex review [P2]: pipeline rows/folders rendered independently of the item filters, so a pipeline still showed under the Flows/Apps tabs, in the archived view, and outside a selected owner. Add `visiblePipelineFolders` applying the same gates the items get (kind ∈ {all, script}, not archived, owner-prefix match) and route the list rows, tree injection, and empty-state check through it. Pipelines are always `f/<folder>`, so the user-folder toggle and kind=script keep including them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): address review — route folder-switch state, AI node guards, diff identity claude[bot] [P1]: the route page's in-app folder switcher navigates same-route (no remount), but nothing reset PipelineEditorState — so folder A's drafts displayed under B and autosave persisted them into B's bundle, and B never hydrated. Reset pe on folder change (mirror the session retarget), and guard the shared hydrateDrafts against a stale folder result landing after a retarget. codex/claude [P2]: build_pipeline_node (proposeNode) only checked drafts.has — now rejects a path outside the open folder and one colliding with an existing deployed node (model should edit_pipeline_node). + 3 regression tests. codex/claude [P2]: exploded pipeline-node diff rows shared `script/<path>` with a standalone script draft at the same path, colliding in the {#each} key + value cache. Add an explicit unique `key` (the distinct bundle-nested path) on DiffRow; pipeline nodes set/look up by it while `path` stays the real edit target. claude [P2]: session AI test_pipeline_node now arms the live run badge (onRunStarted), matching the route page. nit: pipelineAiHelpers.test uses afterEach(restoreAllMocks) instead of an unreachable inline mockRestore. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline): harden AI node mutations + close home label-filter / rename gaps Codex [P1] (AI mutations trust model paths) — fully scoped now: - editNode validates the open folder too (proposeNode already did), via a shared assertInFolder; an edit_pipeline_node for f/other/* no longer persists an unrelated script into the current folder's data_pipeline bundle. - both build_pipeline_node and edit_pipeline_node now require the `// pipeline` annotation (assertPipelineAnnotation) so a staged draft is definitionally a pipeline member, not a silently-non-member script. + tests. (proposeNode's folder + deployed-collision guards landed in the prior commit.) Codex [P2] home label filter — visiblePipelineFolders ignored labelFilter, so a label selection still showed every pipeline (and the empty-state fell through to render pipeline rows). Pipelines carry no labels, so a label filter hides them. Codex [P2] session rename — PipelineEditorView now wires onScriptRenamed (repoint selection + refetch), matching the route page; a persisted-script rename no longer leaves the canvas on the old path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(pipeline-ai): language-specific comment prefix for annotations Codex [P2]: the tool schema and prompt told the model to write `// pipeline` / `// on` / `// materialize` regardless of language, and pipeline-base.md grouped SQL with `#`. A `//` (or `#`) annotation line is invalid in a DuckDB/Postgres node — it passes the frontend parser (which strips `//`/`--`/`#`) but is a SQL syntax error at deploy/run. Make the guidance language-specific everywhere: `--` for SQL (duckdb/postgresql), `#` for python3/bash, `//` for bun/TS — the `//` in examples is the TS form to translate. Regenerated the prompt outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): re-scope Global prompt on folder switch + language-aware base prompt Codex [P2] x2: - The route page resets editor state on an in-app folder switch, but the Global chat's system message kept the old `/pipeline/<folder>` scope (the helper methods read the reactive folder, but the prompt string is only rebuilt on Global-mode reconfigure). Rebuild it on folder change so the next turn targets the new folder. - The pre-editor base Global prompt (seen before open_preview/get_instructions) still showed TS-only `// pipeline` / `// on`. Make it language-aware (`--` SQL, `#` Python/Bash, `//` TS) so the model can't draft invalid DuckDB/Postgres nodes before the pipeline tools are registered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): authoritative new-node probe + SQL-correct eval checklist Codex [P2] x2: - build_pipeline_node's collision check relied on the resolved graph, which can be empty while the session preview races open_preview (a build could shadow a deployed node before the graph loads) and only covered pipeline runnables, not a non-pipeline script at the same path. Add an authoritative backend probe (ScriptService.getScriptByPath): any deployed script at the path → reject with "use edit_pipeline_node". + regression test (empty graph, deployed script). - The DuckLake eval judgeChecklist required the exact `// pipeline` annotation, which would penalize the now-correct `-- pipeline` SQL output (or reward invalid DuckDB syntax). Make both cases syntax-aware. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): rebuild Global prompt on session preview folder retarget Codex [P2]: open_preview(kind="pipeline", path="B") can retarget an existing pipeline preview from folder A to B without remounting. The retarget effect resets editor state and the helper methods read the new path, but the registration effect only depends on isActiveSession, so the Global system message stayed scoped to /pipeline/A. Mirror the route-page fix: rebuild the global system message on retarget (gated on isActiveSession — only the active session's helpers are registered; a hidden session reconfigures when it next becomes active). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(pipeline-ai): edit_pipeline_node preserves deployed script metadata Codex [P1]: editNode kept only the deployed script's language and staged a fresh makePipelineScript draft with empty hash/summary/description/tag/schema/settings. Deploying that edit from the pane (auto_parent) would update the script while wiping its metadata, and the route "Save all" path (no parent_hash) could hit the backend path-conflict branch on the occupied path. Base the draft on the existing draft's / deployed script object and replace ONLY content (+ inferred output assets), preserving hash and metadata. + regression test. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 centdixandClaude Opus 4.8
|
44c25de418 |
feat(ai-chat): add create_folder tool to global chat (#9819)
Global-mode chat could reference the user's existing folders in the system
prompt but had no way to create a new one, so for shared work where no
existing folder fit it would dead-end on "ask the user" or invent a
non-existent f/<folder>/… path (which fails at deploy).
- create_folder: dedicated, confirmation-gated tool for the immediate
(non-draft) folder mutation; the creator becomes an owner. Mirrors the
backend name validation client-side and returns a minimal { success } result.
- Folder path guidance now steers the model to create a folder only when the
user explicitly asks for one, and otherwise to ask which folder to use for
shared intent rather than guessing or inventing a path.
- ai_evals: in-memory create_folder mock + a create-folder case (global-path5);
path3 maxTurns bumped to give room to ask.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
centdixandClaude Opus 4.8
|
89c46c803e |
docs: add ai-chat and ai-evals skills (#9770)
Add two agent skills under .agents/skills (symlinked into .claude/skills): - ai-chat: guidance for improving the Windmill AI chat / copilot, especially global mode — benchmark before/after with ai_evals, optimize finalContextTokens over cumulative, keep tool params and tool-result payloads minimal (no echoing content the model already has), treat prompts/tool-descriptions as benchmarkable surface. - ai-evals: author and run black-box benchmark cases for the AI generation modes, migrated from ai_evals/AGENTS.md and extended with run mechanics (workspace reuse, reading the summary). ai_evals/AGENTS.md becomes a pointer stub to the skill. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdix
|
ae088fd032 | stabilize global ai eval smoke path (#9745) | ||
|
centdixandClaude Opus 4.8
|
b0ddcf31e4 |
ci: add path-gated AI agent + ai_evals smoke workflows (#9640)
* ci: add path-gated AI agent integration tests workflow Runs integration_tests/ai_agent_tests against real LLM providers (Anthropic/OpenAI/Google) only when AI-agent backend code or the tests change, since runs make paid LLM calls. Adds a conftest fixture that skips provider-parametrized cases whose API keys are absent, so CI exercises only the providers it has secrets for. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: add path-gated ai_evals global-mode smoke workflow Runs the global AI chat eval (global-test1) across one cheap model per provider (Anthropic/OpenAI/Google/DeepSeek) only when the eval harness or copilot chat code change, since runs make paid LLM calls. Builds Windmill CE from source as the AI proxy; global tools/drafts run in the Vitest bridge. Gates on the deterministic draft pipeline (run succeeded + produced a draft + used write_script), not the variable LLM judge score. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run AI smokes on PR ready-for-review instead of every push Switch the pull_request trigger from `synchronize` (every commit) to `ready_for_review`, with a job guard skipping draft PRs, so the paid LLM runs only fire when a PR is marked ready to merge (plus push-to-main and manual dispatch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai_evals): lazily load cli mode so non-cli evals skip the cli toolchain The entrypoint eagerly imported modes/cli, which pulls the wmill CLI guidance modules and their JSR deps (@cliffy/*). Global/flow/script/app runs then crashed with "Cannot find module '@cliffy/ansi/colors'" when the cli workspace deps were not installed. Import createCliModeRunner dynamically inside runCliBenchmark instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai_agent): raise low max_completion_tokens to OpenAI's 16 minimum OpenAI's /v1/responses rejects max_output_tokens < 16 with a 400, failing test_low_max_tokens for openai. 16 still exercises a truncated response. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: run ai_evals workflow on Node 22 for the frontend undici 8.x dep The Vitest bridge loads frontend/node_modules/undici@8.x, which requires Node >=22.19; Node 20 failed with "webidl.util.markAsUncloneable is not a function" when loading vitest.config.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai_evals): run frontend evals autonomously + give global-test1 more turns Frontend evals (flow/script/app/global) ran the production chat prompt, which assumes an interactive human — so cheaper models burned their turn budget asking for confirmation, waiting for approval, or presenting a plan, sometimes hitting maxTurns without producing a draft. Append a shared autonomy note in baseEvalRunner (the path all frontend modes share, mirroring cli mode): act directly on clear requests; only ask on genuinely ambiguous ones (preserving the askUserQuestion cases). Also raise global-test1's maxTurns 8 -> 10 so a model that over-explores still converges. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci(ai_evals): watch draft/prompt deps outside copilot/ The global eval runs production frontend code in-process, so the smoke's behavior depends on files outside frontend/src/lib/components/copilot/**: the draft model (userDraft.svelte.ts, userDraftDbSyncer.svelte.ts), script inference (infer.ts), and the chat system prompts ($system_prompts -> system_prompts/auto-generated). Add them to both push and PR path filters so a change there actually triggers the smoke that gates on draft production. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: skip direct provider tests without credentials * feat: add ai evals skip judge flag * fix: simplify ai evals ci gate * fix: simplify ai evals smoke gate * fix: handle ai eval workflow triggers --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
74a2329d2e |
feat(copilot): improve global-mode path selection + add path-selection evals (#9698)
* test: add global-mode path-selection eval cases with seeded user Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(copilot): guide global-mode path selection with injected folder list Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): tailor global-mode folder guidance for workspace admins Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(copilot): type folders_read; isolate global-eval user from store Addresses PR review: - Add folders_read to the User/whoami openapi schema and UserExt; the global prompt builder and eval harness now read it typed instead of via inline casts (regen the client to pick it up). - prepareGlobalSystemMessage takes an explicit user; the eval harness passes it rather than mutating the process-global userStore, removing the concurrency race (path cases no longer need --verbose). - Rewrite the path-selection case comment as a current invariant. - Add buildFolderGuidance unit tests in core.test.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
4296a6ae1f |
feat(ai-chat): cap read_app_file + search_app grep tool to bound context in large raw apps (#9653)
* docs: add global AI chat context-optimization plan for raw apps Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): add global raw-app debugging cases on a large fixture Adds a ~20-file analytics_dashboard raw-app fixture (incl. a 5k-line data module and a planted wrong-totals bug), two global cases (read-heavy debug + small-edit baseline), app-seed support in the mock backend, directory-fixture loading, and a decorateHelpers seam so read-dedupe is measurable. Records tokenUsage for before/ after comparison of the read-tool optimization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): cap and dedupe read_app_file to bound context in large apps read_app_file now defaults to a head slice (1500 lines / 50k chars) with offset/ limit to page further, and skips resending a file whose earlier read is still in context (per-conversation ledger keyed off the originating tool-call id, so it self-heals after compaction). Bounds the file-content portion of global-chat context when working in large raw apps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): add read-heavy raw-app debug case (large data module) global-test31 induces the model to inspect the 5k-line seedData module, exercising the read_app_file cap/offset path. Baseline ~262k tokens vs ~200k with the cap+dedupe change (-24%). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: record A+B benchmark results and fixed-overhead finding Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): clearer read_app_file past-EOF message + unit tests for cap/dedupe Addresses local-review nits: out-of-range offset now reports 'offset N is past the end of the file' instead of a backwards 'lines 11-10' label; adds unit coverage for the slicing (line cap, offset/limit window, char budget, past-EOF) and re-read dedupe (hit + miss-when-not-retained). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): char-level paging + per-range dedupe for read_app_file Adds char_offset/char_limit so minified/long-line files can be paged within a line window, keys the re-read ledger by range (so reading different ranges no longer collides), and dedupes on the full-file hash (a cached range stub is invalidated when any byte of the file changes, not just the returned range). Tests updated for the char-slice behavior plus single-line capping, char paging, and out-of-window change detection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-chat): add read_app_file context micro-benchmark + re-read eval case Adds a deterministic micro-benchmark (no LLM) that drives read_app_file through a realistic big-project read pattern (large file, re-read, minified bundle, paging) and asserts the cap+dedupe cut returned context >50% vs the old whole-file behavior — isolating the feature's effect from model nondeterminism and guarding against silent weakening. Adds global-test32, a cross-file consistency investigation that revisits overlapping files so re-read dedupe is exercised in a real run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): clarify test32 measures the read cap, not dedupe Verified: sonnet and haiku both read each file once per conversation and retain it, so test32 never triggers read_app_file re-read dedupe. Dedupe is measured deterministically by the micro-benchmark instead. Comment corrected to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): drop read_app_file re-read dedupe, ship the cap only Benchmarking showed the per-conversation re-read dedupe never fires in practice: across sonnet/opus/gpt-5.5/haiku, every model reads each file once per conversation and keeps it in context (0 within-conversation re-reads). It was a correct but unused guard, so this removes the ledger, full-file hash, retention predicate, the AIChatManager wiring, and the eval decorateHelpers seam — keeping the read cap + offset/limit/char paging (A), which is the lever that actually bounds context. The micro-benchmark is now cap-only; test32 is kept as a multi-file read-load case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): add search_app grep tool for global raw-app chat (experimental) Client-side grep over a raw app's frontend files and inline runnables (literal, case-insensitive, optional file_glob/context_lines/max_matches, head-capped). Completes the list -> search -> ranged-read triad. Includes the eval A/B gate (WMILL_AI_EVAL_DISABLE_SEARCH_APP), unit tests + micro-benchmark, and a find-all-usages eval case (global-test33). Experimental: A/B benchmarking shows it is not an unconditional win — it helps on find-all-usages but adds agentic iterations on navigable apps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): accept search_app as a valid file-inspection tool in raw-app cases Add requiredToolsAnyOf alternatives-group to ToolValidationSpec and switch global-test29..32 to it so a model that locates files via search_app instead of read_app_file no longer false-fails the tool assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: remove stale ai-chat context-optimization planning doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): drop read_app_file char paging for a hard char cap The char_offset/char_limit params guarded minified files (a single line over the char budget) but were effectively unused in benchmarks. Remove them and the in-window char paging; keep the hard 50k-char budget and, when a read hits it, tell the model to narrow the line limit (or treat the file as unreadable if a single line exceeds the budget). Proper long-line handling is left as a TODO. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): bake search_app context to 1 line, clarify query is literal Drop the context_lines param (models varied it to little effect) for a fixed SEARCH_APP_CONTEXT_LINES=1, and cap on matching lines instead of pushed rows so max_matches stays accurate with context always on. Sharpen the query description to state it is a literal (non-regex) substring and to suggest the call form (e.g. formatCurrency() to hit call sites and skip formatCurrencyPrecise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): widen baked search_app context to 2 lines Models that set the old context_lines param leaned to 2; match the lean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): count every file with a match in search_app header Move fileHadMatch ahead of the render cap so files whose matches fall past max_matches are still counted (with a regression test). Also swap the raw NUL globstar sentinel for a printable escape (the NUL bytes made core.ts read as binary to grep) and reword two comments to describe current constraints instead of drafting history. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): drop redundant input echoes from app tool results read_app_file and search_app no longer prefix results with the tool name or echo back the caller's own inputs (file path, query, file_glob) — the model already has them from the call args, and the unbounded query echo could push the search result past its output budget. Keeps the useful signals (line range, match/file counts, truncation) and the actionable advice. Also reword max_matches to 'matching lines' since it caps lines (each expands to context rows). Unit tests updated to the new format. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
3f5f211a22 |
add final context size metric to ai_evals harness (#9660)
Record finalContextTokens per attempt: the input-token total of the last model request (input + cache-creation + cache-read), i.e. how full the context window ended up. Complements the cumulative tokenUsage.prompt, which conflates context size with loop-iteration count. Captured generically in the shared frontend runEval via the chat loop's lastIterationUsage, so it covers all frontend modes (global/flow/script/ app), plus CLI mode via the last assistant turn's usage. Aggregated as average and max over passed attempts and printed in the run summary. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
e87ff79ecf |
fix(ai_evals): adapt global eval harness to DB-backed user drafts (#9641)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Fable 5
|
f4425fca9f |
feat(ai-chat): self-hosted docs tools via windmill.dev llms.txt + ask benchmark (#9578)
* feat(ai-chat): add self-hosted docs tools fetching from windmill.dev llms.txt Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ai-evals): add ask benchmark mode comparing inkeep vs llms.txt docs tools Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(ai-chat): fix docs link sanitizer tests to match skip-all-`../` guard Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai-chat): add hybrid full-text docs search tool and ask variant Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(ai-chat): expose docs search tools in the global workspace assistant Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai-chat): drop inkeep/llmstxt arms, keep only hybrid docs search Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(ai-chat): remove docs-tool benchmark write-up Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(ai-evals): remove ask mode, cover docs search via global mode Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * nits * refactor(ai-chat): swap navigator + api copilots from inkeep to search_docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(ai-chat): point read_docs_page empty-path hint at search_docs Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
cfe5119035 |
feat(ai): add list_runs and get_job_logs tools to global chat mode (#9488)
* feat(ai): add list_runs and get_job_logs tools to global chat mode Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai): always suppress ansi hint in get_job_logs, drop misnamed param Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai_evals): add global list_runs and get_job_logs eval cases Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(ai): trim get_job_logs description and format global core Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai): surface list_runs/get_job_logs output as tool result The tools set showDetails but never set message.result, so the details panel rendered "No result yet" even on success. Set result in setToolStatus (logs go in result for get_job_logs). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdixandClaude Opus 4.8
|
343368fb5e |
test: add datatable tool coverage to global AI evals (#9398)
* test: add datatable tool coverage to global ai_evals (stage 0+1) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: add seeded datatable difficulty-ladder global ai_evals (stage 2) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: skipJudge datatable evals and make stringIncludesAnyOf existential Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: make ai_evals datatable mock reflect SQL writes Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
centdix
|
5c20d6b4f7 |
feat: add global ai chat test tools (#9391)
* feat: add global ai chat test tools
* fix: avoid session id in flow test preview
* test: cover global flow preview ids
* test: require script and flow test tools
* fix: harden global flow test fallback
* Revert "fix: harden global flow test fallback"
This reverts commit
|
||
|
centdix
|
f300a716a9 | test: add global chat resource variable schedule evals (#9379) | ||
|
centdix
|
3345837574 | chore: add gpt-5.5 eval model (#9377) | ||
|
centdix
|
9e7eaf3684 | feat: inject active editor into global chat (#9361) | ||
|
centdix
|
e29dfbaa87 |
test: add global chat eval coverage (#9320)
* test: improve global chat eval parity * test: add human-style global chat evals |
||
|
centdix
|
4be930f585 |
refactor: unify AI provider credentials (#9317)
* refactor: use provider credentials for worker builders * refactor: resolve api proxy credentials directly * fix: lazy load frontend eval modes |
||
|
centdix
|
fec4008696 |
fix: preserve ai reasoning content (#9208)
* fix: preserve ai reasoning content * fix: avoid text-only reasoning replay * feat: add deepseek ai eval models |
||
|
centdix
|
d243e0cde8 | align global flow tool arguments (#9146) | ||
|
centdix
|
7a7d246a6e |
test: add global ai eval mode (#9129)
* feat: add global ai eval mode * fix: improve global eval validation feedback |
||
|
centdix
|
a305a74e73 | refactor: require ai evals proxy backend (#9119) | ||
|
|
8196857c8f |
add workflow-as-code skill (#8970)
* feat: add workflow-as-code skill
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: make system prompt freshness self-contained
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Revert "fix: make system prompt freshness self-contained"
This reverts commit
|
||
|
centdixandClaude Opus 4.5
|
b883f9a9d2 |
feat: add ai chat schedule and trigger tools (#8961)
* feat: add ai chat schedule and trigger tools * refactor: use zod for ai chat workspace tools * refactor: let ai provide runnable target fields * refactor: generate ai chat workspace tool schemas * fix: add object type to composed tool schemas * fix: avoid top-level trigger schema unions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: block undeployed workspace ai tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: inject ai workspace tool target Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add ai evals for workspace tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: make workspace tool eval prompts realistic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: surface workspace tool errors * fix: show workspace tool success details * fix: describe workspace tool path format * fix: clarify workspace path examples * fix: tighten workspace tool validation * fix: align workspace tool prompts * chore: mark generated chat schemas * chore: mark generated cli skills --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> |