test(ai-evals): drop global quote-heavy inline-code case

The manual global-mode A/B (Sonnet, Gemini 3 flash/pro, GPT-4o) showed no
pass-rate delta: the GPT-5 inline-escaping failure this change targets does
not reproduce on any available model, so the case guards nothing measurable.
Keep the unit tests as the regression guard instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ruben Fiszel
2026-07-22 14:04:30 +00:00
parent ad1209e0e5
commit 4f2d0c77c1
-29
View File
@@ -1739,32 +1739,3 @@
- creates an AI draft for the existing f/evals/global/format_greeting script with the exclamation-mark change
- the draft changes only the returned message's punctuation — summary, language, path, and the rest of the code are untouched
- does not deploy or save the draft to the workspace
- id: global-flow-quote-heavy-inline-code
prompt: |-
Create a draft flow at `f/evals/global/welcome_email`.
It takes a `name` input and returns a multi-line HTML email body that welcomes the user:
a heading, a paragraph greeting the user by name, and an unordered list of three getting-started tips.
Leave it as an AI draft only; do not deploy or save it.
runtime:
maxTurns: 10
validate:
draftCountExactly: 1
requiredDrafts:
- type: flow
path: f/evals/global/welcome_email
valueIncludes:
- modules
- rawscript
toolExpect:
requiredToolsUsed:
- write_flow
forbiddenToolsUsed:
- deploy_workspace_item
- delete_workspace_item
judgeChecklist:
- creates a flow draft at f/evals/global/welcome_email
- the flow takes a name input and returns an HTML email body
- the returned HTML contains a heading, a paragraph that greets the user by name, and an unordered list of three getting-started tips
- the rawscript step actually contains the multi-line HTML code (it is not left empty or as a placeholder)
- the result stays as an AI draft and is not deployed or saved to the workspace