7 Commits

Include renames

Author	SHA1	Message	Date
hugocasaandClaude Opus 5.5	e117871bec	weekly ai evals on current models, and claude 5.5/gpt-6 support (#11409) ... * feat: run ai evals weekly on current models and post results to a dashboard Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat: add current flagship models, a reasoning flag and claude 5.5 defaults Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: never send a reasoning disable claude 5.5 or gpt-6-astra reject, and treat gpt-6 as a reasoning model Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: address review on gpt-6 support, chat completions tools and model metadata Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: leave tiered gpt-6 unpriced and drop the off sentinel on gpt-5 and o-series Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: point the ai_evals readme at the model registry instead of copying it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor: encode the reasoning rules as per-family maps with a shared parity fixture Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: keep the chat completions tools rule open-ended past gpt-5.6 and scope the parity fixture Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-09-30 13:10:24 +02:00
centdixandClaude Opus 4.8	3f5f211a22	add final context size metric to ai_evals harness (#9660) ... Record finalContextTokens per attempt: the input-token total of the last model request (input + cache-creation + cache-read), i.e. how full the context window ended up. Complements the cumulative tokenUsage.prompt, which conflates context size with loop-iteration count. Captured generically in the shared frontend runEval via the chat loop's lastIterationUsage, so it covers all frontend modes (global/flow/script/ app), plus CLI mode via the last assistant turn's usage. Aggregated as average and max over passed attempts and printed in the run summary. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-18 23:21:28 +02:00
centdix	a305a74e73	refactor: require ai evals proxy backend (#9119)	2026-05-12 13:44:00 +00:00
centdixandClaude Opus 4.5	b883f9a9d2	feat: add ai chat schedule and trigger tools (#8961) ... * feat: add ai chat schedule and trigger tools * refactor: use zod for ai chat workspace tools * refactor: let ai provide runnable target fields * refactor: generate ai chat workspace tool schemas * fix: add object type to composed tool schemas * fix: avoid top-level trigger schema unions Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: block undeployed workspace ai tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: inject ai workspace tool target Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: add ai evals for workspace tools Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: make workspace tool eval prompts realistic Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: surface workspace tool errors * fix: show workspace tool success details * fix: describe workspace tool path format * fix: clarify workspace path examples * fix: tighten workspace tool validation * fix: align workspace tool prompts * chore: mark generated chat schemas * chore: mark generated cli skills --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-04-29 14:00:01 +00:00
centdixandClaude Opus 4.5	fddd8e288f	fix: add proxy eval coverage for gemini schemas (#8897) ... * feat: add proxy transport for ai evals Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: strip propertyNames for gemini schemas Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: require explicit eval transport Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-04-21 16:03:44 +00:00
centdixandClaude Opus 4.5	b39671d933	feat: add compact json patch tool to flow chat (#8840) ... * fix: use compact json for flow patches Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: improve flow eval harness Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * test: record flow benchmark history Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: preserve schema in set flow json Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * style: clean set flow json schema guard Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: clean flow patch review followups Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>	2026-04-16 14:31:05 +00:00
centdix	cdcc56461b	feat: add black-box ai eval benchmarks (#8618)	2026-04-13 14:05:46 +02:00