Commit Graph
4 Commits
Author SHA1 Message Date
hugocasaandClaude Opus 5.5 e117871bec weekly ai evals on current models, and claude 5.5/gpt-6 support (#11409)
* feat: run ai evals weekly on current models and post results to a dashboard

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: add current flagship models, a reasoning flag and claude 5.5 defaults

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: never send a reasoning disable claude 5.5 or gpt-6-astra reject, and treat gpt-6 as a reasoning model

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: address review on gpt-6 support, chat completions tools and model metadata

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: leave tiered gpt-6 unpriced and drop the off sentinel on gpt-5 and o-series

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: point the ai_evals readme at the model registry instead of copying it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor: encode the reasoning rules as per-family maps with a shared parity fixture

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep the chat completions tools rule open-ended past gpt-5.6 and scope the parity fixture

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:10:24 +02:00
AlexRV12andClaude Opus 5 850b028778 feat: advertise the pinned artifact version in get_preview_status (#10691)
* test: let global evals seed the session's preview tabs

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: advertise the pinned artifact version in get_preview_status

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* test: reject ambiguous preview-tab and artifact eval fixtures

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-14 18:41:09 +02:00
centdixandClaude Opus 4.5 b39671d933 feat: add compact json patch tool to flow chat (#8840)
* fix: use compact json for flow patches

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: improve flow eval harness

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test: record flow benchmark history

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: preserve schema in set flow json

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* style: clean set flow json schema guard

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: clean flow patch review followups

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-16 14:31:05 +00:00
centdix cdcc56461b feat: add black-box ai eval benchmarks (#8618) 2026-04-13 14:05:46 +02:00