mirror of
https://github.com/l0ng-ai/openidea.git
synced 2026-11-14 16:07:52 +00:00
l0ng-ai
b2d506a929
pnpm eval drives the production workbench in Chromium against a release openidea-server and real jdtls, judging each scenario (navigation, usages, implementations, gutter markers, lenses, diagnostics, completion, docs, rename) on time to feedback, first and complete result, and checking results against evals/java/golden/. Supporting changes: data-* hooks in the usages popup, hints and rows; a shared in-page Tauri shim for smoke tests and evals; a shared perf bundle builder; language-server versions in .github/tool-versions.env shared with Language smoke; the eval agent skill.
12 lines
565 B
JSON
12 lines
565 B
JSON
{
|
|
"$comment": "Human-owned: agents don't change this file or evals/*/golden/ (a PR that does is labelled needs-decision). `*Ms` are upper bounds on medians (the user's view: input → first painted frame), `recall` / `precision` lower bounds vs the golden. Keys: `<scenario>` or `<scenario>@<condition>`. Regressions vs target/eval/baseline.json need both the relative tolerance and the absolute floor.",
|
|
"regressionTolerance": 0.2,
|
|
"regressionFloorMs": 200,
|
|
"defaults": {
|
|
"feedbackMs": 100,
|
|
"recall": 1,
|
|
"precision": 1
|
|
},
|
|
"scenarios": {}
|
|
}
|