* test: add global-mode path-selection eval cases with seeded user
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(copilot): guide global-mode path selection with injected folder list
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(copilot): tailor global-mode folder guidance for workspace admins
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(copilot): type folders_read; isolate global-eval user from store
Addresses PR review:
- Add folders_read to the User/whoami openapi schema and UserExt; the global prompt builder and eval harness now read it typed instead of via inline casts (regen the client to pick it up).
- prepareGlobalSystemMessage takes an explicit user; the eval harness passes it rather than mutating the process-global userStore, removing the concurrency race (path cases no longer need --verbose).
- Rewrite the path-selection case comment as a current invariant.
- Add buildFolderGuidance unit tests in core.test.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: add global AI chat context-optimization plan for raw apps
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai-evals): add global raw-app debugging cases on a large fixture
Adds a ~20-file analytics_dashboard raw-app fixture (incl. a 5k-line data module
and a planted wrong-totals bug), two global cases (read-heavy debug + small-edit
baseline), app-seed support in the mock backend, directory-fixture loading, and a
decorateHelpers seam so read-dedupe is measurable. Records tokenUsage for before/
after comparison of the read-tool optimization.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ai-chat): cap and dedupe read_app_file to bound context in large apps
read_app_file now defaults to a head slice (1500 lines / 50k chars) with offset/
limit to page further, and skips resending a file whose earlier read is still in
context (per-conversation ledger keyed off the originating tool-call id, so it
self-heals after compaction). Bounds the file-content portion of global-chat
context when working in large raw apps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai-evals): add read-heavy raw-app debug case (large data module)
global-test31 induces the model to inspect the 5k-line seedData module, exercising
the read_app_file cap/offset path. Baseline ~262k tokens vs ~200k with the cap+dedupe
change (-24%).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: record A+B benchmark results and fixed-overhead finding
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai-chat): clearer read_app_file past-EOF message + unit tests for cap/dedupe
Addresses local-review nits: out-of-range offset now reports 'offset N is past the
end of the file' instead of a backwards 'lines 11-10' label; adds unit coverage for
the slicing (line cap, offset/limit window, char budget, past-EOF) and re-read dedupe
(hit + miss-when-not-retained).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ai-chat): char-level paging + per-range dedupe for read_app_file
Adds char_offset/char_limit so minified/long-line files can be paged within a line
window, keys the re-read ledger by range (so reading different ranges no longer
collides), and dedupes on the full-file hash (a cached range stub is invalidated
when any byte of the file changes, not just the returned range). Tests updated for
the char-slice behavior plus single-line capping, char paging, and out-of-window
change detection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai-chat): add read_app_file context micro-benchmark + re-read eval case
Adds a deterministic micro-benchmark (no LLM) that drives read_app_file through a
realistic big-project read pattern (large file, re-read, minified bundle, paging)
and asserts the cap+dedupe cut returned context >50% vs the old whole-file behavior
— isolating the feature's effect from model nondeterminism and guarding against
silent weakening. Adds global-test32, a cross-file consistency investigation that
revisits overlapping files so re-read dedupe is exercised in a real run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai-evals): clarify test32 measures the read cap, not dedupe
Verified: sonnet and haiku both read each file once per conversation and retain
it, so test32 never triggers read_app_file re-read dedupe. Dedupe is measured
deterministically by the micro-benchmark instead. Comment corrected to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai-chat): drop read_app_file re-read dedupe, ship the cap only
Benchmarking showed the per-conversation re-read dedupe never fires in practice:
across sonnet/opus/gpt-5.5/haiku, every model reads each file once per conversation
and keeps it in context (0 within-conversation re-reads). It was a correct but unused
guard, so this removes the ledger, full-file hash, retention predicate, the
AIChatManager wiring, and the eval decorateHelpers seam — keeping the read cap +
offset/limit/char paging (A), which is the lever that actually bounds context. The
micro-benchmark is now cap-only; test32 is kept as a multi-file read-load case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ai-chat): add search_app grep tool for global raw-app chat (experimental)
Client-side grep over a raw app's frontend files and inline runnables (literal,
case-insensitive, optional file_glob/context_lines/max_matches, head-capped).
Completes the list -> search -> ranged-read triad. Includes the eval A/B gate
(WMILL_AI_EVAL_DISABLE_SEARCH_APP), unit tests + micro-benchmark, and a
find-all-usages eval case (global-test33).
Experimental: A/B benchmarking shows it is not an unconditional win — it helps
on find-all-usages but adds agentic iterations on navigable apps.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai-evals): accept search_app as a valid file-inspection tool in raw-app cases
Add requiredToolsAnyOf alternatives-group to ToolValidationSpec and switch
global-test29..32 to it so a model that locates files via search_app instead
of read_app_file no longer false-fails the tool assertion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: remove stale ai-chat context-optimization planning doc
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai-chat): drop read_app_file char paging for a hard char cap
The char_offset/char_limit params guarded minified files (a single line over
the char budget) but were effectively unused in benchmarks. Remove them and the
in-window char paging; keep the hard 50k-char budget and, when a read hits it,
tell the model to narrow the line limit (or treat the file as unreadable if a
single line exceeds the budget). Proper long-line handling is left as a TODO.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai-chat): bake search_app context to 1 line, clarify query is literal
Drop the context_lines param (models varied it to little effect) for a fixed
SEARCH_APP_CONTEXT_LINES=1, and cap on matching lines instead of pushed rows so
max_matches stays accurate with context always on. Sharpen the query description
to state it is a literal (non-regex) substring and to suggest the call form
(e.g. formatCurrency() to hit call sites and skip formatCurrencyPrecise.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai-chat): widen baked search_app context to 2 lines
Models that set the old context_lines param leaned to 2; match the lean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai-chat): count every file with a match in search_app header
Move fileHadMatch ahead of the render cap so files whose matches fall past max_matches are still counted (with a regression test). Also swap the raw NUL globstar sentinel for a printable escape (the NUL bytes made core.ts read as binary to grep) and reword two comments to describe current constraints instead of drafting history.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai-chat): drop redundant input echoes from app tool results
read_app_file and search_app no longer prefix results with the tool name or echo back the caller's own inputs (file path, query, file_glob) — the model already has them from the call args, and the unbounded query echo could push the search result past its output budget. Keeps the useful signals (line range, match/file counts, truncation) and the actionable advice. Also reword max_matches to 'matching lines' since it caps lines (each expands to context rows). Unit tests updated to the new format.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ai): add list_runs and get_job_logs tools to global chat mode
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai): always suppress ansi hint in get_job_logs, drop misnamed param
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(ai_evals): add global list_runs and get_job_logs eval cases
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* style(ai): trim get_job_logs description and format global core
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(ai): surface list_runs/get_job_logs output as tool result
The tools set showDetails but never set message.result, so the details panel rendered "No result yet" even on success. Set result in setToolStatus (logs go in result for get_job_logs).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: add datatable tool coverage to global ai_evals (stage 0+1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: add seeded datatable difficulty-ladder global ai_evals (stage 2)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: skipJudge datatable evals and make stringIncludesAnyOf existential
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: make ai_evals datatable mock reflect SQL writes
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
centdix