mirror of
https://github.com/windmill-labs/windmill.git
synced 2026-08-20 08:01:35 +00:00
centdix
4296a6ae1f
* docs: add global AI chat context-optimization plan for raw apps Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): add global raw-app debugging cases on a large fixture Adds a ~20-file analytics_dashboard raw-app fixture (incl. a 5k-line data module and a planted wrong-totals bug), two global cases (read-heavy debug + small-edit baseline), app-seed support in the mock backend, directory-fixture loading, and a decorateHelpers seam so read-dedupe is measurable. Records tokenUsage for before/ after comparison of the read-tool optimization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): cap and dedupe read_app_file to bound context in large apps read_app_file now defaults to a head slice (1500 lines / 50k chars) with offset/ limit to page further, and skips resending a file whose earlier read is still in context (per-conversation ledger keyed off the originating tool-call id, so it self-heals after compaction). Bounds the file-content portion of global-chat context when working in large raw apps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): add read-heavy raw-app debug case (large data module) global-test31 induces the model to inspect the 5k-line seedData module, exercising the read_app_file cap/offset path. Baseline ~262k tokens vs ~200k with the cap+dedupe change (-24%). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: record A+B benchmark results and fixed-overhead finding Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): clearer read_app_file past-EOF message + unit tests for cap/dedupe Addresses local-review nits: out-of-range offset now reports 'offset N is past the end of the file' instead of a backwards 'lines 11-10' label; adds unit coverage for the slicing (line cap, offset/limit window, char budget, past-EOF) and re-read dedupe (hit + miss-when-not-retained). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): char-level paging + per-range dedupe for read_app_file Adds char_offset/char_limit so minified/long-line files can be paged within a line window, keys the re-read ledger by range (so reading different ranges no longer collides), and dedupes on the full-file hash (a cached range stub is invalidated when any byte of the file changes, not just the returned range). Tests updated for the char-slice behavior plus single-line capping, char paging, and out-of-window change detection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-chat): add read_app_file context micro-benchmark + re-read eval case Adds a deterministic micro-benchmark (no LLM) that drives read_app_file through a realistic big-project read pattern (large file, re-read, minified bundle, paging) and asserts the cap+dedupe cut returned context >50% vs the old whole-file behavior — isolating the feature's effect from model nondeterminism and guarding against silent weakening. Adds global-test32, a cross-file consistency investigation that revisits overlapping files so re-read dedupe is exercised in a real run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): clarify test32 measures the read cap, not dedupe Verified: sonnet and haiku both read each file once per conversation and retain it, so test32 never triggers read_app_file re-read dedupe. Dedupe is measured deterministically by the micro-benchmark instead. Comment corrected to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): drop read_app_file re-read dedupe, ship the cap only Benchmarking showed the per-conversation re-read dedupe never fires in practice: across sonnet/opus/gpt-5.5/haiku, every model reads each file once per conversation and keeps it in context (0 within-conversation re-reads). It was a correct but unused guard, so this removes the ledger, full-file hash, retention predicate, the AIChatManager wiring, and the eval decorateHelpers seam — keeping the read cap + offset/limit/char paging (A), which is the lever that actually bounds context. The micro-benchmark is now cap-only; test32 is kept as a multi-file read-load case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(ai-chat): add search_app grep tool for global raw-app chat (experimental) Client-side grep over a raw app's frontend files and inline runnables (literal, case-insensitive, optional file_glob/context_lines/max_matches, head-capped). Completes the list -> search -> ranged-read triad. Includes the eval A/B gate (WMILL_AI_EVAL_DISABLE_SEARCH_APP), unit tests + micro-benchmark, and a find-all-usages eval case (global-test33). Experimental: A/B benchmarking shows it is not an unconditional win — it helps on find-all-usages but adds agentic iterations on navigable apps. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(ai-evals): accept search_app as a valid file-inspection tool in raw-app cases Add requiredToolsAnyOf alternatives-group to ToolValidationSpec and switch global-test29..32 to it so a model that locates files via search_app instead of read_app_file no longer false-fails the tool assertion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: remove stale ai-chat context-optimization planning doc Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): drop read_app_file char paging for a hard char cap The char_offset/char_limit params guarded minified files (a single line over the char budget) but were effectively unused in benchmarks. Remove them and the in-window char paging; keep the hard 50k-char budget and, when a read hits it, tell the model to narrow the line limit (or treat the file as unreadable if a single line exceeds the budget). Proper long-line handling is left as a TODO. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): bake search_app context to 1 line, clarify query is literal Drop the context_lines param (models varied it to little effect) for a fixed SEARCH_APP_CONTEXT_LINES=1, and cap on matching lines instead of pushed rows so max_matches stays accurate with context always on. Sharpen the query description to state it is a literal (non-regex) substring and to suggest the call form (e.g. formatCurrency() to hit call sites and skip formatCurrencyPrecise. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(ai-chat): widen baked search_app context to 2 lines Models that set the old context_lines param leaned to 2; match the lean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): count every file with a match in search_app header Move fileHadMatch ahead of the render cap so files whose matches fall past max_matches are still counted (with a regression test). Also swap the raw NUL globstar sentinel for a printable escape (the NUL bytes made core.ts read as binary to grep) and reword two comments to describe current constraints instead of drafting history. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(ai-chat): drop redundant input echoes from app tool results read_app_file and search_app no longer prefix results with the tool name or echo back the caller's own inputs (file path, query, file_glob) — the model already has them from the call args, and the unbounded query echo could push the search result past its output budget. Keeps the useful signals (line range, match/file counts, truncation) and the actionable advice. Also reword max_matches to 'matching lines' since it caps lines (each expands to context rows). Unit tests updated to the new format. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>