mirror of
https://github.com/whit3rabbit/anyllm-proxy.git
synced 2026-09-21 16:00:49 +00:00
whit3rabbitandClaude Opus 4.8
d155bfd148
RTK (anyllm_rtk crate): command-aware filtering of tool-result text (test/build/git/log output) via a catalog of 55 declarative filters ported from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream + non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS. RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config. Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive Compression of long client-sent conversation history for OpenAI Chat Completions, the Anthropic translate path, and the Anthropic passthrough path (client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow| live env / admin toggle; live places a cache_control breakpoint at the frontier over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx feature (model fetched on demand, never bundled). New optimizer_* metrics counters and GET/POST /admin/api/optimizer/model endpoints. Both features expose runtime config + Settings UI controls and fail open on any error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and gitignore rules for downloaded ONNX artifacts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 lines
1.5 KiB
JSON
3 lines
1.5 KiB
JSON
{"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"I have been reading about the history of the Roman Empire and I found it fascinating how the republic gradually transformed into an empire over the course of several centuries with many political struggles along the way between the senate and various ambitious generals who sought power."},{"role":"assistant","content":"That is indeed a fascinating period. The transition from republic to empire was driven by a combination of military expansion, social inequality, and the ambitions of figures like Julius Caesar and later Augustus who consolidated power and established the principate system that would define the imperial era for centuries to come."},{"role":"user","content":"Can you summarize the main causes in one sentence?"}]}
|
|
{"messages":[{"role":"user","content":"Here is a long document I pasted from a wiki article about photosynthesis: Photosynthesis is the process by which green plants and some other organisms use sunlight to synthesize foods with the help of chlorophyll pigments, converting carbon dioxide and water into glucose and oxygen through a series of light-dependent and light-independent reactions that occur within the chloroplasts of plant cells across the leaves and other green tissues."},{"role":"assistant","content":"Thanks for sharing that context about photosynthesis and how plants convert light energy into chemical energy stored in glucose molecules."},{"role":"user","content":"What is the overall chemical equation?"}]}
|