mirror of
https://github.com/whit3rabbit/anyllm-proxy.git
synced 2026-09-22 00:00:50 +00:00
RTK (anyllm_rtk crate): command-aware filtering of tool-result text (test/build/git/log output) via a catalog of 55 declarative filters ported from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream + non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS. RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config. Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive Compression of long client-sent conversation history for OpenAI Chat Completions, the Anthropic translate path, and the Anthropic passthrough path (client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow| live env / admin toggle; live places a cache_control breakpoint at the frontier over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx feature (model fetched on demand, never bundled). New optimizer_* metrics counters and GET/POST /admin/api/optimizer/model endpoints. Both features expose runtime config + Settings UI controls and fail open on any error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and gitignore rules for downloaded ONNX artifacts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
31 lines
1.1 KiB
Rust
31 lines
1.1 KiB
Rust
//! Embeds every `filters/*.json` at compile time so the proxy binary ships the
|
|
//! built-in RTK filter catalog with no runtime file IO. Regenerates when the
|
|
//! filters directory changes. Emits `FILTER_JSONS: &[&str]`.
|
|
|
|
use std::{env, fs, path::Path};
|
|
|
|
fn main() {
|
|
println!("cargo:rerun-if-changed=filters");
|
|
let manifest = env::var("CARGO_MANIFEST_DIR").unwrap();
|
|
let dir = Path::new(&manifest).join("filters");
|
|
|
|
let mut names: Vec<String> = fs::read_dir(&dir)
|
|
.expect("filters dir")
|
|
.filter_map(|e| e.ok())
|
|
.map(|e| e.file_name().to_string_lossy().into_owned())
|
|
.filter(|n| n.ends_with(".json"))
|
|
.collect();
|
|
names.sort();
|
|
|
|
let mut out = String::from("pub static FILTER_JSONS: &[&str] = &[\n");
|
|
for name in &names {
|
|
out.push_str(&format!(
|
|
" include_str!(concat!(env!(\"CARGO_MANIFEST_DIR\"), \"/filters/{name}\")),\n"
|
|
));
|
|
}
|
|
out.push_str("];\n");
|
|
|
|
let out_dir = env::var("OUT_DIR").unwrap();
|
|
fs::write(Path::new(&out_dir).join("filters_generated.rs"), out).unwrap();
|
|
}
|