RTK (anyllm_rtk crate): command-aware filtering of tool-result text
(test/build/git/log output) via a catalog of 55 declarative filters ported
from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control
blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream +
non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS.
RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config.
Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive
Compression of long client-sent conversation history for OpenAI Chat
Completions, the Anthropic translate path, and the Anthropic passthrough path
(client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow|
live env / admin toggle; live places a cache_control breakpoint at the frontier
over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx
feature (model fetched on demand, never bundled). New optimizer_* metrics
counters and GET/POST /admin/api/optimizer/model endpoints.
Both features expose runtime config + Settings UI controls and fail open on any
error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and
gitignore rules for downloaded ONNX artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Large admin-ui refactor (Performative component system, sidebar nav,
provider/route tabs) plus backend module restructuring.
Admin UI contract fixes (this session):
- Fix Models page crash: useBackends unwraps {backends:[...]}; align
ModelEntry to {model_name, deployments} and ModelsResponse.strategy;
fix add-model body to {model_name, actual_model, backend_name}.
- Fix Backends/Providers health rendering: source per-backend status and
latency from the uptime endpoint (health_checks); narrow Backend type to
the real get_backends shape.
- Route + sidebar-link the previously-unrouted Backends tab.
Verified: cargo test (exit 0), clippy -D warnings (exit 0), fmt --check,
admin-ui tsc + vite build all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces anyllm_providers crate as metadata-only catalog (ProviderDef,
ModelDef, registry). Wires any provider-id as BACKEND via OpenAIClient.
Adds bedrock_native.rs (Converse/InvokeModel with SigV4) and
generic_passthrough.rs catch-all for Translate mode. Updates config,
backend clients, streaming, token counting, and compatibility tests
to support the expanded provider surface. Updates model pricing and docs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Wrap admin token in Zeroizing<String> so memory is wiped on drop
- Use SSRF-safe HTTP client for Langfuse and webhook dispatcher
- Wire up webhook dispatcher at startup (was previously un-started)
- Fix batch expires_at: was using now instead of now+24h
- Extract epoch_secs() helper; replace 4 inline SystemTime::now() blocks
- Gemini tool_choice {type:tool}: use ANY+allowedFunctionNames instead of AUTO
- Map Anthropic thinking budget_tokens to OpenAI reasoning_effort
- Preserve temperature/top_p for GA o-series models (o1/o3/o3-mini/o4-mini);
only strip for o1-preview and o1-mini which reject those params
- Azure simple config: always route through default_base_url; guard against
double-appending deployment path when user provides a full URL
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Move format_epoch_iso8601 from queue/sqlite.rs to db.rs (single canonical location)
- Change cancel() to return BatchJob instead of BatchStatus so callers
get the full job without a second query
- Pass batch_webhook_url directly to fire_webhook to avoid re-fetching the job
- Remove is_openai_or_azure_backend (duplicate of is_batch_supported)
- Replace magic literal 3 with DEFAULT_MAX_RETRIES constant
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Proxy batch handlers now use BatchEngine for job lifecycle, file storage,
and webhook delivery instead of direct SQLite calls. Old batch/db.rs
stripped to Anthropic-specific mapping only. Cancel endpoint at
POST /v1/batches/{id}/cancel. BatchEngine initialized in main.rs startup
with second SQLite connection. Cancel integration test added.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
whit3rabbitandClaude Opus 4.8