Move Unreleased changelog entries (RTK compression, opt-in prompt
optimizer, release binary archives, Bedrock allowlist fix, model
discovery fix) into the 0.14.0 section. Bump workspace + inter-crate
versions 0.13.0 -> 0.14.0 and the README deb filename.
Fix clippy missing_const_for_thread_local in optimize-core alloc_budget
test (const-init the thread-locals) so cargo clippy -D warnings is clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add [Unreleased] entries for RTK tool-output compression and the opt-in prompt
optimizer, and document their RTK_* / OPTIMIZER_* / MODEL_* env vars and config
keys in docs/ENV.md and docs/CONFIG.md. Also records the already-committed
Bedrock native-route model-allowlist fix and the model-discovery URL fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RTK (anyllm_rtk crate): command-aware filtering of tool-result text
(test/build/git/log output) via a catalog of 55 declarative filters ported
from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control
blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream +
non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS.
RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config.
Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive
Compression of long client-sent conversation history for OpenAI Chat
Completions, the Anthropic translate path, and the Anthropic passthrough path
(client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow|
live env / admin toggle; live places a cache_control breakpoint at the frontier
over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx
feature (model fetched on demand, never bundled). New optimizer_* metrics
counters and GET/POST /admin/api/optimizer/model endpoints.
Both features expose runtime config + Settings UI controls and fail open on any
error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and
gitignore rules for downloaded ONNX artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bedrock native routes (converse/converse-stream/invoke/invoke-with-response-
stream) now enforce the virtual key's model allowlist, closing a model-scope
bypass where a model-scoped key could invoke any Bedrock modelId.
Admin UI "Query models" discovery: stop doubling /v1 into /v1/v1/models for
local providers whose catalog base URL already ends in /v1, trim trailing
slashes off api_base, support Anthropic-native providers (x-api-key +
anthropic-version auth, display_name model field), and show/warn about the
discovery target URL in the Add-Backend form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build-release already compiles x86_64/aarch64 Linux and x86_64 Windows binaries
but only uploaded them as CI artifacts. The release-assets job now packages them
into anyllm-proxy-<ver>-linux-{x86_64,arm64}.tar.gz and
anyllm-proxy-<ver>-windows-x86_64.zip and uploads them to the release, matching
the macOS tarball naming.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The [[bin]] name became anyllm-proxy (hyphen) in 0.12.0, but the Dockerfile
runtime COPY and docker-entrypoint.sh still referenced anyllm_proxy
(underscore), breaking the Docker image build ("/app/target/release/anyllm_proxy:
not found"). Both now use the hyphenated name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump workspace + inter-crate versions 0.12.0 -> 0.13.0, README deb filename,
and move CHANGELOG [Unreleased] into [0.13.0] - 2026-07-12.
Also bump crossbeam-epoch 0.9.18 -> 0.9.20 (Cargo.lock) to clear
RUSTSEC-2026-0204 (invalid pointer dereference), which was failing the
CI Security audit step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI UX fixes plus the route-dispatch feature set:
- Providers: selecting a provider shows the full key/options form immediately
instead of hiding it behind a "+ Add key" button; "Cancel" becomes "Reset".
- Routes: enabled routes show a ready-to-run curl snippet (endpoint URL from the
new proxy_port on /admin/api/status, route name as the model) with a copy button.
- Settings: live proxy status badge (running/unreachable) backed by a proxy_running
TCP liveness check on /admin/api/status, plus a per-save "applied live" toast.
- Backend: plumb proxy listen_port into SharedState; expose proxy_port + proxy_running.
- Also includes the RouteRouter dispatch layer, per-route option overrides, local-LLM
backend support, favorites, and model discovery (see CHANGELOG [Unreleased]).
Ignore .gate/ security-scanner scratch artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI: shared centered provider modal, dropdown backend selectors for Add
Model/Backend, inline add-failure errors, copy/dismiss on created keys, metric
tooltips, empty states. status route now counts managed backends. Add InfoTip
and clipboard helpers. Ignore .eatahorse*/ scratch boards and drop the tracked
guardrails board.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Emit binary as `anyllm-proxy` (hyphen) via explicit [[bin]]; package/crate
name stays `anyllm_proxy`. Threads one hyphenated name through deb/brew/
release archives and CI; drops the rename-in-archive shim.
- Terminal logs default to human-readable on a TTY, JSON when piped
(Docker/systemd). Override with LOG_FORMAT=json|text.
- On loopback admin binds, startup banner prints a tokenized `?token=` admin
URL plus a bare Token line; SPA reads `?token=` to auto-login then strips it
from the address bar. Omitted on non-loopback binds to avoid token leak.
- `--webui`/`--admin` with no backend prints a one-line hint instead of the
full backend cheat-sheet (UI already covers it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds opt-in client-credential forwarding for Anthropic passthrough
(single-key/BYOK deployments), plus fixes found in review:
- Startup safeguard now shares one check (server/middleware/auth.rs)
with the live admin PUT /admin/api/config path, closing a bypass
where PROXY_OPEN_RELAY=true alongside 2+ PROXY_API_KEYS entries
slipped past the old startup-only check.
- x-goog-api-key is now recognized as a forwardable credential
(renamed to x-api-key upstream, since Anthropic doesn't understand
that header name), matching validate_auth's precedence.
- Managed (admin-API) backends no longer carry a dead
forward_client_auth field that could never take effect.
- ClientAuthPath forwarding decisions are now double-checked against
live VirtualKeyContext/JwtClaims presence, not just the enum, to
fail closed if the two ever desync.
- Moved from a per-backend BackendConfig field to a global
RuntimeConfig field (like anthropic_thinking_repair), making it
live-toggleable from the admin UI with no restart, and uniform
across every Anthropic-kind backend in a multi-backend deployment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Tool-call guardrails (lsp_first/quiet_command/write_payload_cap nudges,
fingerprint dedup) for local-LLM tool loops. Configurable via YAML
tool_execution.guardrails, FORGE_TOOL_CALL_POLICY env fallback, or the
admin UI (live, no restart).
- Anthropic thinking-block record/repair (ANTHROPIC_THINKING_REPAIR):
records ground truth off the real API and repairs client-corrupted
thinking/redacted_thinking blocks in replayed conversations.
- Bidirectional thinking_blocks (signature/redacted state) round-trip
through the OpenAI-compat wire format for LiteLLM-style clients.
- Review fixes: guardrail-mode divergence between streaming/non-streaming
paths, cross-backend/tenant cache-namespace collision, client-controlled
integer overflow in thinking budget_tokens, dropped reasoning_content and
citations on repair/translation paths, a fail-closed race under cache
eviction, plus dedup/simplification cleanup and doc corrections.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
opentelemetry_sdk <= 0.32.0 allows unbounded memory allocation while
parsing W3C Baggage headers (2 open Dependabot alerts). Bump
opentelemetry/opentelemetry_sdk/opentelemetry-otlp 0.31 -> 0.32 and
tracing-opentelemetry 0.32 -> 0.33 in lockstep behind the optional
otel feature; the old tracing-opentelemetry 0.32.x pinned
opentelemetry 0.31, which produced a duplicate-trait-impl compile
error until bumped together.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Regenerate litellm_snapshot.rs from upstream LiteLLM data: adds 7 new
providers (darkbloom, libertai, pinstripes, scaleway, tencent,
tensormesh, tinyfish) and fixes model/pricing drift across ~28
existing providers. Drop the now-redundant hand-maintained scaleway
legacy stub in favor of the generated entry. Refresh model_pricing.json
for the new claude-sonnet-5 entry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
useKeys treated the {keys:[...]} response as a bare array, so the Keys
tab threw at runtime (.find/.filter on a plain object). Unwrap .keys to
match the VirtualKey[] consumers expect (mirrors useBackends).
useRequests and useAudit sent page/page_size, but the backends only read
limit/offset, so every page returned the first page. Translate at the
hook boundary (limit=page_size, offset=(page-1)*page_size) and align the
RequestsResponse/AuditResponse types with the actual backend shape
(limit/offset/has_more; drop the never-sent total/page/page_size).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Key edit modal sent only 4 of the 8 PUT /admin/api/keys/{id} fields; the
endpoint clears omitted fields to NULL, so every save silently wiped the
enforced budget (max_budget_usd), budget_duration, tpm_limit, allowed_models,
and expires_at. The visible "Spend limit" input wrote the unused spend_limit
column (dropped by serde, never enforced). Now the spend input drives
max_budget_usd and all unsurfaced fields are resent from current values.
Also:
- BackendForm edit mode seeds existing values instead of starting blank,
which combined with the no-NULL-clear patch made saves no-op or look like
they wiped endpoint config.
- App handles config_changed websocket events (Settings uses staleTime:
Infinity, so config changes from another session/CLI never refreshed).
- Request-log and observability backend filters are populated from
useBackends() (were empty, dead <select>s); server already filters by name.
- Removed dead AddBackendForm onCreated no-op prop.
- Revoke flow: ConfirmDialog owns its close; parent closes reactively on
success, avoiding setState on an unmounted component.
Verified: tsc + eslint clean, vite build ok, cargo test 1254 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Main already scopes cache keys by backend + auth (CacheScope, _scope_backend/
_scope_auth) and includes 'system' in CACHE_FIELDS, with a stronger
vk_ctx-aware auth_identity than this branch. Recording as merged; no tree
change to avoid regressing the existing fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Main already enforces VK TPM + cost accounting in the gemini native path
via record_virtual_key_usage (helpers.rs). Kept main's version; the
branch's older handler structure would regress logging/attribution.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Break logs.rs, config/multi.rs, config/simple, chat_completions.rs,
gemini_input.rs, and admin/routes into focused submodules. No behavior
change; build is green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Large admin-ui refactor (Performative component system, sidebar nav,
provider/route tabs) plus backend module restructuring.
Admin UI contract fixes (this session):
- Fix Models page crash: useBackends unwraps {backends:[...]}; align
ModelEntry to {model_name, deployments} and ModelsResponse.strategy;
fix add-model body to {model_name, actual_model, backend_name}.
- Fix Backends/Providers health rendering: source per-backend status and
latency from the uptime endpoint (health_checks); narrow Backend type to
the real get_backends shape.
- Route + sidebar-link the previously-unrouted Backends tab.
Verified: cargo test (exit 0), clippy -D warnings (exit 0), fmt --check,
admin-ui tsc + vite build all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
GitHub's arm64 Docker builders intermittently hit "[16] Error in the
HTTP2 framing layer" during cargo-chef's sparse-index fetches, panicking
the cook step (recipe.rs:224) and failing the v0.9.9 arm64 release build.
Set CARGO_HTTP_MULTIPLEXING=false + CARGO_NET_RETRY=5 in the chef stage,
inherited by planner and builder. Multiplexing-off is the canonical
workaround for transient crates.io HTTP/2 framing errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Providers.tsx: wrap catalog in useMemo so filtered memo doesn't
re-run on every render when catalogQuery.data is undefined
- ci.yml: pin windows-latest -> windows-2025 ahead of June 15 redirect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
whit3rabbitandClaude Opus 4.8
dependabot[bot]