Refresh the LiteLLM provider/model catalog and pricing, and fix
--write-rust-snapshot so it no longer clobbers the hand-maintained
ANTHROPIC_ADAPTIVE_ONLY_THINKING_MODELS list. Running the generator
previously overwrote the whole file and silently dropped that constant,
breaking the build until it was hand-restored.
- Pricing: refresh assets/model_pricing.json + proxy copy from LiteLLM
(177 -> 169 entries; +claude-opus-5, gemini-3.5/3.6-flash,
gemini-omni-flash-preview; -14 gpt-*-realtime*; price changes on
gpt-5.6-luna, gpt-5.6-terra).
- Snapshot: regenerate litellm_snapshot.rs; claude-opus-5 (released
2026-07-24) is now in the data-driven ADAPTIVE_THINKING / MAX / XHIGH
lists from upstream, which matches its documented high/xhigh/max support.
- Generator (the fix): emit ANTHROPIC_ADAPTIVE_ONLY_THINKING_MODELS from a
new ANTHROPIC_ADAPTIVE_ONLY_THINKING Python constant in
check_litellm_providers.py, so litellm_snapshot.rs is 100% generated and
the constant survives every regeneration. Add claude-opus-5 and
claude-sonnet-5, which reject thinking.type:"enabled"/budget_tokens with
a 400 (adaptive-only) per Anthropic's "Configurations each model rejects"
table. Not derivable from LiteLLM flags (extended_thinking is true for
every thinking model), so it stays hand-curated in the script.
- Tests: bump anthropic catalog count 23 -> 24; assert opus-5 and sonnet-5
require adaptive thinking (reject budget_tokens).
- Docs: correct the stale "hand-maintained tail" gotcha in
crates/providers/CLAUDE.md; add Opus 5/Sonnet 5 to the reject-list note.
Verified: idempotent regen (two runs byte-identical), cargo fmt --check
clean, cargo clippy (default features, as CI runs) clean, cargo test
--workspace 1728 passed / 0 failed / 10 ignored (live-API).
Co-Authored-By: Claude <noreply@anthropic.com>
Claude Code's /model picker showed fake 'Claude Sonnet/Opus/Haiku' because /v1/models returned the static Anthropic catalog and the launch command did not enable discovery. With the Auto Router on, /v1/models now advertises the real backend models (autorouter tier targets + managed backend catalogs + model_list), and a model picked from the picker routes straight to its backend (explicit pick wins over tier signals). claude-* alias traffic still flows through the configured tiers.
- /v1/models: real models when router enabled; static Anthropic catalog fallback otherwise
- explicit-pick deferral in /v1/messages and /v1/chat/completions (AppState::resolve_explicit_pick)
- CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=true in the Auto Router copy-command and the README/CLI launch instructions
- tests: unit (push_model_row, RouterConfig::active_tiers) + integration (explicit_pick beats think tier)
Co-Authored-By: Claude <noreply@anthropic.com>
mlx-v is a Rust candle-backed vision-language inference toolkit whose
`vlm serve` exposes an OpenAI-compatible endpoint, defaulting to
localhost:8080. Registering it means the proxy can front a local VLM the same
way it fronts LM Studio or Ollama, which in practice means Anthropic
/v1/messages against a local candle model.
Capabilities differ from the neighbouring local backends in two ways worth
naming: embeddings is false (mlx-v has no /v1/embeddings route, unlike
lm_studio and ollama which advertise one), and tool_choice is false (mlx-v
honors "auto" and "none" but rejects "required" and named functions, having
no constrained decoding).
No litellm_snapshot change: mlx-v is not in LiteLLM, which is what the
LEGACY_ONLY_* arrays are for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bump workspace + all inter-crate version refs from 0.15.1 to 0.16.0
(workspace Cargo.toml, anyllm_client pinned version, batch_engine,
proxy's 8 internal deps, optimizer core/passes/scorer + cli + benches).
Refresh Cargo.lock. Move CHANGELOG [Unreleased] -> [0.16.0] - 2026-07-16
and add a fresh empty [Unreleased]; bump the README deb filename to
anyllm-proxy_0.16.0-1_amd64.deb and add the v0.16.0 compare link.
Co-Authored-By: Claude <noreply@anthropic.com>
- Auth default is now loopback-open (not reject-all). With no
PROXY_API_KEYS, no PROXY_OPEN_RELAY, no virtual keys and no OIDC,
loopback TCP peers are accepted and LAN/remote peers get 401. The
decision uses the real TCP peer (ConnectInfo via
into_make_service_with_connect_info), not the spoofable
X-Forwarded-For. effective_auth_mode() (keys/open_relay/loopback_only)
+ proxy_key_count are surfaced on GET /admin/api/status; the admin UI
shows a warning banner when no key is set.
- Add --port/-p CLI flag that sets LISTEN_PORT for the run. Stripped
before any run/providers subcommand so flags meant for the launched
tool survive; pure scan is unit-tested.
- Startup port handling: the run subcommand pre-checks the listen port
and fails fast with a hint when in use; wait_for_port readiness timeout
10s -> 30s; listener bind failures (proxy + admin) now print an
actionable message and exit(1) instead of panicking.
- POST /v1/chat/completions no longer 400s on a missing max_tokens for
OpenAI-compatible backends. The internal placeholder is stripped via a
new OMIT_MAX_TOKENS_MARKER so the backend applies its own default
(e.g. LM Studio's 8192); the marker never leaks upstream. Explicit
max_tokens is still forwarded verbatim. Anthropic backends unchanged.
- Tier router logs the selected tier at info (tier/backend/model)
instead of routing silently.
- Admin UI modal no longer dismisses when a text-selection drag starts
inside the card (dismiss only on a press that begins on the backdrop).
Co-Authored-By: Claude <noreply@anthropic.com>
- discover/refresh no longer wipe provider_models_cache on an empty result
(DELETE-then-INSERT upsert skipped when zero ids), preserving autorouter
model suggestions.
- Provider editor: removed model chips stick; persisted names seeded once
instead of re-unioned on every cache refetch.
- Managed-backend edit falls back to SQLite on an in-memory miss, so a row
skipped at startup no longer 404s on every save (heals value-fixable
configs; unknown provider_id still 400s).
- Add get_managed_backend DB helper + round-trip test.
- Bump workspace to 0.15.1; CHANGELOG + README deb filename.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin/browser:
- Enable admin UI on bare (zero-arg) launch; auto-open default browser
(main_helpers::bootstrap::admin_enabled / is_default_launch, browser.rs).
- Guard Docker (docker-entrypoint.sh) and systemd (packaging/anyllm-proxy.service)
so headless server installs keep admin opt-in (DISABLE_ADMIN default off).
Claude Code tier router fixes (from code review):
- put.rs: validate only *enabled* tiers, and accept statically-configured
(all_backends) targets via new SharedState.static_backends, not just managed.
- openai_signals: drop historical reasoning_content check so a plain follow-up
in a reasoning conversation isn't misrouted to the Think tier.
- resolve_router_tier: warn! on fail-open when an active tier's backend is
unknown instead of silently bypassing the router.
Tests: static-config backend acceptance; existing router coverage still green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Router (opt-in, disabled by default): classify each request by shape
(image / web_search / think / long_context / background / default) and route
it to a configured backend+model, bypassing model-name routing. Backend in
config/router_config.rs + server/router_signals.rs, wired through app_state,
the chat_completions and messages handlers, and the /admin/api/config
put/get/delete routes; covered by tests/router_config_admin.rs.
Admin UI: merge the former Router and Routes tabs into a single Routing tab
with two subtabs, Auto Router (route by request shape) and Model Routes
(named model aliases, load-balanced). Old #/router and #/routes URLs redirect
to the matching subtab.
Theme: move accent color and light/dark mode into a Settings > Display
section as independent axes (mode layers over any accent), default accent
Blue, migrating existing anyllm.theme=light values.
Also: settings copy tweak for thinking-repair, RouterConfig API types, and the
regenerated dist/index.html bundle.
Note: theme and router share Sidebar.tsx and globals.css at the hunk level, so
they land together rather than as separate commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Sparkline component and a useMetricHistory hook that buffers recent
metric snapshots, and render inline trend sparklines on the Dashboard tiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the provider editor into a larger single-column ProviderForm with an
API-key show/hide toggle, an informational Models list (add + query-models
discovery), and inline Edit of existing managed backends (not just
create/delete). Providers.tsx now delegates to ProviderForm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add per-feature build/test targets (all-features, otel, qdrant, redis,
optimizer) and UI targets to the Makefile help output. Document in CLAUDE.md
that admin-ui/dist/index.html is git-tracked and embedded via include_str!,
so it must be regenerated with `make ui-build` rather than restored from git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Split config.rs into config/{mod,delete,get,put}.rs
- Split routes_api.rs into routes_api/{mod,helpers,providers,routes}.rs
- Split passthrough/handlers.rs into handlers/{mod,errors,generic,messages}.rs
- Split streaming.rs into streaming/{mod,handler,helpers}.rs
- Split main_helpers/async_main/admin.rs into admin/{mod,config,tasks}.rs
- Minor cleanups in chat_completions backends, token_counting, tests
- Add docs/TEST_PARITY_LITELLM.md
Co-Authored-By: Claude <noreply@anthropic.com>
Re-cut of the 0.14.0 release. The v0.14.0 tag was pushed before the CI
publish job knew about the new leaf crates (pxpipe, rtk, optimize_*), so
its pipeline would have failed to publish anyllm_proxy to crates.io. The
publish-order fix landed on main after the tag, and a tag can't be moved
under auto mode, so the release is re-cut as 0.14.1 with the fix in place.
Bump 0.14.0 -> 0.14.1 (workspace, inter-crate deps, client pinned version,
README deb filename) and rename the CHANGELOG section (0.14.0 was never
released).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 0.14.0 proxy gained hard deps on anyllm_pxpipe, anyllm_rtk, and the
anyllm_optimize_* crates, none of which the publish job uploaded. Since
none exist on crates.io, cargo publish -p anyllm_proxy would fail on the
missing deps, and the exit-101 guard (cargo returns 101 for nearly all
errors) masked it as "already published" - proxy never landed while the
job stayed green.
Add publish steps for pxpipe, rtk, optimize_core/passes/scorer in
dependency order before anyllm_proxy, and document the invariant in
CLAUDE.md so future leaf crates get added here too.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move Unreleased changelog entries (RTK compression, opt-in prompt
optimizer, release binary archives, Bedrock allowlist fix, model
discovery fix) into the 0.14.0 section. Bump workspace + inter-crate
versions 0.13.0 -> 0.14.0 and the README deb filename.
Fix clippy missing_const_for_thread_local in optimize-core alloc_budget
test (const-init the thread-locals) so cargo clippy -D warnings is clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add [Unreleased] entries for RTK tool-output compression and the opt-in prompt
optimizer, and document their RTK_* / OPTIMIZER_* / MODEL_* env vars and config
keys in docs/ENV.md and docs/CONFIG.md. Also records the already-committed
Bedrock native-route model-allowlist fix and the model-discovery URL fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RTK (anyllm_rtk crate): command-aware filtering of tool-result text
(test/build/git/log output) via a catalog of 55 declarative filters ported
from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control
blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream +
non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS.
RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config.
Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive
Compression of long client-sent conversation history for OpenAI Chat
Completions, the Anthropic translate path, and the Anthropic passthrough path
(client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow|
live env / admin toggle; live places a cache_control breakpoint at the frontier
over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx
feature (model fetched on demand, never bundled). New optimizer_* metrics
counters and GET/POST /admin/api/optimizer/model endpoints.
Both features expose runtime config + Settings UI controls and fail open on any
error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and
gitignore rules for downloaded ONNX artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bedrock native routes (converse/converse-stream/invoke/invoke-with-response-
stream) now enforce the virtual key's model allowlist, closing a model-scope
bypass where a model-scoped key could invoke any Bedrock modelId.
Admin UI "Query models" discovery: stop doubling /v1 into /v1/v1/models for
local providers whose catalog base URL already ends in /v1, trim trailing
slashes off api_base, support Anthropic-native providers (x-api-key +
anthropic-version auth, display_name model field), and show/warn about the
discovery target URL in the Add-Backend form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
build-release already compiles x86_64/aarch64 Linux and x86_64 Windows binaries
but only uploaded them as CI artifacts. The release-assets job now packages them
into anyllm-proxy-<ver>-linux-{x86_64,arm64}.tar.gz and
anyllm-proxy-<ver>-windows-x86_64.zip and uploads them to the release, matching
the macOS tarball naming.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The [[bin]] name became anyllm-proxy (hyphen) in 0.12.0, but the Dockerfile
runtime COPY and docker-entrypoint.sh still referenced anyllm_proxy
(underscore), breaking the Docker image build ("/app/target/release/anyllm_proxy:
not found"). Both now use the hyphenated name.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump workspace + inter-crate versions 0.12.0 -> 0.13.0, README deb filename,
and move CHANGELOG [Unreleased] into [0.13.0] - 2026-07-12.
Also bump crossbeam-epoch 0.9.18 -> 0.9.20 (Cargo.lock) to clear
RUSTSEC-2026-0204 (invalid pointer dereference), which was failing the
CI Security audit step.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI UX fixes plus the route-dispatch feature set:
- Providers: selecting a provider shows the full key/options form immediately
instead of hiding it behind a "+ Add key" button; "Cancel" becomes "Reset".
- Routes: enabled routes show a ready-to-run curl snippet (endpoint URL from the
new proxy_port on /admin/api/status, route name as the model) with a copy button.
- Settings: live proxy status badge (running/unreachable) backed by a proxy_running
TCP liveness check on /admin/api/status, plus a per-save "applied live" toast.
- Backend: plumb proxy listen_port into SharedState; expose proxy_port + proxy_running.
- Also includes the RouteRouter dispatch layer, per-route option overrides, local-LLM
backend support, favorites, and model discovery (see CHANGELOG [Unreleased]).
Ignore .gate/ security-scanner scratch artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI: shared centered provider modal, dropdown backend selectors for Add
Model/Backend, inline add-failure errors, copy/dismiss on created keys, metric
tooltips, empty states. status route now counts managed backends. Add InfoTip
and clipboard helpers. Ignore .eatahorse*/ scratch boards and drop the tracked
guardrails board.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Emit binary as `anyllm-proxy` (hyphen) via explicit [[bin]]; package/crate
name stays `anyllm_proxy`. Threads one hyphenated name through deb/brew/
release archives and CI; drops the rename-in-archive shim.
- Terminal logs default to human-readable on a TTY, JSON when piped
(Docker/systemd). Override with LOG_FORMAT=json|text.
- On loopback admin binds, startup banner prints a tokenized `?token=` admin
URL plus a bare Token line; SPA reads `?token=` to auto-login then strips it
from the address bar. Omitted on non-loopback binds to avoid token leak.
- `--webui`/`--admin` with no backend prints a one-line hint instead of the
full backend cheat-sheet (UI already covers it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds opt-in client-credential forwarding for Anthropic passthrough
(single-key/BYOK deployments), plus fixes found in review:
- Startup safeguard now shares one check (server/middleware/auth.rs)
with the live admin PUT /admin/api/config path, closing a bypass
where PROXY_OPEN_RELAY=true alongside 2+ PROXY_API_KEYS entries
slipped past the old startup-only check.
- x-goog-api-key is now recognized as a forwardable credential
(renamed to x-api-key upstream, since Anthropic doesn't understand
that header name), matching validate_auth's precedence.
- Managed (admin-API) backends no longer carry a dead
forward_client_auth field that could never take effect.
- ClientAuthPath forwarding decisions are now double-checked against
live VirtualKeyContext/JwtClaims presence, not just the enum, to
fail closed if the two ever desync.
- Moved from a per-backend BackendConfig field to a global
RuntimeConfig field (like anthropic_thinking_repair), making it
live-toggleable from the admin UI with no restart, and uniform
across every Anthropic-kind backend in a multi-backend deployment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Tool-call guardrails (lsp_first/quiet_command/write_payload_cap nudges,
fingerprint dedup) for local-LLM tool loops. Configurable via YAML
tool_execution.guardrails, FORGE_TOOL_CALL_POLICY env fallback, or the
admin UI (live, no restart).
- Anthropic thinking-block record/repair (ANTHROPIC_THINKING_REPAIR):
records ground truth off the real API and repairs client-corrupted
thinking/redacted_thinking blocks in replayed conversations.
- Bidirectional thinking_blocks (signature/redacted state) round-trip
through the OpenAI-compat wire format for LiteLLM-style clients.
- Review fixes: guardrail-mode divergence between streaming/non-streaming
paths, cross-backend/tenant cache-namespace collision, client-controlled
integer overflow in thinking budget_tokens, dropped reasoning_content and
citations on repair/translation paths, a fail-closed race under cache
eviction, plus dedup/simplification cleanup and doc corrections.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
opentelemetry_sdk <= 0.32.0 allows unbounded memory allocation while
parsing W3C Baggage headers (2 open Dependabot alerts). Bump
opentelemetry/opentelemetry_sdk/opentelemetry-otlp 0.31 -> 0.32 and
tracing-opentelemetry 0.32 -> 0.33 in lockstep behind the optional
otel feature; the old tracing-opentelemetry 0.32.x pinned
opentelemetry 0.31, which produced a duplicate-trait-impl compile
error until bumped together.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Regenerate litellm_snapshot.rs from upstream LiteLLM data: adds 7 new
providers (darkbloom, libertai, pinstripes, scaleway, tencent,
tensormesh, tinyfish) and fixes model/pricing drift across ~28
existing providers. Drop the now-redundant hand-maintained scaleway
legacy stub in favor of the generated entry. Refresh model_pricing.json
for the new claude-sonnet-5 entry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
useKeys treated the {keys:[...]} response as a bare array, so the Keys
tab threw at runtime (.find/.filter on a plain object). Unwrap .keys to
match the VirtualKey[] consumers expect (mirrors useBackends).
useRequests and useAudit sent page/page_size, but the backends only read
limit/offset, so every page returned the first page. Translate at the
hook boundary (limit=page_size, offset=(page-1)*page_size) and align the
RequestsResponse/AuditResponse types with the actual backend shape
(limit/offset/has_more; drop the never-sent total/page/page_size).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions[bot]
whit3rabbit
dependabot[bot]
