Refresh the LiteLLM provider/model catalog and pricing, and fix
--write-rust-snapshot so it no longer clobbers the hand-maintained
ANTHROPIC_ADAPTIVE_ONLY_THINKING_MODELS list. Running the generator
previously overwrote the whole file and silently dropped that constant,
breaking the build until it was hand-restored.
- Pricing: refresh assets/model_pricing.json + proxy copy from LiteLLM
(177 -> 169 entries; +claude-opus-5, gemini-3.5/3.6-flash,
gemini-omni-flash-preview; -14 gpt-*-realtime*; price changes on
gpt-5.6-luna, gpt-5.6-terra).
- Snapshot: regenerate litellm_snapshot.rs; claude-opus-5 (released
2026-07-24) is now in the data-driven ADAPTIVE_THINKING / MAX / XHIGH
lists from upstream, which matches its documented high/xhigh/max support.
- Generator (the fix): emit ANTHROPIC_ADAPTIVE_ONLY_THINKING_MODELS from a
new ANTHROPIC_ADAPTIVE_ONLY_THINKING Python constant in
check_litellm_providers.py, so litellm_snapshot.rs is 100% generated and
the constant survives every regeneration. Add claude-opus-5 and
claude-sonnet-5, which reject thinking.type:"enabled"/budget_tokens with
a 400 (adaptive-only) per Anthropic's "Configurations each model rejects"
table. Not derivable from LiteLLM flags (extended_thinking is true for
every thinking model), so it stays hand-curated in the script.
- Tests: bump anthropic catalog count 23 -> 24; assert opus-5 and sonnet-5
require adaptive thinking (reject budget_tokens).
- Docs: correct the stale "hand-maintained tail" gotcha in
crates/providers/CLAUDE.md; add Opus 5/Sonnet 5 to the reject-list note.
Verified: idempotent regen (two runs byte-identical), cargo fmt --check
clean, cargo clippy (default features, as CI runs) clean, cargo test
--workspace 1728 passed / 0 failed / 10 ignored (live-API).
Co-Authored-By: Claude <noreply@anthropic.com>
Claude Code's /model picker showed fake 'Claude Sonnet/Opus/Haiku' because /v1/models returned the static Anthropic catalog and the launch command did not enable discovery. With the Auto Router on, /v1/models now advertises the real backend models (autorouter tier targets + managed backend catalogs + model_list), and a model picked from the picker routes straight to its backend (explicit pick wins over tier signals). claude-* alias traffic still flows through the configured tiers.
- /v1/models: real models when router enabled; static Anthropic catalog fallback otherwise
- explicit-pick deferral in /v1/messages and /v1/chat/completions (AppState::resolve_explicit_pick)
- CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=true in the Auto Router copy-command and the README/CLI launch instructions
- tests: unit (push_model_row, RouterConfig::active_tiers) + integration (explicit_pick beats think tier)
Co-Authored-By: Claude <noreply@anthropic.com>
mlx-v is a Rust candle-backed vision-language inference toolkit whose
`vlm serve` exposes an OpenAI-compatible endpoint, defaulting to
localhost:8080. Registering it means the proxy can front a local VLM the same
way it fronts LM Studio or Ollama, which in practice means Anthropic
/v1/messages against a local candle model.
Capabilities differ from the neighbouring local backends in two ways worth
naming: embeddings is false (mlx-v has no /v1/embeddings route, unlike
lm_studio and ollama which advertise one), and tool_choice is false (mlx-v
honors "auto" and "none" but rejects "required" and named functions, having
no constrained decoding).
No litellm_snapshot change: mlx-v is not in LiteLLM, which is what the
LEGACY_ONLY_* arrays are for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bump workspace + all inter-crate version refs from 0.15.1 to 0.16.0
(workspace Cargo.toml, anyllm_client pinned version, batch_engine,
proxy's 8 internal deps, optimizer core/passes/scorer + cli + benches).
Refresh Cargo.lock. Move CHANGELOG [Unreleased] -> [0.16.0] - 2026-07-16
and add a fresh empty [Unreleased]; bump the README deb filename to
anyllm-proxy_0.16.0-1_amd64.deb and add the v0.16.0 compare link.
Co-Authored-By: Claude <noreply@anthropic.com>
- Auth default is now loopback-open (not reject-all). With no
PROXY_API_KEYS, no PROXY_OPEN_RELAY, no virtual keys and no OIDC,
loopback TCP peers are accepted and LAN/remote peers get 401. The
decision uses the real TCP peer (ConnectInfo via
into_make_service_with_connect_info), not the spoofable
X-Forwarded-For. effective_auth_mode() (keys/open_relay/loopback_only)
+ proxy_key_count are surfaced on GET /admin/api/status; the admin UI
shows a warning banner when no key is set.
- Add --port/-p CLI flag that sets LISTEN_PORT for the run. Stripped
before any run/providers subcommand so flags meant for the launched
tool survive; pure scan is unit-tested.
- Startup port handling: the run subcommand pre-checks the listen port
and fails fast with a hint when in use; wait_for_port readiness timeout
10s -> 30s; listener bind failures (proxy + admin) now print an
actionable message and exit(1) instead of panicking.
- POST /v1/chat/completions no longer 400s on a missing max_tokens for
OpenAI-compatible backends. The internal placeholder is stripped via a
new OMIT_MAX_TOKENS_MARKER so the backend applies its own default
(e.g. LM Studio's 8192); the marker never leaks upstream. Explicit
max_tokens is still forwarded verbatim. Anthropic backends unchanged.
- Tier router logs the selected tier at info (tier/backend/model)
instead of routing silently.
- Admin UI modal no longer dismisses when a text-selection drag starts
inside the card (dismiss only on a press that begins on the backdrop).
Co-Authored-By: Claude <noreply@anthropic.com>
- discover/refresh no longer wipe provider_models_cache on an empty result
(DELETE-then-INSERT upsert skipped when zero ids), preserving autorouter
model suggestions.
- Provider editor: removed model chips stick; persisted names seeded once
instead of re-unioned on every cache refetch.
- Managed-backend edit falls back to SQLite on an in-memory miss, so a row
skipped at startup no longer 404s on every save (heals value-fixable
configs; unknown provider_id still 400s).
- Add get_managed_backend DB helper + round-trip test.
- Bump workspace to 0.15.1; CHANGELOG + README deb filename.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin/browser:
- Enable admin UI on bare (zero-arg) launch; auto-open default browser
(main_helpers::bootstrap::admin_enabled / is_default_launch, browser.rs).
- Guard Docker (docker-entrypoint.sh) and systemd (packaging/anyllm-proxy.service)
so headless server installs keep admin opt-in (DISABLE_ADMIN default off).
Claude Code tier router fixes (from code review):
- put.rs: validate only *enabled* tiers, and accept statically-configured
(all_backends) targets via new SharedState.static_backends, not just managed.
- openai_signals: drop historical reasoning_content check so a plain follow-up
in a reasoning conversation isn't misrouted to the Think tier.
- resolve_router_tier: warn! on fail-open when an active tier's backend is
unknown instead of silently bypassing the router.
Tests: static-config backend acceptance; existing router coverage still green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Router (opt-in, disabled by default): classify each request by shape
(image / web_search / think / long_context / background / default) and route
it to a configured backend+model, bypassing model-name routing. Backend in
config/router_config.rs + server/router_signals.rs, wired through app_state,
the chat_completions and messages handlers, and the /admin/api/config
put/get/delete routes; covered by tests/router_config_admin.rs.
Admin UI: merge the former Router and Routes tabs into a single Routing tab
with two subtabs, Auto Router (route by request shape) and Model Routes
(named model aliases, load-balanced). Old #/router and #/routes URLs redirect
to the matching subtab.
Theme: move accent color and light/dark mode into a Settings > Display
section as independent axes (mode layers over any accent), default accent
Blue, migrating existing anyllm.theme=light values.
Also: settings copy tweak for thinking-repair, RouterConfig API types, and the
regenerated dist/index.html bundle.
Note: theme and router share Sidebar.tsx and globals.css at the hunk level, so
they land together rather than as separate commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Sparkline component and a useMetricHistory hook that buffers recent
metric snapshots, and render inline trend sparklines on the Dashboard tiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the provider editor into a larger single-column ProviderForm with an
API-key show/hide toggle, an informational Models list (add + query-models
discovery), and inline Edit of existing managed backends (not just
create/delete). Providers.tsx now delegates to ProviderForm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add per-feature build/test targets (all-features, otel, qdrant, redis,
optimizer) and UI targets to the Makefile help output. Document in CLAUDE.md
that admin-ui/dist/index.html is git-tracked and embedded via include_str!,
so it must be regenerated with `make ui-build` rather than restored from git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Split config.rs into config/{mod,delete,get,put}.rs
- Split routes_api.rs into routes_api/{mod,helpers,providers,routes}.rs
- Split passthrough/handlers.rs into handlers/{mod,errors,generic,messages}.rs
- Split streaming.rs into streaming/{mod,handler,helpers}.rs
- Split main_helpers/async_main/admin.rs into admin/{mod,config,tasks}.rs
- Minor cleanups in chat_completions backends, token_counting, tests
- Add docs/TEST_PARITY_LITELLM.md
Co-Authored-By: Claude <noreply@anthropic.com>
Re-cut of the 0.14.0 release. The v0.14.0 tag was pushed before the CI
publish job knew about the new leaf crates (pxpipe, rtk, optimize_*), so
its pipeline would have failed to publish anyllm_proxy to crates.io. The
publish-order fix landed on main after the tag, and a tag can't be moved
under auto mode, so the release is re-cut as 0.14.1 with the fix in place.
Bump 0.14.0 -> 0.14.1 (workspace, inter-crate deps, client pinned version,
README deb filename) and rename the CHANGELOG section (0.14.0 was never
released).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
whit3rabbit
github-actions[bot]
dependabot[bot]
