Claude Code's /model picker showed fake 'Claude Sonnet/Opus/Haiku' because /v1/models returned the static Anthropic catalog and the launch command did not enable discovery. With the Auto Router on, /v1/models now advertises the real backend models (autorouter tier targets + managed backend catalogs + model_list), and a model picked from the picker routes straight to its backend (explicit pick wins over tier signals). claude-* alias traffic still flows through the configured tiers.
- /v1/models: real models when router enabled; static Anthropic catalog fallback otherwise
- explicit-pick deferral in /v1/messages and /v1/chat/completions (AppState::resolve_explicit_pick)
- CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=true in the Auto Router copy-command and the README/CLI launch instructions
- tests: unit (push_model_row, RouterConfig::active_tiers) + integration (explicit_pick beats think tier)
Co-Authored-By: Claude <noreply@anthropic.com>
- Auth default is now loopback-open (not reject-all). With no
PROXY_API_KEYS, no PROXY_OPEN_RELAY, no virtual keys and no OIDC,
loopback TCP peers are accepted and LAN/remote peers get 401. The
decision uses the real TCP peer (ConnectInfo via
into_make_service_with_connect_info), not the spoofable
X-Forwarded-For. effective_auth_mode() (keys/open_relay/loopback_only)
+ proxy_key_count are surfaced on GET /admin/api/status; the admin UI
shows a warning banner when no key is set.
- Add --port/-p CLI flag that sets LISTEN_PORT for the run. Stripped
before any run/providers subcommand so flags meant for the launched
tool survive; pure scan is unit-tested.
- Startup port handling: the run subcommand pre-checks the listen port
and fails fast with a hint when in use; wait_for_port readiness timeout
10s -> 30s; listener bind failures (proxy + admin) now print an
actionable message and exit(1) instead of panicking.
- POST /v1/chat/completions no longer 400s on a missing max_tokens for
OpenAI-compatible backends. The internal placeholder is stripped via a
new OMIT_MAX_TOKENS_MARKER so the backend applies its own default
(e.g. LM Studio's 8192); the marker never leaks upstream. Explicit
max_tokens is still forwarded verbatim. Anthropic backends unchanged.
- Tier router logs the selected tier at info (tier/backend/model)
instead of routing silently.
- Admin UI modal no longer dismisses when a text-selection drag starts
inside the card (dismiss only on a press that begins on the backdrop).
Co-Authored-By: Claude <noreply@anthropic.com>
- discover/refresh no longer wipe provider_models_cache on an empty result
(DELETE-then-INSERT upsert skipped when zero ids), preserving autorouter
model suggestions.
- Provider editor: removed model chips stick; persisted names seeded once
instead of re-unioned on every cache refetch.
- Managed-backend edit falls back to SQLite on an in-memory miss, so a row
skipped at startup no longer 404s on every save (heals value-fixable
configs; unknown provider_id still 400s).
- Add get_managed_backend DB helper + round-trip test.
- Bump workspace to 0.15.1; CHANGELOG + README deb filename.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Router (opt-in, disabled by default): classify each request by shape
(image / web_search / think / long_context / background / default) and route
it to a configured backend+model, bypassing model-name routing. Backend in
config/router_config.rs + server/router_signals.rs, wired through app_state,
the chat_completions and messages handlers, and the /admin/api/config
put/get/delete routes; covered by tests/router_config_admin.rs.
Admin UI: merge the former Router and Routes tabs into a single Routing tab
with two subtabs, Auto Router (route by request shape) and Model Routes
(named model aliases, load-balanced). Old #/router and #/routes URLs redirect
to the matching subtab.
Theme: move accent color and light/dark mode into a Settings > Display
section as independent axes (mode layers over any accent), default accent
Blue, migrating existing anyllm.theme=light values.
Also: settings copy tweak for thinking-repair, RouterConfig API types, and the
regenerated dist/index.html bundle.
Note: theme and router share Sidebar.tsx and globals.css at the hunk level, so
they land together rather than as separate commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Sparkline component and a useMetricHistory hook that buffers recent
metric snapshots, and render inline trend sparklines on the Dashboard tiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the provider editor into a larger single-column ProviderForm with an
API-key show/hide toggle, an informational Models list (add + query-models
discovery), and inline Edit of existing managed backends (not just
create/delete). Providers.tsx now delegates to ProviderForm.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RTK (anyllm_rtk crate): command-aware filtering of tool-result text
(test/build/git/log output) via a catalog of 55 declarative filters ported
from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control
blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream +
non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS.
RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config.
Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive
Compression of long client-sent conversation history for OpenAI Chat
Completions, the Anthropic translate path, and the Anthropic passthrough path
(client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow|
live env / admin toggle; live places a cache_control breakpoint at the frontier
over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx
feature (model fetched on demand, never bundled). New optimizer_* metrics
counters and GET/POST /admin/api/optimizer/model endpoints.
Both features expose runtime config + Settings UI controls and fail open on any
error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and
gitignore rules for downloaded ONNX artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bedrock native routes (converse/converse-stream/invoke/invoke-with-response-
stream) now enforce the virtual key's model allowlist, closing a model-scope
bypass where a model-scoped key could invoke any Bedrock modelId.
Admin UI "Query models" discovery: stop doubling /v1 into /v1/v1/models for
local providers whose catalog base URL already ends in /v1, trim trailing
slashes off api_base, support Anthropic-native providers (x-api-key +
anthropic-version auth, display_name model field), and show/warn about the
discovery target URL in the Add-Backend form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI UX fixes plus the route-dispatch feature set:
- Providers: selecting a provider shows the full key/options form immediately
instead of hiding it behind a "+ Add key" button; "Cancel" becomes "Reset".
- Routes: enabled routes show a ready-to-run curl snippet (endpoint URL from the
new proxy_port on /admin/api/status, route name as the model) with a copy button.
- Settings: live proxy status badge (running/unreachable) backed by a proxy_running
TCP liveness check on /admin/api/status, plus a per-save "applied live" toast.
- Backend: plumb proxy listen_port into SharedState; expose proxy_port + proxy_running.
- Also includes the RouteRouter dispatch layer, per-route option overrides, local-LLM
backend support, favorites, and model discovery (see CHANGELOG [Unreleased]).
Ignore .gate/ security-scanner scratch artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI: shared centered provider modal, dropdown backend selectors for Add
Model/Backend, inline add-failure errors, copy/dismiss on created keys, metric
tooltips, empty states. status route now counts managed backends. Add InfoTip
and clipboard helpers. Ignore .eatahorse*/ scratch boards and drop the tracked
guardrails board.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Emit binary as `anyllm-proxy` (hyphen) via explicit [[bin]]; package/crate
name stays `anyllm_proxy`. Threads one hyphenated name through deb/brew/
release archives and CI; drops the rename-in-archive shim.
- Terminal logs default to human-readable on a TTY, JSON when piped
(Docker/systemd). Override with LOG_FORMAT=json|text.
- On loopback admin binds, startup banner prints a tokenized `?token=` admin
URL plus a bare Token line; SPA reads `?token=` to auto-login then strips it
from the address bar. Omitted on non-loopback binds to avoid token leak.
- `--webui`/`--admin` with no backend prints a one-line hint instead of the
full backend cheat-sheet (UI already covers it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds opt-in client-credential forwarding for Anthropic passthrough
(single-key/BYOK deployments), plus fixes found in review:
- Startup safeguard now shares one check (server/middleware/auth.rs)
with the live admin PUT /admin/api/config path, closing a bypass
where PROXY_OPEN_RELAY=true alongside 2+ PROXY_API_KEYS entries
slipped past the old startup-only check.
- x-goog-api-key is now recognized as a forwardable credential
(renamed to x-api-key upstream, since Anthropic doesn't understand
that header name), matching validate_auth's precedence.
- Managed (admin-API) backends no longer carry a dead
forward_client_auth field that could never take effect.
- ClientAuthPath forwarding decisions are now double-checked against
live VirtualKeyContext/JwtClaims presence, not just the enum, to
fail closed if the two ever desync.
- Moved from a per-backend BackendConfig field to a global
RuntimeConfig field (like anthropic_thinking_repair), making it
live-toggleable from the admin UI with no restart, and uniform
across every Anthropic-kind backend in a multi-backend deployment.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Tool-call guardrails (lsp_first/quiet_command/write_payload_cap nudges,
fingerprint dedup) for local-LLM tool loops. Configurable via YAML
tool_execution.guardrails, FORGE_TOOL_CALL_POLICY env fallback, or the
admin UI (live, no restart).
- Anthropic thinking-block record/repair (ANTHROPIC_THINKING_REPAIR):
records ground truth off the real API and repairs client-corrupted
thinking/redacted_thinking blocks in replayed conversations.
- Bidirectional thinking_blocks (signature/redacted state) round-trip
through the OpenAI-compat wire format for LiteLLM-style clients.
- Review fixes: guardrail-mode divergence between streaming/non-streaming
paths, cross-backend/tenant cache-namespace collision, client-controlled
integer overflow in thinking budget_tokens, dropped reasoning_content and
citations on repair/translation paths, a fail-closed race under cache
eviction, plus dedup/simplification cleanup and doc corrections.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
useKeys treated the {keys:[...]} response as a bare array, so the Keys
tab threw at runtime (.find/.filter on a plain object). Unwrap .keys to
match the VirtualKey[] consumers expect (mirrors useBackends).
useRequests and useAudit sent page/page_size, but the backends only read
limit/offset, so every page returned the first page. Translate at the
hook boundary (limit=page_size, offset=(page-1)*page_size) and align the
RequestsResponse/AuditResponse types with the actual backend shape
(limit/offset/has_more; drop the never-sent total/page/page_size).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Key edit modal sent only 4 of the 8 PUT /admin/api/keys/{id} fields; the
endpoint clears omitted fields to NULL, so every save silently wiped the
enforced budget (max_budget_usd), budget_duration, tpm_limit, allowed_models,
and expires_at. The visible "Spend limit" input wrote the unused spend_limit
column (dropped by serde, never enforced). Now the spend input drives
max_budget_usd and all unsurfaced fields are resent from current values.
Also:
- BackendForm edit mode seeds existing values instead of starting blank,
which combined with the no-NULL-clear patch made saves no-op or look like
they wiped endpoint config.
- App handles config_changed websocket events (Settings uses staleTime:
Infinity, so config changes from another session/CLI never refreshed).
- Request-log and observability backend filters are populated from
useBackends() (were empty, dead <select>s); server already filters by name.
- Removed dead AddBackendForm onCreated no-op prop.
- Revoke flow: ConfirmDialog owns its close; parent closes reactively on
success, avoiding setState on an unmounted component.
Verified: tsc + eslint clean, vite build ok, cargo test 1254 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Large admin-ui refactor (Performative component system, sidebar nav,
provider/route tabs) plus backend module restructuring.
Admin UI contract fixes (this session):
- Fix Models page crash: useBackends unwraps {backends:[...]}; align
ModelEntry to {model_name, deployments} and ModelsResponse.strategy;
fix add-model body to {model_name, actual_model, backend_name}.
- Fix Backends/Providers health rendering: source per-backend status and
latency from the uptime endpoint (health_checks); narrow Backend type to
the real get_backends shape.
- Route + sidebar-link the previously-unrouted Backends tab.
Verified: cargo test (exit 0), clippy -D warnings (exit 0), fmt --check,
admin-ui tsc + vite build all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Providers.tsx: wrap catalog in useMemo so filtered memo doesn't
re-run on every render when catalogQuery.data is undefined
- ci.yml: pin windows-latest -> windows-2025 ahead of June 15 redirect
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace top nav with sidebar layout; add shared Modal, Toast, ConfirmDialog, AsyncBoundary components
- Add Routes tab with full CRUD UI backed by new routes_api admin endpoints
- Split admin rate limiter into separate read/write buckets with method-aware middleware
- Add allowed_routes scoping to virtual keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
whit3rabbit
dependabot[bot]
