Claude Code's /model picker showed fake 'Claude Sonnet/Opus/Haiku' because /v1/models returned the static Anthropic catalog and the launch command did not enable discovery. With the Auto Router on, /v1/models now advertises the real backend models (autorouter tier targets + managed backend catalogs + model_list), and a model picked from the picker routes straight to its backend (explicit pick wins over tier signals). claude-* alias traffic still flows through the configured tiers.
- /v1/models: real models when router enabled; static Anthropic catalog fallback otherwise
- explicit-pick deferral in /v1/messages and /v1/chat/completions (AppState::resolve_explicit_pick)
- CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY=true in the Auto Router copy-command and the README/CLI launch instructions
- tests: unit (push_model_row, RouterConfig::active_tiers) + integration (explicit_pick beats think tier)
Co-Authored-By: Claude <noreply@anthropic.com>
- Auth default is now loopback-open (not reject-all). With no
PROXY_API_KEYS, no PROXY_OPEN_RELAY, no virtual keys and no OIDC,
loopback TCP peers are accepted and LAN/remote peers get 401. The
decision uses the real TCP peer (ConnectInfo via
into_make_service_with_connect_info), not the spoofable
X-Forwarded-For. effective_auth_mode() (keys/open_relay/loopback_only)
+ proxy_key_count are surfaced on GET /admin/api/status; the admin UI
shows a warning banner when no key is set.
- Add --port/-p CLI flag that sets LISTEN_PORT for the run. Stripped
before any run/providers subcommand so flags meant for the launched
tool survive; pure scan is unit-tested.
- Startup port handling: the run subcommand pre-checks the listen port
and fails fast with a hint when in use; wait_for_port readiness timeout
10s -> 30s; listener bind failures (proxy + admin) now print an
actionable message and exit(1) instead of panicking.
- POST /v1/chat/completions no longer 400s on a missing max_tokens for
OpenAI-compatible backends. The internal placeholder is stripped via a
new OMIT_MAX_TOKENS_MARKER so the backend applies its own default
(e.g. LM Studio's 8192); the marker never leaks upstream. Explicit
max_tokens is still forwarded verbatim. Anthropic backends unchanged.
- Tier router logs the selected tier at info (tier/backend/model)
instead of routing silently.
- Admin UI modal no longer dismisses when a text-selection drag starts
inside the card (dismiss only on a press that begins on the backdrop).
Co-Authored-By: Claude <noreply@anthropic.com>
- discover/refresh no longer wipe provider_models_cache on an empty result
(DELETE-then-INSERT upsert skipped when zero ids), preserving autorouter
model suggestions.
- Provider editor: removed model chips stick; persisted names seeded once
instead of re-unioned on every cache refetch.
- Managed-backend edit falls back to SQLite on an in-memory miss, so a row
skipped at startup no longer 404s on every save (heals value-fixable
configs; unknown provider_id still 400s).
- Add get_managed_backend DB helper + round-trip test.
- Bump workspace to 0.15.1; CHANGELOG + README deb filename.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Router (opt-in, disabled by default): classify each request by shape
(image / web_search / think / long_context / background / default) and route
it to a configured backend+model, bypassing model-name routing. Backend in
config/router_config.rs + server/router_signals.rs, wired through app_state,
the chat_completions and messages handlers, and the /admin/api/config
put/get/delete routes; covered by tests/router_config_admin.rs.
Admin UI: merge the former Router and Routes tabs into a single Routing tab
with two subtabs, Auto Router (route by request shape) and Model Routes
(named model aliases, load-balanced). Old #/router and #/routes URLs redirect
to the matching subtab.
Theme: move accent color and light/dark mode into a Settings > Display
section as independent axes (mode layers over any accent), default accent
Blue, migrating existing anyllm.theme=light values.
Also: settings copy tweak for thinking-repair, RouterConfig API types, and the
regenerated dist/index.html bundle.
Note: theme and router share Sidebar.tsx and globals.css at the hunk level, so
they land together rather than as separate commits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
RTK (anyllm_rtk crate): command-aware filtering of tool-result text
(test/build/git/log output) via a catalog of 55 declarative filters ported
from OmniRoute (MIT). IO-free, deterministic, prompt-cache safe (cache_control
blocks preserved byte-for-byte). Wired into the Anthropic passthrough (stream +
non-stream) and OpenAI-translate paths, gated per-model via RTK_MODELS.
RTK_COMPRESS env / admin toggle, rtk_compress/rtk_models runtime config.
Optimizer (anyllm_optimize_* crates): opt-in Frozen-Frontier Extractive
Compression of long client-sent conversation history for OpenAI Chat
Completions, the Anthropic translate path, and the Anthropic passthrough path
(client history only, never proxy tool-loop turns). OPTIMIZER_MODE=off|shadow|
live env / admin toggle; live places a cache_control breakpoint at the frontier
over raw bytes. Optional LLMLingua-2 ONNX scorer behind the optimizer-onnx
feature (model fetched on demand, never bundled). New optimizer_* metrics
counters and GET/POST /admin/api/optimizer/model endpoints.
Both features expose runtime config + Settings UI controls and fail open on any
error. Adds workspace members, CI lint/test for the optimizer-onnx feature, and
gitignore rules for downloaded ONNX artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bedrock native routes (converse/converse-stream/invoke/invoke-with-response-
stream) now enforce the virtual key's model allowlist, closing a model-scope
bypass where a model-scoped key could invoke any Bedrock modelId.
Admin UI "Query models" discovery: stop doubling /v1 into /v1/v1/models for
local providers whose catalog base URL already ends in /v1, trim trailing
slashes off api_base, support Anthropic-native providers (x-api-key +
anthropic-version auth, display_name model field), and show/warn about the
discovery target URL in the Add-Backend form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI UX fixes plus the route-dispatch feature set:
- Providers: selecting a provider shows the full key/options form immediately
instead of hiding it behind a "+ Add key" button; "Cancel" becomes "Reset".
- Routes: enabled routes show a ready-to-run curl snippet (endpoint URL from the
new proxy_port on /admin/api/status, route name as the model) with a copy button.
- Settings: live proxy status badge (running/unreachable) backed by a proxy_running
TCP liveness check on /admin/api/status, plus a per-save "applied live" toast.
- Backend: plumb proxy listen_port into SharedState; expose proxy_port + proxy_running.
- Also includes the RouteRouter dispatch layer, per-route option overrides, local-LLM
backend support, favorites, and model discovery (see CHANGELOG [Unreleased]).
Ignore .gate/ security-scanner scratch artifacts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Admin UI: shared centered provider modal, dropdown backend selectors for Add
Model/Backend, inline add-failure errors, copy/dismiss on created keys, metric
tooltips, empty states. status route now counts managed backends. Add InfoTip
and clipboard helpers. Ignore .eatahorse*/ scratch boards and drop the tracked
guardrails board.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Emit binary as `anyllm-proxy` (hyphen) via explicit [[bin]]; package/crate
name stays `anyllm_proxy`. Threads one hyphenated name through deb/brew/
release archives and CI; drops the rename-in-archive shim.
- Terminal logs default to human-readable on a TTY, JSON when piped
(Docker/systemd). Override with LOG_FORMAT=json|text.
- On loopback admin binds, startup banner prints a tokenized `?token=` admin
URL plus a bare Token line; SPA reads `?token=` to auto-login then strips it
from the address bar. Omitted on non-loopback binds to avoid token leak.
- `--webui`/`--admin` with no backend prints a one-line hint instead of the
full backend cheat-sheet (UI already covers it).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Tool-call guardrails (lsp_first/quiet_command/write_payload_cap nudges,
fingerprint dedup) for local-LLM tool loops. Configurable via YAML
tool_execution.guardrails, FORGE_TOOL_CALL_POLICY env fallback, or the
admin UI (live, no restart).
- Anthropic thinking-block record/repair (ANTHROPIC_THINKING_REPAIR):
records ground truth off the real API and repairs client-corrupted
thinking/redacted_thinking blocks in replayed conversations.
- Bidirectional thinking_blocks (signature/redacted state) round-trip
through the OpenAI-compat wire format for LiteLLM-style clients.
- Review fixes: guardrail-mode divergence between streaming/non-streaming
paths, cross-backend/tenant cache-namespace collision, client-controlled
integer overflow in thinking budget_tokens, dropped reasoning_content and
citations on repair/translation paths, a fail-closed race under cache
eviction, plus dedup/simplification cleanup and doc corrections.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Key edit modal sent only 4 of the 8 PUT /admin/api/keys/{id} fields; the
endpoint clears omitted fields to NULL, so every save silently wiped the
enforced budget (max_budget_usd), budget_duration, tpm_limit, allowed_models,
and expires_at. The visible "Spend limit" input wrote the unused spend_limit
column (dropped by serde, never enforced). Now the spend input drives
max_budget_usd and all unsurfaced fields are resent from current values.
Also:
- BackendForm edit mode seeds existing values instead of starting blank,
which combined with the no-NULL-clear patch made saves no-op or look like
they wiped endpoint config.
- App handles config_changed websocket events (Settings uses staleTime:
Infinity, so config changes from another session/CLI never refreshed).
- Request-log and observability backend filters are populated from
useBackends() (were empty, dead <select>s); server already filters by name.
- Removed dead AddBackendForm onCreated no-op prop.
- Revoke flow: ConfirmDialog owns its close; parent closes reactively on
success, avoiding setState on an unmounted component.
Verified: tsc + eslint clean, vite build ok, cargo test 1254 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Large admin-ui refactor (Performative component system, sidebar nav,
provider/route tabs) plus backend module restructuring.
Admin UI contract fixes (this session):
- Fix Models page crash: useBackends unwraps {backends:[...]}; align
ModelEntry to {model_name, deployments} and ModelsResponse.strategy;
fix add-model body to {model_name, actual_model, backend_name}.
- Fix Backends/Providers health rendering: source per-backend status and
latency from the uptime endpoint (health_checks); narrow Backend type to
the real get_backends shape.
- Route + sidebar-link the previously-unrouted Backends tab.
Verified: cargo test (exit 0), clippy -D warnings (exit 0), fmt --check,
admin-ui tsc + vite build all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Replace top nav with sidebar layout; add shared Modal, Toast, ConfirmDialog, AsyncBoundary components
- Add Routes tab with full CRUD UI backed by new routes_api admin endpoints
- Split admin rate limiter into separate read/write buckets with method-aware middleware
- Add allowed_routes scoping to virtual keys
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New providers: assemblyai, baidu, blackboxai, brave, cartesia, deepgram,
elevenlabs, exa, iflytek, lmsys, playht, pollinations, serper, siliconflow,
stability, tavily. Registered in mod.rs and registry.rs.
Adds a Providers tab to the admin UI with a ProviderIcon component and
catalog route. Updates backend, streaming, config, and test infrastructure
to align with the expanded provider set. Bumps all crate versions to 0.9.0.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add sanitize_api_key() to strip curly/smart quotes silently injected by
copy-paste from rich-text sources (Slack, docs). Add strip_v1_suffix()
to prevent doubled /v1/v1 paths when provider URLs already include /v1.
Applied across all config paths (env, simple YAML, LiteLLM YAML, TOML).
Also adds crate structure section to proxy-architecture.md and rebuilds
admin UI dist after vite upgrade.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- managed_backends.rs: stamp updated_row.updated_at after SQLite write succeeds,
before inserting into in-memory map, so the PUT response reflects a post-update
timestamp rather than the pre-update value
- providerFields.ts: add hint to rpm/tpm FieldDef entries noting they are stored
for reference and not enforced on managed backends
- queries.ts: type useCatalogProviders queryFn with explicit generic so the
server-guaranteed shape { providers: CatalogProvider[] } is reflected at compile
time; removes the ?? r fallback
- state.rs: correct managed_backends field comment from UUID key to name key
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fix 1: ManagedBackendsSection.tsx line 24 now uses p.display_name instead of
p.name, matching what the Rust catalog handler serializes.
Fix 2: BackendForm.tsx now uses useEffect to initialize providerId when providers
load, eliminating the split identity between state ('') and compensation logic
('value={providerId || providers[0]?.id ?? ''}'). The effect sets providerId to
the first provider when the list first populates and no initial provider is set.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Copy catalog.rs from main; add pub mod catalog + GET routes in admin router
- Add price_per_million_for_model to cost module (missing from this branch)
- Expand CatalogProvider with display_name, status, capabilities, model_count
- BackendForm provider dropdown: use display_name, group by status via optgroup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add BackendForm and ManagedBackendsSection for creating/editing managed
backends at runtime. Integrate ManagedBackendsSection as first section in
Settings tab. Extend Models form with optional backend_name datalist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The field was storing AVG(latency_ms) from the query but named as if it
held payload size. Rename across Rust struct, TypeScript type, RouteTable
column header/cell, and TrafficView bar chart. Remove now-unused formatBytes
helpers from both components.
- Add traffic/uptime module declarations and route registrations
- Update include_str! from admin-ui/index.html to admin-ui/dist/index.html
- Spawn health checker after SharedState construction with backend URL snapshot
- Fix started_at field in cost/mod.rs test SharedState construction
- Unignore dist/ in .gitignore so compiled-in SPA works without a build step
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
