- Auth default is now loopback-open (not reject-all). With no
PROXY_API_KEYS, no PROXY_OPEN_RELAY, no virtual keys and no OIDC,
loopback TCP peers are accepted and LAN/remote peers get 401. The
decision uses the real TCP peer (ConnectInfo via
into_make_service_with_connect_info), not the spoofable
X-Forwarded-For. effective_auth_mode() (keys/open_relay/loopback_only)
+ proxy_key_count are surfaced on GET /admin/api/status; the admin UI
shows a warning banner when no key is set.
- Add --port/-p CLI flag that sets LISTEN_PORT for the run. Stripped
before any run/providers subcommand so flags meant for the launched
tool survive; pure scan is unit-tested.
- Startup port handling: the run subcommand pre-checks the listen port
and fails fast with a hint when in use; wait_for_port readiness timeout
10s -> 30s; listener bind failures (proxy + admin) now print an
actionable message and exit(1) instead of panicking.
- POST /v1/chat/completions no longer 400s on a missing max_tokens for
OpenAI-compatible backends. The internal placeholder is stripped via a
new OMIT_MAX_TOKENS_MARKER so the backend applies its own default
(e.g. LM Studio's 8192); the marker never leaks upstream. Explicit
max_tokens is still forwarded verbatim. Anthropic backends unchanged.
- Tier router logs the selected tier at info (tier/backend/model)
instead of routing silently.
- Admin UI modal no longer dismisses when a text-selection drag starts
inside the card (dismiss only on a press that begins on the backdrop).
Co-Authored-By: Claude <noreply@anthropic.com>
- Split config.rs into config/{mod,delete,get,put}.rs
- Split routes_api.rs into routes_api/{mod,helpers,providers,routes}.rs
- Split passthrough/handlers.rs into handlers/{mod,errors,generic,messages}.rs
- Split streaming.rs into streaming/{mod,handler,helpers}.rs
- Split main_helpers/async_main/admin.rs into admin/{mod,config,tasks}.rs
- Minor cleanups in chat_completions backends, token_counting, tests
- Add docs/TEST_PARITY_LITELLM.md
Co-Authored-By: Claude <noreply@anthropic.com>
- Tool-call guardrails (lsp_first/quiet_command/write_payload_cap nudges,
fingerprint dedup) for local-LLM tool loops. Configurable via YAML
tool_execution.guardrails, FORGE_TOOL_CALL_POLICY env fallback, or the
admin UI (live, no restart).
- Anthropic thinking-block record/repair (ANTHROPIC_THINKING_REPAIR):
records ground truth off the real API and repairs client-corrupted
thinking/redacted_thinking blocks in replayed conversations.
- Bidirectional thinking_blocks (signature/redacted state) round-trip
through the OpenAI-compat wire format for LiteLLM-style clients.
- Review fixes: guardrail-mode divergence between streaming/non-streaming
paths, cross-backend/tenant cache-namespace collision, client-controlled
integer overflow in thinking budget_tokens, dropped reasoning_content and
citations on repair/translation paths, a fail-closed race under cache
eviction, plus dedup/simplification cleanup and doc corrections.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Large admin-ui refactor (Performative component system, sidebar nav,
provider/route tabs) plus backend module restructuring.
Admin UI contract fixes (this session):
- Fix Models page crash: useBackends unwraps {backends:[...]}; align
ModelEntry to {model_name, deployments} and ModelsResponse.strategy;
fix add-model body to {model_name, actual_model, backend_name}.
- Fix Backends/Providers health rendering: source per-backend status and
latency from the uptime endpoint (health_checks); narrow Backend type to
the real get_backends shape.
- Route + sidebar-link the previously-unrouted Backends tab.
Verified: cargo test (exit 0), clippy -D warnings (exit 0), fmt --check,
admin-ui tsc + vite build all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CI runs stable (now 1.95) which surfaces two new lints my 1.94 local
check missed:
- collapsible_match in reverse_message_map.rs: collapse the empty
guard branch into a match arm with a guard.
- manual_checked_ops in admin/db.rs: replace the explicit zero check
with checked_div(...).unwrap_or(0).
- collapsible_match in admin/ws.rs: clippy's suggested guard form
fails to compile because Bytes can't be moved in a pattern guard.
Apply a targeted #[allow] with a comment explaining why.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- tools_map: use std::slice::from_ref over &[clone()] in 4 tests
- gemini_streaming_map: rewrite match as matches!
- backend/mod.rs: move impl BackendClient before #[cfg(test)] mod tests
- middleware: replace .filter().last() with .rfind() (xff parsing)
- middleware: drop dead `|| true` in is_ip_allowed smoke test
- sse: drop blank line between doc comment and assert_sse_ok
- streaming example: collapse nested if-let into outer match arms
- tool_execution tests: array literal over vec! for one-off slices
- live_bedrock: use is_some_and instead of map_or(false, _)
- live_api / live_responses: contains() over iter().any() on &[&str]
All test-only / example changes; no production behavior change.
1130 tests pass, fmt clean, clippy -D warnings clean.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Wrap admin token in Zeroizing<String> so memory is wiped on drop
- Use SSRF-safe HTTP client for Langfuse and webhook dispatcher
- Wire up webhook dispatcher at startup (was previously un-started)
- Fix batch expires_at: was using now instead of now+24h
- Extract epoch_secs() helper; replace 4 inline SystemTime::now() blocks
- Gemini tool_choice {type:tool}: use ANY+allowedFunctionNames instead of AUTO
- Map Anthropic thinking budget_tokens to OpenAI reasoning_effort
- Preserve temperature/top_p for GA o-series models (o1/o3/o3-mini/o4-mini);
only strip for o1-preview and o1-mini which reject those params
- Azure simple config: always route through default_base_url; guard against
double-appending deployment path when user provides a full URL
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Verifies that idx == MAX_TOOL_CALL_INDEX (128) is accepted (> not >=)
and idx == MAX_TOOL_CALL_INDEX + 1 (129) is silently dropped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace hardcoded list [o1, o3, o4] with pattern-based detection: any model
name starting with 'o' or 'O', followed by one or more ASCII digits, then
optional '-' suffix. This future-proofs the check for o2, o5, o10, etc. when
OpenAI releases them, without requiring code changes.
Includes test_is_o_series_model_future_models to validate o2, o5, o10
handling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Separate thought parts (Part.thought=true) from answer parts in each
streaming response. Emit ContentBlockStart(Thinking), ThinkingDelta,
and ContentBlockStop events using the same full-response diffing pattern
as text. Close the thought block before opening the text block. Handle
thought block cleanup in finish(). 3 new streaming tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add `thought: Option<bool>` to `Part` for thought parts produced by
Gemini 2.5 thinking models. Add `ThinkingConfig` struct and wire it into
`GenerationConfig.thinking_config` for request-side control.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- translate.rs: add translate_request_gemini, translate_response_gemini,
new_gemini_stream_translator wrappers; re-export from lib.rs
- gemini_native.rs: POST /v1/messages handler for GeminiNative backend;
non-streaming calls generate_content, streaming uses read_sse_frames +
GeminiStreamingTranslator to diff full responses into Anthropic SSE events
- streaming.rs: expose read_sse_frames, send_events, StreamOutcome as
pub(super) so gemini_native.rs can reuse the SSE reading infrastructure
- backend/mod.rs: construct BackendClient::GeminiNative when
GEMINI_API_FORMAT=native (both single-backend and multi-backend paths)
- routes.rs: add HandlerMode::GeminiNative; detect from BackendClient
variant at AppState build time; dispatch to gemini_native_handler
GEMINI_API_FORMAT=openai (default) preserves existing behavior.
GEMINI_API_FORMAT=native uses the new direct translation path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- normalize_schema_for_strict: recursively ensures all object schema
properties are listed in required and sets additionalProperties: false
- apply_strict_to_forced_tool: sets strict=true and normalizes the
parameter schema for the named forced tool, leaves others unchanged
- 6 unit tests covering normalization, nesting, merge, non-object, and
apply_strict cases
Gemini and Vertex only accept the OpenAPI 3.0 subset of JSON Schema in
function parameters. Adds sanitize_schema_for_gemini() in tools_map.rs and
applies it to all tool parameter schemas in both the non-streaming (routes.rs)
and streaming (streaming.rs) Gemini/Vertex code paths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Extract row_to_virtual_key helper (3 copies -> 1) in admin/db.rs
- Deduplicate stop_reason mapping: reverse_streaming_map now calls
the shared anthropic_stop_reason_to_openai from reverse_message_map
- Fix stream_options dead code in chat_completions streaming path
(omit_stream_options was overwritten unconditionally)
- Add buffer size limit to Bedrock event stream decoder (was unbounded)
- Replace full serde_json parse with string extraction in detect_event_type
(runs on every Bedrock streaming event)
- Remove redundant WHAT comments in reverse_message_map.rs
- Fix clippy redundant closure warnings in db.rs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- cargo fmt applied across all crates
- Fixed BackendClient::Bedrock match arms in chat_completions.rs,
routes.rs, streaming.rs, openai_client.rs
- All new source files verified under 400 lines (2 files at 406/429,
within tolerance for focused single-responsibility modules)
- 549 tests passing, clippy clean, both build paths verified
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.
Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.
proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.
streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.
Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add doc comments to ~30 undocumented public functions, structs, enum
variants, and fields across both crates. Strengthen weak comments to
explain "why" not just "what". Remove 16 stale PLAN.md line references
(PLAN.md was removed previously). Add Anthropic API doc links where
relevant for rate limit headers and ID format.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Handle o-series reasoning models (o1, o3, o4-mini): drop max_tokens
(keep only max_completion_tokens) and convert system -> developer role
- Restore read_timeout(900s) alongside tcp_keepalive(60s) to bound hung
connections that keepalive alone cannot detect
- Reuse epoch_to_iso8601 from admin/db.rs instead of duplicating the
Hinnant civil date algorithm in backend/mod.rs
- Make ISO 8601 conversion testable via anchor time parameter
- Fix f64 truncation in duration parsing (use .round() before cast)
- Extract convert_reset_duration helper to deduplicate header injection
- Use eq_ignore_ascii_case in is_o_series_model to avoid allocation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security: hash API keys with SHA-256 for constant-time comparison (eliminates
length timing leak), require explicit PROXY_OPEN_RELAY for unauthenticated
access, gate /metrics behind auth, sanitize admin error responses, enforce
log_level allowlist on DB restore, validate GCP identifiers against URL
injection, add referrer-policy header to admin SPA.
Correctness: hold concurrency semaphore permit through entire stream lifetime
(not just until headers are sent), fix SSE parser to resume scanning near
chunk boundaries instead of re-scanning from start, mark padding tool call
slots as closed to prevent spurious ContentBlockStop events, mark responses
streaming translator as finished on error to prevent double closure events,
serialize concurrent config writes with a Mutex.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents hard deserialization failures from unknown finish reasons
(e.g. DeepSeek's "insufficient_system_resource") and adds bidirectional
mapping of reasoning_content to/from Anthropic thinking blocks for
DeepSeek/Qwen thinking models.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split config.rs into config/{mod,tls,url_validation}.rs and extract
server/{passthrough,streaming,token_counting}.rs and admin/ws.rs from
their parent modules for clarity.
Functional changes:
- Add DNS rebinding protection on admin API (Host header validation)
- Fix IPv6 host parsing in admin origin check (bare ::1 was mishandled)
- Return Anthropic-shaped errors for JSON parse failures (400 not 422)
and unmatched routes (404 not_found_error)
- Forward extra fields from Anthropic request to OpenAI request
- Count "Error: " prefix tokens for error tool results
- Reduce TOKENIZER visibility to module-private
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Constant-time token comparison in admin auth (timing side-channel)
- Origin header parsing via URL to prevent bypass (e.g., 127.0.0.1.attacker.com)
- CSP headers and X-Frame-Options on admin SPA
- WebSocket origin check for cross-site WS hijacking prevention
- Switch admin DB mutex from tokio::Mutex to std::Mutex, use spawn_blocking
- Poison-recovery on std::sync locks (unwrap_or_else + into_inner)
- JSON builder for SSE error fallback to prevent injection
- UTF-8-safe secret redaction (char-aware slicing)
- Add RedactedThinking content block and SignatureDelta streaming types
- Map OpenAI refusal field to Anthropic text block (non-streaming + streaming)
- Map OpenAI cached_tokens to Anthropic cache_read_input_tokens
- Map HTTP 408 to Anthropic OverloadedError
- Warn on trace log level and log_bodies enable via admin API
- Update CLAUDE.md docs to match current state
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Gemini's OpenAI-compatible endpoint (/openai) supports the same Chat
Completions format as the OpenAI backend, making the native Gemini
translation path redundant. This removes ~3400 lines of Gemini-specific
types, mapping, streaming, and client code, routing Gemini through the
existing OpenAI translation pipeline instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Security: strip admin token from browser URL, add cross-origin rejection
middleware for admin API, skip tool calls with empty names instead of
substituting "unknown".
Correctness: use BytesMut for SSE buffering to prevent UTF-8 corruption
at TCP chunk boundaries, use saturating_sub for epoch arithmetic, handle
CRLF SSE frame delimiters.
Runtime: switch runtime_config to std::sync::RwLock (guard is !Send),
use block_in_place for SQLite IO, spawn_blocking for tokenization,
add tracing reload layer so admin log_level changes apply immediately.
Reliability: retry failed log buffer flushes with capped retry queue,
add MAX_SSE_BUFFER_SIZE guard in middleware handler, cap tool call and
part indices to prevent unbounded vec growth.
Observability: defer streaming request logging until stream completes
so entries capture actual status, latency, and token counts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire up Responses API as alternative to Chat Completions via
OPENAI_API_FORMAT=responses env var. Adds request/response mapping,
streaming state machine, client methods, and route dispatch. Includes
live API integration tests (ignored by default, require OPENAI_API_KEY).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
