Commit Graph
125 Commits
Author SHA1 Message Date
whit3rabbitandClaude Sonnet 4.6 6d3285dc87 feat(config): add expose_degradation_warnings; auto-enable with PROXY_CONFIG
Adds `expose_degradation_warnings: bool` to `Config`. Defaults to false
(simple mode). Set ANYLLM_DEGRADATION_WARNINGS=true/1 to opt in, or it
enables automatically when PROXY_CONFIG is set (advanced/config-file mode).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 17:00:00 -05:00
whit3rabbit aeed1a6a86 chore: ignore .worktrees/ directory 2026-03-30 16:46:32 -05:00
whit3rabbitandClaude Sonnet 4.6 e26bc71638 docs(security): document CSRF public-route decision; update CLAUDE.md audit summary
Expand get_csrf_token doc comment to explain why the route is public
(login form needs a token before auth), and document the two layers
that make it safe: reject_cross_origin middleware and SameSite=Strict
cookie.

Add 2026-03-30 security audit fix summary to CLAUDE.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:25 -05:00
whit3rabbitandClaude Sonnet 4.6 530093b226 fix(security): reject private/loopback webhook URLs; use SSRF-safe client
Add validate_base_url() check in with_named() URL filter. URLs pointing
to private RFC 1918 ranges, loopback (127.x, ::1, localhost), and
cloud metadata (169.254.169.254) are now rejected with a warning instead
of accepted.

Replace plain reqwest::Client::builder() with build_http_client()
configured with ssrf_protection:true for DNS-level SSRF protection.

Update tests: filters_non_url_callbacks now expects localhost to be
rejected; add rejects_private_and_loopback_webhook_urls test;
rename http_urls_accepted_not_rejected to http_plaintext_to_public_host_accepted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:19 -05:00
whit3rabbitandClaude Sonnet 4.6 58ecc4dd49 fix(security): validate OIDC issuer and JWKS URLs; use SSRF-safe client
Add validate_oidc_url() that delegates to validate_base_url(). Call it
before fetching the discovery document and before fetching the JWKS URI
from the discovery response.

Replace plain reqwest::Client::builder() with build_http_client()
configured with ssrf_protection:true, so DNS-level SSRF protection
applies to both the discovery and JWKS refresh calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:13 -05:00
whit3rabbitandClaude Sonnet 4.6 4fbaf43011 fix(security): populate source_ip in all admin audit log entries
Add ConnectInfo<SocketAddr> extractor to all seven admin mutation
handlers (put_config, delete_config_override, create_key, update_key,
revoke_key, add_model, remove_model). Pass addr.ip().to_string() as
source_ip in each emit_audit call.

Update integration tests to use into_make_service_with_connect_info
and MockConnectInfo so the ConnectInfo extractor is satisfied in test
servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:05 -05:00
whit3rabbitandClaude Sonnet 4.6 3a000fd8b9 fix(security): replace fixed-window admin rate limiter with sliding window
The old implementation used a (window_start, count) tuple that reset every
60 seconds, allowing burst-reset attacks achieving 2x the intended RPM. The
new implementation uses a VecDeque<u64> of millisecond timestamps, evicting
entries older than 60 000 ms on each check, matching the approach already
used by the virtual key rate limiter in keys.rs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 19:23:20 -05:00
whit3rabbitandClaude Sonnet 4.6 be19ac1c36 fix(security): redact AWS_ACCESS_KEY_ID and add GOOGLE_ACCESS_TOKEN to env endpoint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 19:09:37 -05:00
whit3rabbitandClaude Opus 4.6 5c73adfaea feat(admin): team-focused UI overhaul with key editing and audit log
Backend:
- PUT /admin/api/keys/{id}: update virtual key fields (role immutable),
  refreshes DashMap, emits key_updated audit entry
- update_virtual_key(): resets spend period when budget_duration changes
- query_audit_log(): action, target_type, since, until filter params
- query_request_log(): add until filter param; wire through RequestsQuery
- GET /admin/api/metrics: include streaming counters (started/completed/failed/disconnected)
- GET /admin/api/env: include RATE_LIMIT_FAIL_POLICY

UI:
- Dashboard: streaming metrics stat row
- Request Log: key filter dropdown, since/until date pickers, Key column
- Access Control tab: allowed_models tag input, edit modal, budget progress
  bars, key prefix link navigates to filtered request log
- Settings: override badges from overridden_keys, Security section
  (IP allowlist, rate limit fail policy)
- Audit tab: action/target/date filters, paginated table

Tests:
- 7 unit tests for update_virtual_key and query_audit_log filters
- 4 integration tests for PUT /admin/api/keys/{id} lifecycle

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 18:08:34 -05:00
whit3rabbitandClaude Opus 4.6 29badbf639 fix(test): eliminate ADMIN_RPM race in admin_rate_limit_enforced test
Extract check_admin_rate_limit_with_rpm(ip, rpm) so the test passes
rpm directly instead of mutating the global ADMIN_RPM atomic, which
raced with test_router() calling set_admin_rpm(10_000) in parallel.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 20:49:51 -05:00
whit3rabbitandClaude Opus 4.6 e13610667d style: use rsplit+find instead of filter+last to satisfy clippy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:35:14 -05:00
whit3rabbitandClaude Opus 4.6 ea3580fe2e fix: retain millisecond precision in Langfuse start-time calculation
iso8601_to_epoch returned seconds, losing sub-second precision. The
start_time was always off by up to 999ms. New iso8601_to_epoch_ms parses
the fractional-seconds component (.SSS or .SSSSSS) and returns epoch
milliseconds directly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:29:05 -05:00
whit3rabbitandClaude Opus 4.6 33ae113360 fix: persist budget period reset to SQLite to prevent desync across restarts
check_and_reset_period resets period_spend_usd in the DashMap but never
writes to SQLite. accumulate_spend then adds new costs to the stale
old-period total in SQLite. On restart, the bloated value locks keys out.

Fix: propagate period_reset through VirtualKeyContext so record_cost can
call reset_period_spend before accumulate_spend in the same blocking task.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:27:24 -05:00
whit3rabbitandClaude Opus 4.6 4a76511e38 fix(security): block IPv6 ULA (fc00::/7) and link-local (fe80::/10) in SSRF guard
The IPv6 arm of is_private_ip only checked loopback, unspecified, and
IPv4-mapped addresses. Unique Local Addresses and link-local addresses
were not blocked, allowing SSRF via URLs like http://[fc00::1]/v1.
Uses bitwise checks since Ipv6Addr::is_unique_local() is unstable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:21:28 -05:00
whit3rabbitandClaude Opus 4.6 51231d30f3 fix(security): take rightmost X-Forwarded-For IP to prevent allowlist spoofing
The leftmost IP in X-Forwarded-For is attacker-controlled. A trusted
reverse proxy appends the real client IP to the right. Taking .next()
allowed bypassing the IP allowlist with a spoofed header like
"X-Forwarded-For: 127.0.0.1". Now uses .last() on the comma-split.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:18:43 -05:00
whit3rabbitandClaude Sonnet 4.6 3588f52ccb docs: expand library usage examples; borrow header slices in passthrough
Rewrote the "Using as a Library" README section: added cargo dependency
snippets, reorganized HTTP client/pure translation/reverse translation
examples, and added error handling + tool calling code blocks.

Changed AnthropicClient::forward/forward_stream to accept
`&[(&str, &str)]` instead of `&[(String, String)]` so the passthrough
handler can pass string literals directly without allocating.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 14:48:19 -05:00
whit3rabbitandClaude Sonnet 4.6 aeb1e5efb3 feat(passthrough): forward anthropic-beta and x-claude-code-session-id headers upstream
Claude Code v2.1.86+ sends x-claude-code-session-id for session routing/aggregation.
The anthropic-beta header enables beta API features that must reach the upstream to
take effect (e.g., extended thinking, interleaved thinking).

- passthrough.rs: collect and forward the two headers via extra_headers
- anthropic_client.rs: accept extra_headers in forward/forward_stream/send_with_retry
- middleware.rs: log x-claude-code-session-id at debug level alongside anthropic-beta
- rate_limit.rs: parse and forward anthropic-organization-id response header; 3 new tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:43:47 -05:00
whit3rabbitandClaude Sonnet 4.6 9db0465732 docs: update CLAUDE.md with current state
- Test count: ~555 -> ~906
- Remove stale streaming metrics note (fixed)
- Expand Working section with 14 new features added since last update
- Fix batch endpoint from "stub" to real implementation
- Update routes.rs architecture description
- Add 20260327 entry to Recent Changes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:38:43 -05:00
whit3rabbitandClaude Sonnet 4.6 4ed08524fb feat(gemini): emit thinking_delta events in GeminiStreamingTranslator
Separate thought parts (Part.thought=true) from answer parts in each
streaming response. Emit ContentBlockStart(Thinking), ThinkingDelta,
and ContentBlockStop events using the same full-response diffing pattern
as text. Close the thought block before opening the text block. Handle
thought block cleanup in finish(). 3 new streaming tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:38:37 -05:00
whit3rabbitandClaude Sonnet 4.6 dcee73ce2b feat(gemini): map thinking_config to thinkingConfig, thought parts to ThinkingBlock
- Request direction: Anthropic ThinkingConfig::Enabled{budget_tokens} maps to
  generationConfig.thinkingConfig{thinkingBudget, includeThoughts: true}
- Response direction: Part{thought: true, text} maps to ContentBlock::Thinking
- 5 new tests covering all request and response thinking scenarios

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:38:31 -05:00
whit3rabbitandClaude Sonnet 4.6 69d25d0d63 feat(gemini): add Part.thought and GenerationConfig.thinkingConfig types
Add `thought: Option<bool>` to `Part` for thought parts produced by
Gemini 2.5 thinking models. Add `ThinkingConfig` struct and wire it into
`GenerationConfig.thinking_config` for request-side control.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:38:26 -05:00
whit3rabbitandClaude Sonnet 4.6 9b90e5049d fix: upgrade jsonwebtoken 9 -> 10 to resolve CVE (type confusion / auth bypass)
Resolves Dependabot alert #1 (moderate severity).
The jsonwebtoken v10 API is compatible with the existing oidc.rs usage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 10:25:20 -05:00
whit3rabbitandClaude Sonnet 4.6 7961436a14 feat: wire Gemini native path end-to-end (Phases 5-8)
- translate.rs: add translate_request_gemini, translate_response_gemini,
  new_gemini_stream_translator wrappers; re-export from lib.rs
- gemini_native.rs: POST /v1/messages handler for GeminiNative backend;
  non-streaming calls generate_content, streaming uses read_sse_frames +
  GeminiStreamingTranslator to diff full responses into Anthropic SSE events
- streaming.rs: expose read_sse_frames, send_events, StreamOutcome as
  pub(super) so gemini_native.rs can reuse the SSE reading infrastructure
- backend/mod.rs: construct BackendClient::GeminiNative when
  GEMINI_API_FORMAT=native (both single-backend and multi-backend paths)
- routes.rs: add HandlerMode::GeminiNative; detect from BackendClient
  variant at AppState build time; dispatch to gemini_native_handler

GEMINI_API_FORMAT=openai (default) preserves existing behavior.
GEMINI_API_FORMAT=native uses the new direct translation path.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 10:05:01 -05:00
whit3rabbitandClaude Sonnet 4.6 c19a7c137c feat(proxy): add Gemini native HTTP client for generateContent endpoints
- New GeminiNativeClient with generate_content and generate_content_stream methods
- GeminiNative variant added to BackendClient enum and BackendError
- All existing match arms updated to handle the new variant
- 10 unit tests for URL construction, model mapping, error display

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 09:49:28 -05:00
whit3rabbitandClaude Sonnet 4.6 ef8d7d26ca feat(translator): add Gemini streaming state machine with full-response diffing
- GeminiStreamingTranslator diffs accumulated Gemini responses to emit Anthropic SSE deltas
- Handles text diffing, tool call detection, finish reason mapping, usage extraction
- 26 tests covering multi-event streams, tool calls, edge cases

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 09:46:19 -05:00
whit3rabbitandClaude Sonnet 4.6 61276876b8 feat(translator): add Anthropic<->Gemini message mapping with role merge and tool ID synthesis
- anthropic_to_gemini_request: system prompt, messages, tools, tool_choice, generation config
- gemini_to_anthropic_response: text, function calls, stop reason, usage metadata
- Role alternation merge for consecutive same-role messages
- Tool ID map for ToolResult -> FunctionResponse name lookup
- Drops unsupported blocks (thinking, redacted_thinking, document, URL images)
- Reuses sanitize_schema_for_gemini from tools_map
- 42 tests covering request mapping, response mapping, and edge cases

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 09:46:12 -05:00
whit3rabbitandClaude Opus 4.6 ff55dba6e5 feat(translator): add Gemini native API types for generateContent
- GenerateContentRequest, Content, Part (struct with optional fields)
- GenerationConfig, Tool, FunctionDeclaration, ToolConfig, SafetySetting
- GenerateContentResponse, Candidate, FinishReason, UsageMetadata
- Convenience constructors on Part (text, function_call, function_response, inline_data)
- 25 tests: serialization camelCase, round-trips, unknown FinishReason fallback

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 09:41:14 -05:00
whit3rabbitandClaude Opus 4.6 320b44b6c0 fix: resolve clippy warnings from parallel subagent work
- Remove needless borrows in openai_batch_client.rs
- Replace redundant closure with function reference in CSRF middleware
- Narrow handler visibility to pub(crate) for anthropic_batch routes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 08:38:43 -05:00
whit3rabbit 09968ffca1 feat(proxy): register /v1/messages/batches routes, remove legacy stub 2026-03-28 00:04:06 -05:00
whit3rabbit 78def09d43 feat(admin-ui): send CSRF token in all state-mutating requests
- fetchCsrfToken(): fetches from GET /admin/csrf-token on load, stores in _csrfToken
- mutatingHeaders(): returns authHeaders merged with X-CSRF-Token header
- All POST/PUT/DELETE fetch calls now use mutatingHeaders() instead of authHeaders
- fetchCsrfToken() called before loadDashboard() so token is ready at startup
2026-03-27 23:49:51 -05:00
whit3rabbit 1d167e9863 feat(proxy): add Anthropic batch route handlers 2026-03-27 23:47:00 -05:00
whit3rabbit c9e9221caf ci: add cargo-audit step to catch crates with known CVEs
Runs after cargo test in the test job. Uses --locked for reproducibility.
2026-03-27 23:45:36 -05:00
whit3rabbit 6cd37068e3 feat(admin): add CSRF token middleware and /admin/csrf-token endpoint
- validate_csrf middleware: rejects POST/PUT/DELETE without matching X-CSRF-Token header + csrf_token cookie
- GET /admin/csrf-token: public endpoint returning JSON + Set-Cookie (not httpOnly, SameSite=Strict)
- CSRF middleware layered inside validate_admin_token (auth checked first)
- 7 new unit tests in routes.rs for CSRF behavior
- Updated virtual_keys.rs integration tests to include CSRF headers on all POST/DELETE admin calls
2026-03-27 23:45:03 -05:00
whit3rabbit 21bf301b24 feat(proxy): add anthropic_batch_map table for id mapping 2026-03-27 23:44:26 -05:00
whit3rabbit aabe7fe982 feat(proxy): add OpenAI batch API HTTP client 2026-03-27 23:41:47 -05:00
whit3rabbit c494c9a216 feat(translator): apply OpenAI strict mode when Anthropic forces a specific tool
- add extract_forced_tool_name helper: matches ToolChoice::Tool{name}
- add apply_strict_mode_to_tool helper: sets strict=true and normalizes
  parameter schema on the matched ChatTool directly (no serde round-trip)
- anthropic_to_openai_request: calls apply_strict_mode_to_tool when
  tool_choice forces a named tool
- apply_strict_to_forced_tool signature changed to &mut [Value] (clippy)
- apply_strict_mode_to_tool uses &mut [ChatTool] (clippy)
- end-to-end test: forced tool_choice produces strict=true, normalized
  schema with additionalProperties:false and all properties in required
2026-03-27 23:39:06 -05:00
whit3rabbit ee06732f02 feat(translator): add batch JSONL translation functions 2026-03-27 23:38:56 -05:00
whit3rabbit d81f0c32f4 feat(admin): add CSRF token generation and validation helpers
- generate_csrf_token: uses two UUID v4 values for 244 bits of entropy
- extract_csrf_cookie: parses csrf_token from Cookie header string
- validate_csrf_tokens: constant-time comparison, rejects empty tokens
- 7 unit tests covering all cases
2026-03-27 23:37:36 -05:00
whit3rabbit 5e9cb61f4b feat(translator): add normalize_schema_for_strict for OpenAI strict mode
- normalize_schema_for_strict: recursively ensures all object schema
  properties are listed in required and sets additionalProperties: false
- apply_strict_to_forced_tool: sets strict=true and normalizes the
  parameter schema for the named forced tool, leaves others unchanged
- 6 unit tests covering normalization, nesting, merge, non-object, and
  apply_strict cases
2026-03-27 23:34:18 -05:00
whit3rabbit 60b13b6cb6 feat(translator): add Anthropic batch request/response types 2026-03-27 23:34:16 -05:00
whit3rabbit 3bd1db1e5c fix: batch JSONL error messages report absolute line number not non-empty-line count 2026-03-27 23:00:58 -05:00
whit3rabbit 00242ca653 fix: batch JSONL validation uses BufRead to avoid redundant Vec<u8> copy 2026-03-27 22:57:46 -05:00
whit3rabbit f7e4334bee fix: apply Gemini tool schema sanitizer to OpenAI-format chat_completions path 2026-03-27 22:52:36 -05:00
whit3rabbitandClaude Sonnet 4.6 5dacb8cf63 feat: sanitize Gemini tool schemas by stripping unsupported JSON Schema fields
Gemini and Vertex only accept the OpenAPI 3.0 subset of JSON Schema in
function parameters. Adds sanitize_schema_for_gemini() in tools_map.rs and
applies it to all tool parameter schemas in both the non-streaming (routes.rs)
and streaming (streaming.rs) Gemini/Vertex code paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 22:46:11 -05:00
whit3rabbit 52c6ffa3dd feat: compute requests_per_second from SQLite in admin metrics snapshot 2026-03-27 22:38:45 -05:00
whit3rabbit cad85745f3 fix: replace brittle Bedrock event-type substring search with serde_json parse 2026-03-27 22:34:05 -05:00
whit3rabbit 08a03001c3 feat: wire Responses API streaming usage into request log and cost tracking 2026-03-27 22:27:08 -05:00
whit3rabbit c9ce57ebb1 fix: langfuse startTime uses ms precision for sub-second requests 2026-03-27 22:21:09 -05:00
whit3rabbit dbd63437b0 fix: remove stale is_streaming doc comment about metrics tracking 2026-03-27 22:16:33 -05:00
whit3rabbitandClaude Sonnet 4.6 c78e15e48a docs: mark Langfuse integration complete in COMPARISON_LITELLM.md
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 21:34:03 -05:00