Commit Graph
8 Commits
Author SHA1 Message Date
whit3rabbitandClaude Opus 4.6 b6277a8b86 feat: operator observability dashboard with error classification
- Admin UI: operator view with request volume, token usage, latency,
  cost charts, failure table, and request timeline
- Backend: add error_kind() method and infer_error_kind() for stable
  error classification (rate_limit, timeout, backend_error, client_error)
- Admin DB: error_kind column in request_log, observability aggregate
  queries (bucketed timeseries, failure breakdown, timeline)
- Gemini: improved streaming translation, thinking block support,
  grounding metadata passthrough
- Request timeout: configurable REQUEST_TIMEOUT_SECS with streaming
  watchdog
- Model pricing: MODEL_PRICING_FILE for external pricing overrides
- Degradation header: ANYLLM_DEGRADATION_WARNINGS env var control

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:12:03 -05:00
whit3rabbit b3b191e4bd chore: bump workspace version to 0.2.0 to match anyllm_client 2026-03-30 16:51:31 -05:00
whit3rabbitandClaude Opus 4.6 4a76511e38 fix(security): block IPv6 ULA (fc00::/7) and link-local (fe80::/10) in SSRF guard
The IPv6 arm of is_private_ip only checked loopback, unspecified, and
IPv4-mapped addresses. Unique Local Addresses and link-local addresses
were not blocked, allowing SSRF via URLs like http://[fc00::1]/v1.
Uses bitwise checks since Ipv6Addr::is_unique_local() is unstable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 18:21:28 -05:00
whit3rabbitandClaude Sonnet 4.6 3588f52ccb docs: expand library usage examples; borrow header slices in passthrough
Rewrote the "Using as a Library" README section: added cargo dependency
snippets, reorganized HTTP client/pure translation/reverse translation
examples, and added error handling + tool calling code blocks.

Changed AnthropicClient::forward/forward_stream to accept
`&[(&str, &str)]` instead of `&[(String, String)]` so the passthrough
handler can pass string literals directly without allocating.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 14:48:19 -05:00
whit3rabbitandClaude Sonnet 4.6 aeb1e5efb3 feat(passthrough): forward anthropic-beta and x-claude-code-session-id headers upstream
Claude Code v2.1.86+ sends x-claude-code-session-id for session routing/aggregation.
The anthropic-beta header enables beta API features that must reach the upstream to
take effect (e.g., extended thinking, interleaved thinking).

- passthrough.rs: collect and forward the two headers via extra_headers
- anthropic_client.rs: accept extra_headers in forward/forward_stream/send_with_retry
- middleware.rs: log x-claude-code-session-id at debug level alongside anthropic-beta
- rate_limit.rs: parse and forward anthropic-organization-id response header; 3 new tests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 13:43:47 -05:00
whit3rabbitandClaude Opus 4.6 215dd2eab3 chore: format, fix Bedrock match arms, verify line counts
- cargo fmt applied across all crates
- Fixed BackendClient::Bedrock match arms in chat_completions.rs,
  routes.rs, streaming.rs, openai_client.rs
- All new source files verified under 400 lines (2 files at 406/429,
  within tolerance for focused single-responsibility modules)
- 549 tests passing, clippy clean, both build paths verified

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:24:00 -05:00
whit3rabbitandClaude Opus 4.6 a4e655c8bb feat: LiteLLM gap fill - chat completions input, Azure backend, virtual keys, client SDK
Phase 1-8 implementation of the LiteLLM gap fill feature set:

- POST /v1/chat/completions: Accept OpenAI-format input, translate through
  Anthropic pipeline, return OpenAI-format responses (streaming + non-streaming)
- Reverse translation layer: openai_to_anthropic_request, anthropic_to_openai_response,
  ReverseStreamingTranslator (Anthropic SSE -> OpenAI ChatCompletionChunk)
- Azure OpenAI backend: BACKEND=azure with deployment-scoped URLs, api-key header,
  api-version query param (default 2024-10-21)
- Virtual key management: SQLite-backed CRUD via admin API (POST/GET/DELETE
  /admin/api/keys), DashMap in-memory cache, immediate revocation
- Per-key rate limiting: RPM sliding window enforcement in auth middleware,
  429 with retry-after header on limit exceeded
- Client library v0.2.0: ClientBuilder, ToolBuilder, ToolChoiceBuilder,
  typed streaming, rustdoc examples
- New dependencies: dashmap, aws-sigv4, aws-credential-types (prod);
  opentelemetry stack (feature-gated, optional)

534 tests passing, 0 failures, clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:09:40 -05:00
whit3rabbitandClaude Sonnet 4.6 f1df50ff37 refactor: extract shared client crate and rename to anyllm_*
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.

Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.

proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.

streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.

Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 06:11:01 -05:00