starred/anyllm-proxy
1
0
Fork 0
mirror of https://github.com/whit3rabbit/anyllm-proxy.git synced 2026-09-21 16:00:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
31 Commits 14 Branches 21 Tags
215dd2eab3c96ec7940a9799693272fde9727513
Commit Graph
8 Commits
Author SHA1 Message Date
whit3rabbitandClaude Opus 4.6 215dd2eab3 chore: format, fix Bedrock match arms, verify line counts 
- cargo fmt applied across all crates
- Fixed BackendClient::Bedrock match arms in chat_completions.rs,
  routes.rs, streaming.rs, openai_client.rs
- All new source files verified under 400 lines (2 files at 406/429,
  within tolerance for focused single-responsibility modules)
- 549 tests passing, clippy clean, both build paths verified

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 20:24:00 -05:00
whit3rabbitandClaude Opus 4.6 abb5b90e1d feat: add Bedrock backend, OpenTelemetry export, integration tests 
- AWS Bedrock backend: SigV4 signing, InvokeModel + InvokeModelWithResponseStream
  with binary event stream decoding, passthrough handler for /v1/messages
- OpenTelemetry export: feature-gated (--features otel), OTLP/HTTP with
  reqwest transport, OtelGuard for graceful shutdown flush
- Chat completions integration tests: 6 tests covering non-streaming,
  error handling, degradation headers, system messages
- Updated COMPARISON_LITELLM.md to reflect all closed gaps
- Fixed Bedrock match arms across all handler files

549 tests passing, 0 failures, clippy clean.
Both `cargo build` and `cargo build --features otel` compile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 20:21:24 -05:00
whit3rabbitandClaude Opus 4.6 a4e655c8bb feat: LiteLLM gap fill - chat completions input, Azure backend, virtual keys, client SDK 
Phase 1-8 implementation of the LiteLLM gap fill feature set:

- POST /v1/chat/completions: Accept OpenAI-format input, translate through
  Anthropic pipeline, return OpenAI-format responses (streaming + non-streaming)
- Reverse translation layer: openai_to_anthropic_request, anthropic_to_openai_response,
  ReverseStreamingTranslator (Anthropic SSE -> OpenAI ChatCompletionChunk)
- Azure OpenAI backend: BACKEND=azure with deployment-scoped URLs, api-key header,
  api-version query param (default 2024-10-21)
- Virtual key management: SQLite-backed CRUD via admin API (POST/GET/DELETE
  /admin/api/keys), DashMap in-memory cache, immediate revocation
- Per-key rate limiting: RPM sliding window enforcement in auth middleware,
  429 with retry-after header on limit exceeded
- Client library v0.2.0: ClientBuilder, ToolBuilder, ToolChoiceBuilder,
  typed streaming, rustdoc examples
- New dependencies: dashmap, aws-sigv4, aws-credential-types (prod);
  opentelemetry stack (feature-gated, optional)

534 tests passing, 0 failures, clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 20:09:40 -05:00
whit3rabbitandClaude Sonnet 4.6 f1df50ff37 refactor: extract shared client crate and rename to anyllm_* 
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.

Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.

proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.

streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.

Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-25 06:11:01 -05:00
whit3rabbitandClaude Opus 4.6 0a9ca1ae31 fix: simplify retry loop, sanitize request IDs, and apply rustfmt (Phase 22) 
Collapse retry + final-attempt into single inclusive loop, drain response
body before retry to return connections to pool, and replace panic on
invalid x-request-id with UUID fallback. Update CLAUDE.md/TASKS.md for
Phase 22 status. Apply cargo fmt across touched files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 13:41:06 -05:00
whit3rabbitandClaude Opus 4.6 76e9392656 Add native Gemini backend, library mode, and middleware (Phases 20d-20g, 21a-21c) 
Gemini native backend:
- Schema sanitizer strips unsupported JSON Schema keys for Gemini (20d)
- Anthropic-to-Gemini message mapping with role coercion and turn merging (20e)
- Streaming state machine for Gemini SSE and Vertex AI responses (20f)
- GeminiClient with retry/backoff, BackendClient enum dispatch (20g)
- Unified BackendError with per-backend error helpers
- Shared retry logic extracted to backend/mod.rs

Library mode (21a-21c):
- TranslationConfig with builder pattern, model mapping, lossy behavior control
- translate_request/translate_response convenience functions
- Public TranslateError enum, crate-level doc examples
- Axum middleware layer (AnthropicTranslationLayer) for embedding in existing services
- Feature-gated middleware deps (axum, reqwest, tokio)
- library_usage and middleware_integration test suites

Also: new Claude Code tool call fixtures, updated CLAUDE.md and TASKS.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 09:20:22 -05:00
whit3rabbitandClaude Opus 4.6 d37d1e25f1 Add phases 12-20: release infra, transparent proxy, model mapping, mTLS, extended thinking, Gemini research 
Phases 12-18: release infrastructure (LICENSE, README, Dockerfile, CI,
CHANGELOG), transparent proxy with anthropic-version/anthropic-beta header
passthrough and lossy translation warnings, BIG_MODEL/SMALL_MODEL env-based
model mapping, mTLS client cert support (P12/PEM), max_completion_tokens
and reasoning_effort passthrough via serde flatten, extended thinking type
support (thinking blocks stripped in translation), top_k typed field.

Phase 20: Gemini backend research with docs/gemini-api-diffs.md covering
native API format, tool calling, streaming, auth, schema restrictions,
and Vertex AI OpenAI-compatible endpoint. Task roadmap through Phase 22.

Test count: 169 -> expanded with new fixture and unit tests for all phases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 20:06:50 -05:00
whit3rabbitandClaude Opus 4.6 f2e3ed15f4 Initial commit: Anthropic-to-OpenAI API translation proxy 
Rust workspace with two crates:
- translator: pure, IO-free mapping between Anthropic Messages API and OpenAI Chat Completions
- proxy: axum HTTP server with auth, streaming SSE, retry/backoff, concurrency limits

169 tests passing (unit, golden fixture, integration).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 13:25:43 -05:00