starred/anyllm-proxy
1
0
Fork 0
mirror of https://github.com/whit3rabbit/anyllm-proxy.git synced 2026-09-22 16:00:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
194 Commits 14 Branches 21 Tags
8ac9ef2677c14fc024494d846894de3c086510a5
Commit Graph
5 Commits
Author SHA1 Message Date
whit3rabbitandClaude Sonnet 4.6 4bf6b65bb8 security: configurable admin token path via ADMIN_TOKEN_PATH 
Extract resolve_admin_token_path() function to read ADMIN_TOKEN_PATH
env var (falling back to .admin_token). Replaces the previous inline
ADMIN_TOKEN_FILE env var. Updates non-Unix warning to reference the
new env var name.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-27 15:17:53 -05:00
whit3rabbitandClaude Opus 4.6 abb5b90e1d feat: add Bedrock backend, OpenTelemetry export, integration tests 
- AWS Bedrock backend: SigV4 signing, InvokeModel + InvokeModelWithResponseStream
  with binary event stream decoding, passthrough handler for /v1/messages
- OpenTelemetry export: feature-gated (--features otel), OTLP/HTTP with
  reqwest transport, OtelGuard for graceful shutdown flush
- Chat completions integration tests: 6 tests covering non-streaming,
  error handling, degradation headers, system messages
- Updated COMPARISON_LITELLM.md to reflect all closed gaps
- Fixed Bedrock match arms across all handler files

549 tests passing, 0 failures, clippy clean.
Both `cargo build` and `cargo build --features otel` compile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 20:21:24 -05:00
whit3rabbitandClaude Opus 4.6 a4e655c8bb feat: LiteLLM gap fill - chat completions input, Azure backend, virtual keys, client SDK 
Phase 1-8 implementation of the LiteLLM gap fill feature set:

- POST /v1/chat/completions: Accept OpenAI-format input, translate through
  Anthropic pipeline, return OpenAI-format responses (streaming + non-streaming)
- Reverse translation layer: openai_to_anthropic_request, anthropic_to_openai_response,
  ReverseStreamingTranslator (Anthropic SSE -> OpenAI ChatCompletionChunk)
- Azure OpenAI backend: BACKEND=azure with deployment-scoped URLs, api-key header,
  api-version query param (default 2024-10-21)
- Virtual key management: SQLite-backed CRUD via admin API (POST/GET/DELETE
  /admin/api/keys), DashMap in-memory cache, immediate revocation
- Per-key rate limiting: RPM sliding window enforcement in auth middleware,
  429 with retry-after header on limit exceeded
- Client library v0.2.0: ClientBuilder, ToolBuilder, ToolChoiceBuilder,
  typed streaming, rustdoc examples
- New dependencies: dashmap, aws-sigv4, aws-credential-types (prod);
  opentelemetry stack (feature-gated, optional)

534 tests passing, 0 failures, clippy clean.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-25 20:09:40 -05:00
whit3rabbitandClaude Sonnet 4.6 f1df50ff37 refactor: extract shared client crate and rename to anyllm_* 
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.

Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.

proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.

streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.

Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-25 06:11:01 -05:00
whit3rabbitandClaude Opus 4.6 d37d1e25f1 Add phases 12-20: release infra, transparent proxy, model mapping, mTLS, extended thinking, Gemini research 
Phases 12-18: release infrastructure (LICENSE, README, Dockerfile, CI,
CHANGELOG), transparent proxy with anthropic-version/anthropic-beta header
passthrough and lossy translation warnings, BIG_MODEL/SMALL_MODEL env-based
model mapping, mTLS client cert support (P12/PEM), max_completion_tokens
and reasoning_effort passthrough via serde flatten, extended thinking type
support (thinking blocks stripped in translation), top_k typed field.

Phase 20: Gemini backend research with docs/gemini-api-diffs.md covering
native API format, tool calling, streaming, auth, schema restrictions,
and Vertex AI OpenAI-compatible endpoint. Task roadmap through Phase 22.

Test count: 169 -> expanded with new fixture and unit tests for all phases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-21 20:06:50 -05:00