diff --git a/CLAUDE.md b/CLAUDE.md index 0262758..8580e39 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## What This Is -An Anthropic-to-OpenAI API translation proxy in Rust. Accepts Anthropic Messages API requests, translates them to OpenAI Chat Completions format, forwards to OpenAI, and translates back. Supports streaming SSE, tool calling, file/document blocks. +**anyllm-proxy** is an Anthropic-to-OpenAI API translation proxy in Rust. Accepts Anthropic Messages API requests, translates them to OpenAI Chat Completions format, forwards to OpenAI, and translates back. Supports streaming SSE, tool calling, file/document blocks. All 11 implementation phases are complete. @@ -28,8 +28,8 @@ All 11 implementation phases are complete. ```bash cargo build # build everything cargo test # run all tests (~417 tests, 4 ignored) -cargo test -p anthropic_openai_translate # translator crate only -cargo test -p anthropic_openai_proxy # proxy crate only +cargo test -p anyllm_translate # translator crate only +cargo test -p anyllm_proxy # proxy crate only cargo test health_endpoint # single test by name cargo clippy -- -D warnings # lint cargo fmt --check # format check @@ -37,7 +37,7 @@ cargo fmt --check # format check Run the proxy (requires OPENAI_API_KEY): ```bash -OPENAI_API_KEY=sk-... cargo run -p anthropic_openai_proxy +OPENAI_API_KEY=sk-... cargo run -p anyllm_proxy # Listens on 0.0.0.0:3000, health at GET /health ``` @@ -50,7 +50,7 @@ OPENAI_API_KEY=sk-... cargo run -p anthropic_openai_proxy - `LISTEN_PORT`: Server port (default: `3000`) - `BIG_MODEL`: Backend model for sonnet/opus requests (default: `gpt-4o` for OpenAI, `gemini-2.5-pro` for Vertex/Gemini) - `SMALL_MODEL`: Backend model for haiku requests (default: `gpt-4o-mini` for OpenAI, `gemini-2.5-flash` for Vertex/Gemini) -- `RUST_LOG`: Tracing filter (e.g., `info`, `anthropic_openai_proxy=debug`) +- `RUST_LOG`: Tracing filter (e.g., `info`, `anyllm_proxy=debug`) - `TLS_CLIENT_CERT_P12`: Path to PKCS#12 (.p12/.pfx) client certificate for mTLS to the backend (optional) - `TLS_CLIENT_CERT_PASSWORD`: Password to decrypt the P12 file (required if P12 is set) - `TLS_CA_CERT`: Path to PEM-encoded CA certificate for verifying the backend server (optional) @@ -67,7 +67,7 @@ OPENAI_API_KEY=sk-... cargo run -p anthropic_openai_proxy Cargo workspace with two crates: -### `crates/translator` (lib: `anthropic_openai_translate`) +### `crates/translator` (lib: `anyllm_translate`) Pure translation logic, no IO. Key modules: - **`anthropic/`**: Anthropic Messages API types (request, response, streaming events, errors) - **`openai/`**: OpenAI types for both Chat Completions and Responses APIs @@ -83,7 +83,7 @@ Pure translation logic, no IO. Key modules: - **`util/`**: JSON helpers, ID generation (uuid v4), secret redaction - **`config.rs`**: Translator-level configuration, **`error.rs`**: Error types, **`translate.rs`**: Top-level translation entry points -### `crates/proxy` (bin: `anthropic_openai_proxy`) +### `crates/proxy` (bin: `anyllm_proxy`) HTTP proxy built on axum + reqwest: - **`config/`**: Env-based configuration (`mod.rs`), TLS client cert setup (`tls.rs`), URL validation (`url_validation.rs`) - **`server/routes.rs`**: Axum router (POST /v1/messages, GET /health, GET /metrics, GET /v1/models, stubs for count_tokens and batches) diff --git a/Cargo.toml b/Cargo.toml index 4020761..e0811c0 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,9 +1,9 @@ [workspace] -members = ["crates/translator", "crates/proxy"] +members = ["crates/translator", "crates/client", "crates/proxy"] resolver = "2" [workspace.package] version = "0.1.0" edition = "2021" license = "MIT" -repository = "https://github.com/whit3rabbit/llm-translate-api" +repository = "https://github.com/whit3rabbit/anyllm-proxy" diff --git a/Dockerfile b/Dockerfile index ccaa1fb..1da14c9 100644 --- a/Dockerfile +++ b/Dockerfile @@ -3,10 +3,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends libssl-dev pkg- WORKDIR /app COPY Cargo.toml Cargo.lock ./ COPY crates crates -RUN cargo build --release -p anthropic_openai_proxy +RUN cargo build --release -p anyllm_proxy FROM debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates libssl3 && rm -rf /var/lib/apt/lists/* -COPY --from=builder /app/target/release/anthropic_openai_proxy /usr/local/bin/ +COPY --from=builder /app/target/release/anyllm_proxy /usr/local/bin/ EXPOSE 3000 -ENTRYPOINT ["anthropic_openai_proxy"] +ENTRYPOINT ["anyllm_proxy"] diff --git a/README.md b/README.md index b092124..6c40835 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# llm-translate-api (Project Name Pending) +# anyllm-proxy (Project Name Pending) An API translation proxy that allows Anthropic-based applications (like Claude Code, Cursor, Windsurf, or Cline) to interact seamlessly with any OpenAI-compatible backend, local LLMs, or alternative API providers. @@ -8,6 +8,7 @@ An API translation proxy that allows Anthropic-based applications (like Claude C A lightweight, fast Rust-based proxy that accepts Anthropic Messages API requests, translates them to OpenAI Chat Completions formats, forwards them to any compliant backend, and translates the responses back to the Anthropic format in real-time. It completely supports streaming SSE, tool calling, and image/document blocks. ### Why use it? + - **Local AI Coding:** Run powerful tools like Claude Code against local models (Llama 3, DeepSeek, Qwen) without spending expensive API credits. - **Broad Compatibility:** Readily works with leading open-weights and alternative models, including Chinese models like Qwen and DeepSeek. - **Multi-Backend Routing:** Define multiple routes simultaneously. Send simple `haiku` prompts to a fast local model, and complex `opus` requests to external providers, all transparently. @@ -21,7 +22,7 @@ First, ensure you have Rust installed. cargo build # Run the proxy -cargo run -p anthropic_openai_proxy +cargo run -p anyllm_proxy ``` By default, the proxy listens on `0.0.0.0:3000` for incoming API requests and starts a local admin dashboard on `127.0.0.1:3001`. @@ -44,7 +45,7 @@ OPENAI_API_KEY=unused \ OPENAI_BASE_URL=http://localhost:11434/v1 \ BIG_MODEL=qwen2.5-coder:32b \ SMALL_MODEL=qwen2.5-coder:32b \ -cargo run -p anthropic_openai_proxy & +cargo run -p anyllm_proxy & # 3. Use Claude Code targeting the local proxy ANTHROPIC_BASE_URL=http://localhost:3000 claude @@ -88,7 +89,7 @@ small_model = "google/gemini-2.5-flash" Run with this configuration: ```bash -PROXY_CONFIG=config.toml cargo run -p anthropic_openai_proxy +PROXY_CONFIG=config.toml cargo run -p anyllm_proxy ``` This sets up multiple endpoints that your client applications can hit: @@ -116,7 +117,7 @@ OPENAI_API_KEY=sk-or-... \ OPENAI_BASE_URL=https://openrouter.ai/api/v1 \ BIG_MODEL=anthropic/claude-3.5-sonnet \ SMALL_MODEL=anthropic/claude-3-haiku \ -cargo run -p anthropic_openai_proxy +cargo run -p anyllm_proxy ``` **OpenAI Example:** @@ -124,7 +125,7 @@ cargo run -p anthropic_openai_proxy OPENAI_API_KEY=sk-... \ BIG_MODEL=gpt-4o \ SMALL_MODEL=gpt-4o-mini \ -cargo run -p anthropic_openai_proxy +cargo run -p anyllm_proxy ``` **Google Gemini Example:** @@ -133,9 +134,124 @@ BACKEND=gemini \ GEMINI_API_KEY=AIza... \ BIG_MODEL=gemini-2.5-pro \ SMALL_MODEL=gemini-2.5-flash \ -cargo run -p anthropic_openai_proxy +cargo run -p anyllm_proxy ``` +--- + +## 4. Using as a Library + +Beyond the proxy binary, the translation engine is available as reusable Rust crates. Pick the level of abstraction that fits your project: + +``` +crates/translator (lib, IO-free pure translation) + | +crates/client (lib, async HTTP client wrapping translator) + | +crates/proxy (bin, full proxy server) +``` + +| Level | Crate | Use Case | +|---|---|---| +| **Pure translation** | `anyllm_translate` | Stateless type conversion between Anthropic and OpenAI formats. No IO, no HTTP. Bring your own transport. | +| **HTTP client** | `anyllm_client` | `client.messages(req).await` -- send Anthropic requests, get Anthropic responses. Handles translation, HTTP, retry, and streaming internally. | +| **Embedded middleware** | `anyllm_translate` with `middleware` feature | Drop-in Tower Layer or axum Router that adds `/v1/messages` to an existing server. | +| **Full proxy** | `anyllm_proxy` | Multi-backend routing, admin UI, metrics, auth. Everything in this README. | + +### Pure Translation (no IO) + +Use `anyllm_translate` when you want full control over HTTP and just need the format conversion. + +```rust +use anyllm_translate::{TranslationConfig, translate_request, translate_response}; +use anyllm_translate::anthropic::MessageCreateRequest; + +// Configure model mapping +let config = TranslationConfig::builder() + .model_map("haiku", "gpt-4o-mini") + .model_map("sonnet", "gpt-4o") + .build(); + +// Translate an Anthropic request to OpenAI format +let anthropic_req: MessageCreateRequest = serde_json::from_str(&body)?; +let openai_req = translate_request(&anthropic_req, &config)?; + +// ... send openai_req with your own HTTP client ... + +// Translate the OpenAI response back to Anthropic format +let anthropic_resp = translate_response(&openai_resp, &anthropic_req.model); +``` + +For streaming, use the stateful translator: + +```rust +use anyllm_translate::new_stream_translator; + +let mut translator = new_stream_translator(model); +// Feed OpenAI chunks as they arrive: +let events = translator.process_chunk(&chunk); +// After the stream ends: +let final_events = translator.finish(); +``` + +### HTTP Client (translation + transport) + +Use `anyllm_client` when you want Anthropic-in, Anthropic-out with no boilerplate. + +```rust +use anyllm_client::{Client, ClientConfig, Auth}; +use anyllm_translate::TranslationConfig; + +let client = Client::new( + ClientConfig::builder() + .backend_url("https://api.openai.com/v1/chat/completions") + .auth(Auth::Bearer("sk-...".into())) + .translation( + TranslationConfig::builder() + .model_map("sonnet", "gpt-4o") + .build() + ) + .build() +); + +// Non-streaming +let response = client.messages(&anthropic_request).await?; + +// Streaming (returns a Stream of Anthropic SSE events) +let (stream, rate_limits) = client.messages_stream(&anthropic_request).await?; +``` + +The client includes retry with exponential backoff, SSRF-safe DNS resolution, mTLS support, and rate limit header forwarding. + +### Embedded Middleware (for existing axum apps) + +Enable the `middleware` feature on `anyllm_translate` to add Anthropic-compatible endpoints to your own server: + +```rust +use anyllm_translate::middleware::{anthropic_compat_router, AnthropicCompatConfig}; + +let config = AnthropicCompatConfig::builder() + .backend_url("https://api.openai.com") + .api_key("sk-...") + .build(); + +// Merge into your existing axum Router +let app = Router::new() + .merge(anthropic_compat_router(config)) + .route("/my-other-endpoint", get(handler)); +``` + +Or use the Tower Layer to intercept requests transparently: + +```rust +use anyllm_translate::middleware::AnthropicTranslationLayer; + +let app = Router::new() + .layer(AnthropicTranslationLayer::new(config)); +``` + +--- + ## Advanced Features - **Streaming SSE**: Real-time translation of chunked responses, preserving typing feel. diff --git a/crates/client/Cargo.toml b/crates/client/Cargo.toml new file mode 100644 index 0000000..e548191 --- /dev/null +++ b/crates/client/Cargo.toml @@ -0,0 +1,30 @@ +[package] +name = "anyllm_client" +description = "Async HTTP client for Anthropic-to-OpenAI translation with retry, SSRF protection, and SSE streaming" +version.workspace = true +edition.workspace = true +license.workspace = true +repository.workspace = true +categories = ["api-bindings", "web-programming::http-client"] +keywords = ["anthropic", "openai", "translation", "llm", "client"] + +[features] +default = ["ssrf-protection"] +ssrf-protection = [] + +[dependencies] +anyllm_translate = { path = "../translator" } +reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "native-tls", "http2"] } +tokio = { version = "1", features = ["rt", "sync", "time"] } +serde = { version = "1", features = ["derive"] } +serde_json = "1" +tracing = "0.1" +bytes = "1" +futures = "0.3" +httpdate = "1" +thiserror = "2" +pin-project-lite = "0.2" + +[dev-dependencies] +tokio = { version = "1", features = ["full"] } +pretty_assertions = "1" diff --git a/crates/client/src/client.rs b/crates/client/src/client.rs new file mode 100644 index 0000000..58a4b1a --- /dev/null +++ b/crates/client/src/client.rs @@ -0,0 +1,361 @@ +//! High-level async client: Anthropic request in, Anthropic response out. +//! +//! Combines translation, HTTP, retry, and SSE streaming into a single ergonomic API. + +use anyllm_translate::anthropic::messages::MessageResponse; +use anyllm_translate::anthropic::streaming::StreamEvent; +use anyllm_translate::anthropic::MessageCreateRequest; +use anyllm_translate::openai::{ + ChatCompletionChunk, ChatCompletionRequest, ChatCompletionResponse, +}; +use anyllm_translate::{mapping, translate_request, translate_response, TranslationConfig}; +use futures::Stream; +use pin_project_lite::pin_project; + +use crate::error::ClientError; +use crate::http::{build_http_client, HttpClientConfig}; +use crate::rate_limit::RateLimitHeaders; +use crate::retry::{self, RetryableError}; + +/// Authentication for the backend API. +#[derive(Clone, Debug)] +pub enum Auth { + /// Bearer token (e.g., OpenAI API key). + Bearer(String), + /// Custom header (e.g., `x-goog-api-key` for Google). + Header { name: String, value: String }, +} + +/// Configuration for the [`Client`]. +#[derive(Clone, Debug)] +pub struct ClientConfig { + /// URL for the chat completions endpoint (e.g., `https://api.openai.com/v1/chat/completions`). + pub chat_completions_url: String, + /// Authentication credentials. + pub auth: Auth, + /// HTTP client configuration (TLS, timeouts, SSRF protection). + pub http: HttpClientConfig, + /// Translation configuration (model mapping, lossy behavior). + pub translation: TranslationConfig, +} + +impl ClientConfig { + pub fn builder() -> ClientConfigBuilder { + ClientConfigBuilder::default() + } +} + +/// Builder for [`ClientConfig`]. +#[derive(Default)] +pub struct ClientConfigBuilder { + backend_url: String, + auth: Option, + http: Option, + translation: Option, +} + +impl ClientConfigBuilder { + /// Set the chat completions endpoint URL. + /// For OpenAI: `https://api.openai.com/v1/chat/completions` + pub fn backend_url(mut self, url: impl Into) -> Self { + self.backend_url = url.into(); + self + } + + /// Set authentication credentials. + pub fn auth(mut self, auth: Auth) -> Self { + self.auth = Some(auth); + self + } + + /// Set HTTP client configuration. Uses secure defaults if not specified. + pub fn http(mut self, http: HttpClientConfig) -> Self { + self.http = Some(http); + self + } + + /// Set translation configuration. + pub fn translation(mut self, translation: TranslationConfig) -> Self { + self.translation = Some(translation); + self + } + + pub fn build(self) -> ClientConfig { + ClientConfig { + chat_completions_url: self.backend_url, + auth: self.auth.unwrap_or(Auth::Bearer(String::new())), + http: self.http.unwrap_or_default(), + translation: self.translation.unwrap_or_default(), + } + } +} + +/// Internal error type implementing [`RetryableError`] for the generic retry loop. +#[derive(Debug)] +enum InternalError { + Request(reqwest::Error), + ApiError { status: u16, body: String }, +} + +impl RetryableError for InternalError { + fn from_request(e: reqwest::Error) -> Self { + Self::Request(e) + } + fn from_api_response(status: u16, body: &str) -> Self { + Self::ApiError { + status, + body: body.to_string(), + } + } +} + +impl From for ClientError { + fn from(e: InternalError) -> Self { + match e { + InternalError::Request(e) => ClientError::Transport(e), + InternalError::ApiError { status, body } => ClientError::ApiError { + status, + message: format!("Backend returned status {status}"), + body, + }, + } + } +} + +/// Async HTTP client for Anthropic-to-OpenAI translation. +/// +/// Accepts Anthropic Messages API requests, translates to OpenAI format, +/// sends to the configured backend, and translates the response back. +#[derive(Clone)] +pub struct Client { + http: reqwest::Client, + config: ClientConfig, +} + +impl Client { + /// Create a new client from configuration. + pub fn new(config: ClientConfig) -> Self { + let http = build_http_client(&config.http); + Self { http, config } + } + + /// Create from an existing reqwest client and configuration. + /// Useful when you want to share an HTTP client across multiple instances. + pub fn with_http_client(http: reqwest::Client, config: ClientConfig) -> Self { + Self { http, config } + } + + fn auth(&self) -> retry::RequestAuth<'_> { + match &self.config.auth { + Auth::Bearer(token) => retry::RequestAuth::Bearer(token), + Auth::Header { name, value } => retry::RequestAuth::Header { name, value }, + } + } + + /// Send an Anthropic Messages API request and get an Anthropic response. + /// + /// Translates the request to OpenAI format, sends it, and translates the + /// response back. Retries on 429/5xx with exponential backoff. + pub async fn messages( + &self, + req: &MessageCreateRequest, + ) -> Result { + let openai_req = translate_request(req, &self.config.translation)?; + let (resp, _status, _rate_limits) = self.chat_completion(&openai_req).await?; + let anthropic_resp = translate_response(&resp, &req.model); + Ok(anthropic_resp) + } + + /// Send an Anthropic Messages API request and get a stream of Anthropic SSE events. + /// + /// The returned stream yields `StreamEvent` items. Translation happens + /// incrementally as chunks arrive from the backend. + pub async fn messages_stream( + &self, + req: &MessageCreateRequest, + ) -> Result< + ( + impl Stream>, + RateLimitHeaders, + ), + ClientError, + > { + let mut openai_req = translate_request(req, &self.config.translation)?; + openai_req.stream = Some(true); + let (response, rate_limits) = self.chat_completion_stream_raw(&openai_req).await?; + + let model = req.model.clone(); + let stream = SseTranslatingStream::new(response, model); + Ok((stream, rate_limits)) + } + + /// Send a pre-translated OpenAI Chat Completion request. + /// + /// Useful when you want to handle translation yourself and just need the + /// HTTP client with retry logic. + pub async fn chat_completion( + &self, + req: &ChatCompletionRequest, + ) -> Result<(ChatCompletionResponse, u16, RateLimitHeaders), ClientError> { + let response: reqwest::Response = retry::send_with_retry::( + &self.http, + &self.config.chat_completions_url, + &self.auth(), + req, + "backend", + ) + .await + .map_err(ClientError::from)?; + + let status = response.status().as_u16(); + let rate_limits = RateLimitHeaders::from_openai_headers(response.headers()); + let body = response + .json::() + .await + .map_err(|e| ClientError::Deserialization(e.to_string()))?; + Ok((body, status, rate_limits)) + } + + /// Send a streaming Chat Completion request and get the raw response. + async fn chat_completion_stream_raw( + &self, + req: &ChatCompletionRequest, + ) -> Result<(reqwest::Response, RateLimitHeaders), ClientError> { + let response: reqwest::Response = retry::send_with_retry::( + &self.http, + &self.config.chat_completions_url, + &self.auth(), + req, + "backend", + ) + .await + .map_err(ClientError::from)?; + + let rate_limits = RateLimitHeaders::from_openai_headers(response.headers()); + Ok((response, rate_limits)) + } +} + +// -- Streaming implementation -- + +pin_project! { + /// A stream that reads SSE frames from a reqwest response, translates + /// OpenAI chunks to Anthropic StreamEvents, and yields them. + struct SseTranslatingStream { + #[pin] + inner: futures::channel::mpsc::Receiver>, + } +} + +impl SseTranslatingStream { + fn new(response: reqwest::Response, model: String) -> Self { + let (mut tx, rx) = futures::channel::mpsc::channel(32); + + // Spawn a task to read SSE frames and translate them. + tokio::spawn(async move { + let mut translator = mapping::streaming_map::StreamingTranslator::new(model); + let mut done = false; + + let result = crate::sse::read_sse_stream( + response, + |json_str| { + if json_str == "[DONE]" { + done = true; + return Some(translator.finish()); + } + match serde_json::from_str::(json_str) { + Ok(chunk) => Some(translator.process_chunk(&chunk)), + Err(e) => { + tracing::debug!("failed to parse streaming chunk: {e}"); + None + } + } + }, + |events| { + for event in events { + // Block on send; if receiver is dropped, stop. + if tx.try_send(Ok(event.clone())).is_err() { + return false; + } + } + true + }, + ) + .await; + + if let Err(e) = result { + let _ = tx.try_send(Err(ClientError::Sse(e))); + } else if !done { + // Stream ended without [DONE]; flush remaining events. + let events = translator.finish(); + for event in events { + if tx.try_send(Ok(event)).is_err() { + break; + } + } + } + }); + + Self { inner: rx } + } +} + +impl Stream for SseTranslatingStream { + type Item = Result; + + fn poll_next( + self: std::pin::Pin<&mut Self>, + cx: &mut std::task::Context<'_>, + ) -> std::task::Poll> { + self.project().inner.poll_next(cx) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn client_config_builder_defaults() { + let config = ClientConfig::builder() + .backend_url("https://api.openai.com/v1/chat/completions") + .auth(Auth::Bearer("sk-test".into())) + .build(); + + assert_eq!( + config.chat_completions_url, + "https://api.openai.com/v1/chat/completions" + ); + assert!(matches!(config.auth, Auth::Bearer(ref s) if s == "sk-test")); + } + + #[test] + fn client_config_builder_with_translation() { + let translation = TranslationConfig::builder() + .model_map("haiku", "gpt-4o-mini") + .model_map("sonnet", "gpt-4o") + .build(); + + let config = ClientConfig::builder() + .backend_url("https://api.openai.com/v1/chat/completions") + .auth(Auth::Bearer("sk-test".into())) + .translation(translation) + .build(); + + assert!(config.translation.map_model("claude-3-haiku").is_ok()); + } + + #[test] + fn client_creates_without_panic() { + let config = ClientConfig::builder() + .backend_url("https://api.openai.com/v1/chat/completions") + .auth(Auth::Bearer("sk-test".into())) + .http(HttpClientConfig { + ssrf_protection: false, + ..Default::default() + }) + .build(); + + let _client = Client::new(config); + } +} diff --git a/crates/client/src/error.rs b/crates/client/src/error.rs new file mode 100644 index 0000000..6f753aa --- /dev/null +++ b/crates/client/src/error.rs @@ -0,0 +1,41 @@ +//! Error types for the client crate. + +use anyllm_translate::TranslateError; + +/// Errors from the high-level [`Client`](crate::Client). +#[derive(Debug, thiserror::Error)] +pub enum ClientError { + /// Translation failed (e.g., unsupported feature with `LossyBehavior::Error`). + #[error("translation error: {0}")] + Translation(#[from] TranslateError), + + /// HTTP transport failure (DNS, TLS, connection refused, timeout). + #[error("request failed: {0}")] + Transport(#[from] reqwest::Error), + + /// Backend returned a non-2xx status with an error body. + #[error("API error ({status}): {message}")] + ApiError { + status: u16, + message: String, + body: String, + }, + + /// Response body could not be deserialized. + #[error("response deserialization failed: {0}")] + Deserialization(String), + + /// SSE stream error. + #[error("SSE stream error: {0}")] + Sse(#[from] crate::sse::SseError), +} + +impl ClientError { + /// HTTP status code for API errors, or 500 for transport/deserialization errors. + pub fn status_code(&self) -> u16 { + match self { + Self::ApiError { status, .. } => *status, + _ => 500, + } + } +} diff --git a/crates/client/src/http.rs b/crates/client/src/http.rs new file mode 100644 index 0000000..c47261b --- /dev/null +++ b/crates/client/src/http.rs @@ -0,0 +1,207 @@ +//! HTTP client builder with optional mTLS, custom CA, and SSRF-safe DNS resolution. + +use reqwest::Client; +use std::net::IpAddr; +use std::time::Duration; + +/// Configuration for building an HTTP client. +#[derive(Clone, Debug, Default)] +pub struct HttpClientConfig { + /// PKCS#12 identity bytes and password for mTLS. + pub p12_identity: Option<(Vec, String)>, + /// PEM-encoded CA certificate for verifying the backend server. + pub ca_cert_pem: Option>, + /// Connection timeout (default: 10s). + pub connect_timeout: Option, + /// Read timeout (default: 900s, generous for reasoning models). + pub read_timeout: Option, + /// TCP keepalive interval (default: 60s). + pub tcp_keepalive: Option, + /// Enable SSRF-safe DNS resolution (default: true when `ssrf-protection` feature enabled). + pub ssrf_protection: bool, +} + +impl HttpClientConfig { + pub fn new() -> Self { + Self { + ssrf_protection: cfg!(feature = "ssrf-protection"), + ..Default::default() + } + } +} + +/// Build a reqwest HTTP client from configuration. +/// +/// Includes hardened defaults: 10s connect timeout, 900s read timeout (for slow +/// reasoning models like o1/o3), 60s TCP keepalive, and SSRF-safe DNS resolution. +pub fn build_http_client(config: &HttpClientConfig) -> Client { + let mut builder = Client::builder(); + + if let Some((ref p12_bytes, ref password)) = config.p12_identity { + let identity = reqwest::Identity::from_pkcs12_der(p12_bytes, password) + .expect("P12 identity was validated at startup"); + builder = builder.identity(identity); + } + + if let Some(ref ca_pem) = config.ca_cert_pem { + let cert = + reqwest::Certificate::from_pem(ca_pem).expect("CA cert was validated at startup"); + builder = builder.add_root_certificate(cert); + } + + let connect_timeout = config.connect_timeout.unwrap_or(Duration::from_secs(10)); + let read_timeout = config.read_timeout.unwrap_or(Duration::from_secs(900)); + let tcp_keepalive = config.tcp_keepalive.unwrap_or(Duration::from_secs(60)); + + builder = builder + .connect_timeout(connect_timeout) + .read_timeout(read_timeout) + .tcp_keepalive(tcp_keepalive); + + #[cfg(feature = "ssrf-protection")] + if config.ssrf_protection { + builder = builder.dns_resolver(std::sync::Arc::new(SsrfSafeDnsResolver)); + } + + builder.build().expect("failed to build HTTP client") +} + +/// DNS resolver that rejects private/loopback IPs at connection time, +/// preventing DNS rebinding attacks where a domain resolves to a public IP +/// at startup validation but later resolves to a private/metadata IP. +#[cfg(feature = "ssrf-protection")] +struct SsrfSafeDnsResolver; + +#[cfg(feature = "ssrf-protection")] +impl reqwest::dns::Resolve for SsrfSafeDnsResolver { + fn resolve(&self, name: reqwest::dns::Name) -> reqwest::dns::Resolving { + Box::pin(async move { + let name_str = name.as_str().to_string(); + // DNS resolution (ToSocketAddrs) blocks the calling thread. + // Must run on the blocking threadpool to avoid stalling the + // async runtime and all other in-flight requests. + let addrs: Vec = + tokio::task::spawn_blocking(move || -> Result, _> { + use std::net::ToSocketAddrs; + // Port 0 is a placeholder; reqwest replaces it with the actual port. + let lookup = format!("{name_str}:0"); + Ok(lookup.to_socket_addrs()?.collect()) + }) + .await + .map_err(|e| -> Box { Box::new(e) })? + .map_err( + |e: std::io::Error| -> Box { Box::new(e) }, + )?; + + // Filter out private/loopback IPs to prevent SSRF attacks where + // an attacker-controlled DNS record resolves to internal endpoints + // (e.g., cloud metadata at 169.254.169.254). + let safe: Vec = addrs + .into_iter() + .filter(|addr| !is_private_ip(addr.ip())) + .collect(); + + if safe.is_empty() { + return Err(Box::new(std::io::Error::new( + std::io::ErrorKind::PermissionDenied, + "DNS resolved only to private/loopback IPs (SSRF blocked)".to_string(), + )) + as Box); + } + + Ok(Box::new(safe.into_iter()) as Box + Send>) + }) + } +} + +/// Returns true for loopback, private (RFC 1918), link-local, and +/// cloud metadata IPs (169.254.169.254). +pub fn is_private_ip(ip: IpAddr) -> bool { + match ip { + IpAddr::V4(v4) => { + v4.is_loopback() + || v4.is_private() + || v4.is_link_local() + || v4.is_broadcast() + || v4.is_unspecified() + // AWS/GCP/Azure metadata endpoint. SSRF to this IP lets + // attackers exfiltrate instance credentials. + || v4 == std::net::Ipv4Addr::new(169, 254, 169, 254) + } + IpAddr::V6(v6) => { + v6.is_loopback() || v6.is_unspecified() + // Check IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) recursively; + // attackers can bypass IPv4 checks using the mapped representation. + || matches!(v6.to_ipv4_mapped(), Some(v4) if is_private_ip(IpAddr::V4(v4))) + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn private_ipv4_loopback() { + assert!(is_private_ip("127.0.0.1".parse().unwrap())); + } + + #[test] + fn private_ipv4_rfc1918() { + assert!(is_private_ip("10.0.0.1".parse().unwrap())); + assert!(is_private_ip("172.16.0.1".parse().unwrap())); + assert!(is_private_ip("192.168.1.1".parse().unwrap())); + } + + #[test] + fn private_ipv4_link_local() { + assert!(is_private_ip("169.254.1.1".parse().unwrap())); + } + + #[test] + fn private_ipv4_metadata() { + assert!(is_private_ip("169.254.169.254".parse().unwrap())); + } + + #[test] + fn private_ipv4_unspecified() { + assert!(is_private_ip("0.0.0.0".parse().unwrap())); + } + + #[test] + fn public_ipv4() { + assert!(!is_private_ip("8.8.8.8".parse().unwrap())); + assert!(!is_private_ip("1.1.1.1".parse().unwrap())); + } + + #[test] + fn private_ipv6_loopback() { + assert!(is_private_ip("::1".parse().unwrap())); + } + + #[test] + fn private_ipv6_mapped_private() { + // ::ffff:192.168.1.1 + assert!(is_private_ip("::ffff:192.168.1.1".parse().unwrap())); + } + + #[test] + fn public_ipv6() { + assert!(!is_private_ip("2001:4860:4860::8888".parse().unwrap())); + } + + #[test] + fn default_config_has_ssrf_protection() { + let config = HttpClientConfig::new(); + assert_eq!(config.ssrf_protection, cfg!(feature = "ssrf-protection")); + } + + #[test] + fn build_client_default_config() { + let config = HttpClientConfig { + ssrf_protection: false, // avoid DNS in tests + ..Default::default() + }; + let _client = build_http_client(&config); + } +} diff --git a/crates/client/src/lib.rs b/crates/client/src/lib.rs new file mode 100644 index 0000000..eeec0d1 --- /dev/null +++ b/crates/client/src/lib.rs @@ -0,0 +1,65 @@ +//! # anyllm_client +//! +//! Async HTTP client for Anthropic-to-OpenAI API translation. +//! +//! Accepts Anthropic Messages API requests, translates them to OpenAI Chat Completions +//! format, sends them to an OpenAI-compatible backend, and translates the response back. +//! Supports non-streaming and streaming (SSE) modes, retry with exponential backoff, +//! SSRF-safe DNS resolution, and mTLS. +//! +//! # Quick start +//! +//! ```rust,no_run +//! use anyllm_client::{Client, ClientConfig, Auth}; +//! use anyllm_translate::TranslationConfig; +//! use anyllm_translate::anthropic::MessageCreateRequest; +//! +//! # async fn example() -> Result<(), anyllm_client::ClientError> { +//! let config = ClientConfig::builder() +//! .backend_url("https://api.openai.com/v1/chat/completions") +//! .auth(Auth::Bearer("sk-...".into())) +//! .translation( +//! TranslationConfig::builder() +//! .model_map("haiku", "gpt-4o-mini") +//! .model_map("sonnet", "gpt-4o") +//! .build() +//! ) +//! .build(); +//! +//! let client = Client::new(config); +//! +//! let req: MessageCreateRequest = serde_json::from_str(r#"{ +//! "model": "claude-sonnet-4-6", +//! "max_tokens": 100, +//! "messages": [{"role": "user", "content": "Hello"}] +//! }"#).unwrap(); +//! +//! let response = client.messages(&req).await?; +//! println!("{:?}", response); +//! # Ok(()) +//! # } +//! ``` +//! +//! # Modules +//! +//! - [`client`] -- High-level `Client` for Anthropic-in, Anthropic-out API calls +//! - [`http`] -- HTTP client builder with TLS and SSRF protection +//! - [`retry`] -- Generic retry logic with exponential backoff +//! - [`rate_limit`] -- Rate limit header extraction and format conversion +//! - [`sse`] -- Framework-agnostic SSE frame parser +//! - [`error`] -- Error types + +pub mod client; +pub mod error; +pub mod http; +pub mod rate_limit; +pub mod retry; +pub mod sse; + +// Convenience re-exports +pub use client::{Auth, Client, ClientConfig, ClientConfigBuilder}; +pub use error::ClientError; +pub use http::{build_http_client, HttpClientConfig}; +pub use rate_limit::RateLimitHeaders; +pub use retry::{backoff_delay, is_retryable, parse_retry_after, send_with_retry, RetryableError}; +pub use sse::{find_double_newline, SseError}; diff --git a/crates/client/src/rate_limit.rs b/crates/client/src/rate_limit.rs new file mode 100644 index 0000000..5db15b8 --- /dev/null +++ b/crates/client/src/rate_limit.rs @@ -0,0 +1,413 @@ +//! Rate limit header extraction, format conversion, and duration parsing. +//! +//! Converts between OpenAI-style `x-ratelimit-*` headers and Anthropic-style +//! `anthropic-ratelimit-*` headers. OpenAI uses relative durations ("1s", "500ms") +//! for reset fields; Anthropic uses ISO 8601 UTC timestamps. + +use std::time::Duration; + +/// Rate limit headers extracted from backend responses. +/// Forwarded to clients as Anthropic-style `anthropic-ratelimit-*` headers. +/// See: +#[derive(Debug, Default, Clone)] +pub struct RateLimitHeaders { + /// Maximum requests allowed in the current window. + pub requests_limit: Option, + /// Requests remaining before rate limiting kicks in. + pub requests_remaining: Option, + /// Reset value for request limits (raw from backend). + pub requests_reset: Option, + /// Maximum tokens allowed in the current window. + pub tokens_limit: Option, + /// Tokens remaining before rate limiting kicks in. + pub tokens_remaining: Option, + /// Reset value for token limits (raw from backend). + pub tokens_reset: Option, + /// Seconds to wait before retrying (from `retry-after` header on 429s). + pub retry_after: Option, +} + +/// Extract a header value as a trimmed string. +fn header_str(headers: &reqwest::header::HeaderMap, name: &str) -> Option { + headers + .get(name) + .and_then(|v| v.to_str().ok()) + .map(|s| s.trim().to_string()) +} + +impl RateLimitHeaders { + /// Extract rate limit headers from an OpenAI (or Vertex) response. + pub fn from_openai_headers(headers: &reqwest::header::HeaderMap) -> Self { + Self { + requests_limit: header_str(headers, "x-ratelimit-limit-requests"), + requests_remaining: header_str(headers, "x-ratelimit-remaining-requests"), + requests_reset: header_str(headers, "x-ratelimit-reset-requests"), + tokens_limit: header_str(headers, "x-ratelimit-limit-tokens"), + tokens_remaining: header_str(headers, "x-ratelimit-remaining-tokens"), + tokens_reset: header_str(headers, "x-ratelimit-reset-tokens"), + retry_after: header_str(headers, "retry-after"), + } + } + + /// Extract rate limit headers from an Anthropic response. + /// Anthropic uses `anthropic-ratelimit-*` headers natively. + pub fn from_anthropic_headers(headers: &reqwest::header::HeaderMap) -> Self { + Self { + requests_limit: header_str(headers, "anthropic-ratelimit-requests-limit"), + requests_remaining: header_str(headers, "anthropic-ratelimit-requests-remaining"), + requests_reset: header_str(headers, "anthropic-ratelimit-requests-reset"), + tokens_limit: header_str(headers, "anthropic-ratelimit-tokens-limit"), + tokens_remaining: header_str(headers, "anthropic-ratelimit-tokens-remaining"), + tokens_reset: header_str(headers, "anthropic-ratelimit-tokens-reset"), + retry_after: header_str(headers, "retry-after"), + } + } + + /// Inject Anthropic-format response headers (rate limits + version) into an + /// `axum::http::HeaderMap`. The `*_reset` fields are converted from OpenAI's + /// relative duration format (e.g., "1s") to Anthropic's ISO 8601 UTC timestamp. + /// Falls back to the raw value with a warning if parsing fails. + /// + /// This method accepts generic `http::HeaderMap` (used by both reqwest and axum). + pub fn inject_anthropic_response_headers(&self, map: &mut http_types::HeaderMap) { + set_if_some( + map, + "anthropic-ratelimit-requests-limit", + &self.requests_limit, + ); + set_if_some( + map, + "anthropic-ratelimit-requests-remaining", + &self.requests_remaining, + ); + let req_reset = convert_reset_duration(&self.requests_reset, "requests_reset"); + set_if_some(map, "anthropic-ratelimit-requests-reset", &req_reset); + set_if_some(map, "anthropic-ratelimit-tokens-limit", &self.tokens_limit); + set_if_some( + map, + "anthropic-ratelimit-tokens-remaining", + &self.tokens_remaining, + ); + let tok_reset = convert_reset_duration(&self.tokens_reset, "tokens_reset"); + set_if_some(map, "anthropic-ratelimit-tokens-reset", &tok_reset); + set_if_some(map, "retry-after", &self.retry_after); + map.insert( + http_types::HeaderName::from_static("anthropic-version"), + http_types::HeaderValue::from_static("2023-06-01"), + ); + } +} + +// Use the http crate types that reqwest re-exports, avoiding an extra dependency. +mod http_types { + pub use reqwest::header::{HeaderMap, HeaderName, HeaderValue}; +} + +/// Set a header on a HeaderMap if the value is Some. +fn set_if_some(map: &mut http_types::HeaderMap, name: &str, value: &Option) { + if let Some(v) = value { + if let (Ok(header_name), Ok(header_value)) = ( + http_types::HeaderName::from_bytes(name.as_bytes()), + http_types::HeaderValue::from_str(v), + ) { + map.insert(header_name, header_value); + } + } +} + +/// Convert an OpenAI relative duration to ISO 8601, falling back to the raw +/// value with a warning if parsing fails. +fn convert_reset_duration(raw: &Option, field: &str) -> Option { + raw.as_deref().map(|v| { + openai_duration_to_iso8601(v).unwrap_or_else(|| { + tracing::warn!( + value = v, + field, + "failed to parse reset duration, forwarding raw" + ); + v.to_string() + }) + }) +} + +/// Convert an OpenAI relative duration string to an ISO 8601 UTC timestamp. +fn openai_duration_to_iso8601(s: &str) -> Option { + openai_duration_to_iso8601_at(s, std::time::SystemTime::now()) +} + +/// Convert an OpenAI relative duration string to an ISO 8601 UTC timestamp +/// by adding it to the given anchor time. Testable variant. +pub fn openai_duration_to_iso8601_at(s: &str, anchor: std::time::SystemTime) -> Option { + let dur = parse_openai_duration(s)?; + let reset_time = anchor + dur; + let secs = reset_time + .duration_since(std::time::UNIX_EPOCH) + .ok()? + .as_secs(); + Some(epoch_to_iso8601(secs)) +} + +/// Parse OpenAI's duration format (e.g., "6ms", "1s", "1m30s", "2m") into a +/// [`Duration`]. Returns `None` for unrecognized formats. +pub fn parse_openai_duration(s: &str) -> Option { + let s = s.trim(); + if s.is_empty() { + return None; + } + let mut total_ms: u64 = 0; + let mut num_start: Option = None; + let bytes = s.as_bytes(); + let mut i = 0; + while i < bytes.len() { + let c = bytes[i]; + if c.is_ascii_digit() || c == b'.' { + if num_start.is_none() { + num_start = Some(i); + } + i += 1; + } else if c.is_ascii_alphabetic() { + let start = num_start?; + let num_str = &s[start..i]; + let unit_start = i; + while i < bytes.len() && bytes[i].is_ascii_alphabetic() { + i += 1; + } + let unit = &s[unit_start..i]; + let value: f64 = num_str.parse().ok()?; + let ms = match unit { + "ms" => value, + "s" => value * 1_000.0, + "m" => value * 60_000.0, + "h" => value * 3_600_000.0, + _ => return None, + }; + total_ms += ms.round() as u64; + num_start = None; + } else { + return None; + } + } + // Trailing number with no unit is invalid + if num_start.is_some() { + return None; + } + Some(Duration::from_millis(total_ms)) +} + +/// Convert epoch seconds to ISO 8601 UTC string (e.g., "2025-06-16T12:00:01Z"). +pub fn epoch_to_iso8601(epoch: u64) -> String { + let secs = epoch; + let days = secs / 86400; + let time_of_day = secs % 86400; + let hours = time_of_day / 3600; + let minutes = (time_of_day % 3600) / 60; + let seconds = time_of_day % 60; + + let (year, month, day) = days_to_ymd(days); + + format!( + "{:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z", + year, month, day, hours, minutes, seconds + ) +} + +/// Days since 1970-01-01 to (year, month, day). +/// Algorithm from +fn days_to_ymd(days: u64) -> (u64, u64, u64) { + let z = days + 719468; + let era = z / 146097; + let doe = z - era * 146097; + let yoe = (doe - doe / 1460 + doe / 36524 - doe / 146096) / 365; + let y = yoe + era * 400; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let d = doy - (153 * mp + 2) / 5 + 1; + let m = if mp < 10 { mp + 3 } else { mp - 9 }; + let y = if m <= 2 { y + 1 } else { y }; + (y, m, d) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn from_openai_headers_extracts_all() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert("x-ratelimit-limit-requests", "100".parse().unwrap()); + headers.insert("x-ratelimit-remaining-requests", "99".parse().unwrap()); + headers.insert("x-ratelimit-reset-requests", "1s".parse().unwrap()); + headers.insert("x-ratelimit-limit-tokens", "40000".parse().unwrap()); + headers.insert("x-ratelimit-remaining-tokens", "39500".parse().unwrap()); + headers.insert("x-ratelimit-reset-tokens", "500ms".parse().unwrap()); + headers.insert("retry-after", "2".parse().unwrap()); + + let rl = RateLimitHeaders::from_openai_headers(&headers); + assert_eq!(rl.requests_limit.as_deref(), Some("100")); + assert_eq!(rl.requests_remaining.as_deref(), Some("99")); + assert_eq!(rl.requests_reset.as_deref(), Some("1s")); + assert_eq!(rl.tokens_limit.as_deref(), Some("40000")); + assert_eq!(rl.tokens_remaining.as_deref(), Some("39500")); + assert_eq!(rl.tokens_reset.as_deref(), Some("500ms")); + assert_eq!(rl.retry_after.as_deref(), Some("2")); + } + + #[test] + fn from_openai_headers_missing_are_none() { + let headers = reqwest::header::HeaderMap::new(); + let rl = RateLimitHeaders::from_openai_headers(&headers); + assert!(rl.requests_limit.is_none()); + assert!(rl.requests_remaining.is_none()); + assert!(rl.requests_reset.is_none()); + assert!(rl.tokens_limit.is_none()); + assert!(rl.tokens_remaining.is_none()); + assert!(rl.tokens_reset.is_none()); + assert!(rl.retry_after.is_none()); + } + + #[test] + fn inject_anthropic_response_headers_sets_values() { + let rl = RateLimitHeaders { + requests_limit: Some("100".into()), + tokens_remaining: Some("39500".into()), + retry_after: Some("3".into()), + ..Default::default() + }; + let mut map = reqwest::header::HeaderMap::new(); + rl.inject_anthropic_response_headers(&mut map); + + assert_eq!( + map.get("anthropic-ratelimit-requests-limit").unwrap(), + "100" + ); + assert_eq!( + map.get("anthropic-ratelimit-tokens-remaining").unwrap(), + "39500" + ); + assert_eq!(map.get("retry-after").unwrap(), "3"); + assert_eq!(map.get("anthropic-version").unwrap(), "2023-06-01"); + assert!(map.get("anthropic-ratelimit-requests-remaining").is_none()); + assert!(map.get("anthropic-ratelimit-tokens-limit").is_none()); + } + + #[test] + fn inject_anthropic_response_headers_default_sets_version_only() { + let rl = RateLimitHeaders::default(); + let mut map = reqwest::header::HeaderMap::new(); + rl.inject_anthropic_response_headers(&mut map); + assert_eq!(map.len(), 1); + assert_eq!(map.get("anthropic-version").unwrap(), "2023-06-01"); + } + + #[test] + fn inject_anthropic_response_headers_converts_reset_to_iso8601() { + let rl = RateLimitHeaders { + requests_reset: Some("1s".into()), + tokens_reset: Some("500ms".into()), + ..Default::default() + }; + let mut map = reqwest::header::HeaderMap::new(); + rl.inject_anthropic_response_headers(&mut map); + + let req_reset = map + .get("anthropic-ratelimit-requests-reset") + .unwrap() + .to_str() + .unwrap(); + let tok_reset = map + .get("anthropic-ratelimit-tokens-reset") + .unwrap() + .to_str() + .unwrap(); + assert!( + req_reset.contains('T') && req_reset.ends_with('Z'), + "expected ISO 8601 timestamp, got: {req_reset}" + ); + assert!( + tok_reset.contains('T') && tok_reset.ends_with('Z'), + "expected ISO 8601 timestamp, got: {tok_reset}" + ); + } + + #[test] + fn parse_openai_duration_various_formats() { + assert_eq!(parse_openai_duration("6ms"), Some(Duration::from_millis(6))); + assert_eq!( + parse_openai_duration("1s"), + Some(Duration::from_millis(1000)) + ); + assert_eq!( + parse_openai_duration("2m"), + Some(Duration::from_millis(120_000)) + ); + assert_eq!( + parse_openai_duration("1m30s"), + Some(Duration::from_millis(90_000)) + ); + assert_eq!( + parse_openai_duration("1h"), + Some(Duration::from_millis(3_600_000)) + ); + assert_eq!( + parse_openai_duration("1h30m"), + Some(Duration::from_millis(5_400_000)) + ); + } + + #[test] + fn parse_openai_duration_invalid() { + assert_eq!(parse_openai_duration(""), None); + assert_eq!(parse_openai_duration("abc"), None); + assert_eq!(parse_openai_duration("123"), None); + assert_eq!(parse_openai_duration("1x"), None); + } + + #[test] + fn openai_duration_to_iso8601_at_pinned_time() { + let anchor = std::time::UNIX_EPOCH + Duration::from_secs(1_750_075_200); + assert_eq!( + openai_duration_to_iso8601_at("1s", anchor).unwrap(), + "2025-06-16T12:00:01Z" + ); + assert_eq!( + openai_duration_to_iso8601_at("1m30s", anchor).unwrap(), + "2025-06-16T12:01:30Z" + ); + assert_eq!( + openai_duration_to_iso8601_at("500ms", anchor).unwrap(), + "2025-06-16T12:00:00Z" + ); + } + + #[test] + fn openai_duration_to_iso8601_invalid_returns_none() { + assert!(openai_duration_to_iso8601("garbage").is_none()); + assert!(openai_duration_to_iso8601("").is_none()); + } + + #[test] + fn epoch_to_iso8601_unix_epoch() { + assert_eq!(epoch_to_iso8601(0), "1970-01-01T00:00:00Z"); + } + + #[test] + fn from_anthropic_headers_extracts_all() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert("anthropic-ratelimit-requests-limit", "100".parse().unwrap()); + headers.insert( + "anthropic-ratelimit-requests-remaining", + "99".parse().unwrap(), + ); + headers.insert( + "anthropic-ratelimit-requests-reset", + "2025-01-01T00:00:00Z".parse().unwrap(), + ); + headers.insert("retry-after", "5".parse().unwrap()); + + let rl = RateLimitHeaders::from_anthropic_headers(&headers); + assert_eq!(rl.requests_limit.as_deref(), Some("100")); + assert_eq!(rl.requests_remaining.as_deref(), Some("99")); + assert_eq!(rl.requests_reset.as_deref(), Some("2025-01-01T00:00:00Z")); + assert_eq!(rl.retry_after.as_deref(), Some("5")); + } +} diff --git a/crates/client/src/retry.rs b/crates/client/src/retry.rs new file mode 100644 index 0000000..7093e4d --- /dev/null +++ b/crates/client/src/retry.rs @@ -0,0 +1,198 @@ +//! Generic retry logic with exponential backoff and jitter. + +use reqwest::Client; +use serde::Serialize; +use std::time::Duration; +use tokio::time::sleep; + +/// Default maximum number of retries. +pub const MAX_RETRIES: u32 = 3; + +/// Default base delay between retries in milliseconds. +pub const BASE_DELAY_MS: u64 = 500; + +/// Backend error types implement this to enable the generic [`send_with_retry`]. +pub trait RetryableError: Sized { + fn from_request(e: reqwest::Error) -> Self; + fn from_api_response(status: u16, body: &str) -> Self; +} + +/// Authentication to apply to outgoing requests. +#[derive(Clone, Debug)] +pub enum RequestAuth<'a> { + Bearer(&'a str), + Header { name: &'a str, value: &'a str }, +} + +fn apply_auth(rb: reqwest::RequestBuilder, auth: &RequestAuth<'_>) -> reqwest::RequestBuilder { + match auth { + RequestAuth::Bearer(token) => rb.bearer_auth(token), + RequestAuth::Header { name, value } => rb.header(*name, *value), + } +} + +/// Send a POST request with retry on 429/5xx. Returns the raw successful response. +pub async fn send_with_retry( + client: &Client, + url: &str, + auth: &RequestAuth<'_>, + body: &impl Serialize, + label: &str, +) -> Result { + for attempt in 0..=MAX_RETRIES { + let rb = apply_auth(client.post(url).json(body), auth); + let response = rb.send().await.map_err(E::from_request)?; + let status = response.status().as_u16(); + + if (200..300).contains(&status) { + return Ok(response); + } + + if attempt < MAX_RETRIES && is_retryable(status) { + let retry_after = parse_retry_after(response.headers()); + let delay = backoff_delay(attempt, retry_after); + tracing::warn!( + status, + attempt = attempt + 1, + max_retries = MAX_RETRIES, + delay_ms = delay.as_millis() as u64, + "retryable error from {label}, backing off" + ); + // Drain the response body before retrying so the HTTP connection + // returns to the pool. Leaving it unread causes connection leaks. + drop(response.bytes().await); + sleep(delay).await; + continue; + } + + let text = response.text().await.unwrap_or_else(|e| { + tracing::warn!("failed to read error response body: {e}"); + String::new() + }); + return Err(E::from_api_response(status, &text)); + } + + unreachable!("loop runs MAX_RETRIES+1 times and always returns") +} + +/// Check if a status code is retryable (408, 429, or 5xx). +pub fn is_retryable(status: u16) -> bool { + status == 408 || status == 429 || (500..=599).contains(&status) +} + +/// Parse retry-after header as integer seconds or HTTP date (RFC 7231). +pub fn parse_retry_after(headers: &reqwest::header::HeaderMap) -> Option { + let value = headers + .get("retry-after") + .and_then(|v| v.to_str().ok()) + .map(|s| s.trim().to_string())?; + // Fast path: integer seconds + if let Ok(secs) = value.parse::() { + return Some(Duration::from_secs(secs)); + } + // HTTP date (RFC 7231). Past dates return None (no wait needed). + let date = httpdate::parse_http_date(&value).ok()?; + date.duration_since(std::time::SystemTime::now()).ok() +} + +/// Compute backoff delay with jitter. +/// +/// Uses deterministic 25% jitter (upper bound, not random) to keep tests +/// predictable while still spreading retry storms across backends. +pub fn backoff_delay(attempt: u32, retry_after: Option) -> Duration { + if let Some(ra) = retry_after { + return ra; + } + let base = Duration::from_millis(BASE_DELAY_MS * 2u64.pow(attempt)); + let jitter_ms = (base.as_millis() as u64) / 4; + base + Duration::from_millis(jitter_ms) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn is_retryable_429() { + assert!(is_retryable(429)); + } + + #[test] + fn is_retryable_5xx() { + assert!(is_retryable(500)); + assert!(is_retryable(502)); + assert!(is_retryable(503)); + assert!(is_retryable(599)); + } + + #[test] + fn is_retryable_408() { + assert!(is_retryable(408)); + } + + #[test] + fn is_not_retryable_4xx() { + assert!(!is_retryable(400)); + assert!(!is_retryable(401)); + assert!(!is_retryable(404)); + assert!(!is_retryable(409)); + } + + #[test] + fn backoff_respects_retry_after() { + let delay = backoff_delay(0, Some(Duration::from_secs(5))); + assert_eq!(delay, Duration::from_secs(5)); + } + + #[test] + fn backoff_increases_with_attempt() { + let d0 = backoff_delay(0, None); + let d1 = backoff_delay(1, None); + let d2 = backoff_delay(2, None); + assert!(d1 > d0); + assert!(d2 > d1); + } + + #[test] + fn parse_retry_after_valid() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert("retry-after", "3".parse().unwrap()); + let dur = parse_retry_after(&headers); + assert_eq!(dur, Some(Duration::from_secs(3))); + } + + #[test] + fn parse_retry_after_missing() { + let headers = reqwest::header::HeaderMap::new(); + assert_eq!(parse_retry_after(&headers), None); + } + + #[test] + fn parse_retry_after_http_date_future() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert( + "retry-after", + "Wed, 21 Oct 2037 07:28:00 GMT".parse().unwrap(), + ); + let dur = parse_retry_after(&headers); + assert!(dur.is_some(), "future HTTP date should parse to Some"); + assert!(dur.unwrap().as_secs() > 0); + } + + #[test] + fn parse_retry_after_http_date_past() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert( + "retry-after", + "Mon, 01 Jan 2024 00:00:00 GMT".parse().unwrap(), + ); + assert_eq!(parse_retry_after(&headers), None); + } + + #[test] + fn parse_retry_after_garbage() { + let mut headers = reqwest::header::HeaderMap::new(); + headers.insert("retry-after", "not-a-date-or-number".parse().unwrap()); + assert_eq!(parse_retry_after(&headers), None); + } +} diff --git a/crates/client/src/sse.rs b/crates/client/src/sse.rs new file mode 100644 index 0000000..3a9ee09 --- /dev/null +++ b/crates/client/src/sse.rs @@ -0,0 +1,160 @@ +//! Framework-agnostic SSE frame parser. +//! +//! Reads raw bytes from a `reqwest::Response` stream, splits on SSE frame +//! boundaries (`\n\n` or `\r\n\r\n`), and delivers each `data:` line to a +//! caller-supplied callback. No dependency on axum or any web framework. + +use bytes::BytesMut; + +/// Maximum SSE buffer size (10 MB). Protects against unbounded memory growth +/// if the backend sends data without frame delimiters. +pub const MAX_SSE_BUFFER_SIZE: usize = 10 * 1024 * 1024; + +/// Errors from SSE stream parsing. +#[derive(Debug, thiserror::Error)] +pub enum SseError { + #[error("stream read error: {0}")] + ReadError(#[from] reqwest::Error), + #[error("SSE buffer exceeded maximum size ({MAX_SSE_BUFFER_SIZE} bytes)")] + BufferOverflow, +} + +/// Find the first SSE frame boundary (`\n\n` or `\r\n\r\n`) in a byte slice, +/// starting the search at `start`. Returns `(position, delimiter_length)` so +/// the caller can skip the full delimiter. +pub fn find_double_newline(buf: &[u8], start: usize) -> Option<(usize, usize)> { + let len = buf.len(); + let mut i = start; + while i < len.saturating_sub(1) { + if buf[i] == b'\n' && buf[i + 1] == b'\n' { + return Some((i, 2)); + } + if buf[i] == b'\r' + && i + 3 < len + && buf[i + 1] == b'\n' + && buf[i + 2] == b'\r' + && buf[i + 3] == b'\n' + { + return Some((i, 4)); + } + i += 1; + } + None +} + +/// Read SSE frames from a response stream, calling `on_data` for each `data:` line. +/// +/// Returns `Ok(())` on normal stream completion, or an `SseError` on failure. +/// The `on_data` callback receives the JSON string after `data: ` and returns +/// an optional list of translated events. The `on_events` callback is called +/// with each batch of events from a complete SSE frame. +/// +/// This is the framework-agnostic core of SSE parsing. It does not depend on +/// axum, tokio channels, or any specific event type. +pub async fn read_sse_stream( + response: reqwest::Response, + mut on_data: F, + mut on_events: G, +) -> Result<(), SseError> +where + F: FnMut(&str) -> Option>, + G: FnMut(&[T]) -> bool, // returns false if consumer disconnected +{ + use futures::StreamExt; + let mut stream = response.bytes_stream(); + // BytesMut (not String) because TCP chunks may split mid-UTF-8 character. + let mut buffer = BytesMut::new(); + let mut frame_events: Vec = Vec::new(); + let mut search_from: usize = 0; + + while let Some(chunk_result) = stream.next().await { + let bytes = chunk_result?; + buffer.extend_from_slice(&bytes); + + if buffer.len() > MAX_SSE_BUFFER_SIZE { + return Err(SseError::BufferOverflow); + } + + while let Some((pos, delim_len)) = find_double_newline(&buffer, search_from) { + frame_events.clear(); + match std::str::from_utf8(&buffer[..pos]) { + Ok(frame_str) => { + for line in frame_str.lines() { + let line = line.trim(); + if let Some(json_str) = line.strip_prefix("data: ") { + if let Some(mut events) = on_data(json_str) { + frame_events.append(&mut events); + } + } + } + } + Err(e) => { + tracing::warn!("skipping non-UTF-8 SSE frame: {e}"); + } + } + let _ = buffer.split_to(pos + delim_len); + search_from = 0; + + if !on_events(&frame_events) { + return Ok(()); // consumer disconnected + } + } + // Next chunk: resume scanning 3 bytes back from the end. The 4-byte + // delimiter \r\n\r\n could straddle the chunk boundary. + search_from = buffer.len().saturating_sub(3); + } + + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn find_double_newline_lf() { + let buf = b"data: hello\n\ndata: world\n\n"; + let (pos, len) = find_double_newline(buf, 0).unwrap(); + assert_eq!(pos, 11); + assert_eq!(len, 2); + } + + #[test] + fn find_double_newline_crlf() { + let buf = b"data: hello\r\n\r\ndata: world\r\n\r\n"; + let (pos, len) = find_double_newline(buf, 0).unwrap(); + assert_eq!(pos, 11); + assert_eq!(len, 4); + } + + #[test] + fn find_double_newline_from_offset() { + let buf = b"data: hello\n\ndata: world\n\n"; + let (pos, len) = find_double_newline(buf, 13).unwrap(); + assert_eq!(pos, 24); + assert_eq!(len, 2); + } + + #[test] + fn find_double_newline_none() { + let buf = b"data: hello\n"; + assert!(find_double_newline(buf, 0).is_none()); + } + + #[test] + fn find_double_newline_empty() { + assert!(find_double_newline(b"", 0).is_none()); + } + + #[test] + fn find_double_newline_single_newline() { + assert!(find_double_newline(b"\n", 0).is_none()); + } + + #[test] + fn find_double_newline_just_delimiter() { + let (pos, len) = find_double_newline(b"\n\n", 0).unwrap(); + assert_eq!(pos, 0); + assert_eq!(len, 2); + } +} diff --git a/crates/proxy/Cargo.toml b/crates/proxy/Cargo.toml index 0684938..d54755a 100644 --- a/crates/proxy/Cargo.toml +++ b/crates/proxy/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "anthropic_openai_proxy" +name = "anyllm_proxy" description = "HTTP proxy translating Anthropic Messages API to OpenAI Chat Completions" version.workspace = true edition.workspace = true @@ -7,7 +7,8 @@ license.workspace = true repository.workspace = true [dependencies] -anthropic_openai_translate = { path = "../translator" } +anyllm_translate = { path = "../translator" } +anyllm_client = { path = "../client" } axum = { version = "0.8", features = ["ws"] } tokio = { version = "1", features = ["full"] } reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "native-tls", "http2"] } diff --git a/crates/proxy/src/admin/mod.rs b/crates/proxy/src/admin/mod.rs index 5b11853..005976c 100644 --- a/crates/proxy/src/admin/mod.rs +++ b/crates/proxy/src/admin/mod.rs @@ -1,5 +1,10 @@ +/// Token-based authentication for admin endpoints. pub mod auth; +/// SQLite persistence for request logs and config overrides. pub mod db; +/// Admin HTTP router: config management, request log queries, metrics. pub mod routes; +/// Shared mutable state between proxy handlers and admin server. pub mod state; +/// WebSocket handler for live admin event streaming. pub(crate) mod ws; diff --git a/crates/proxy/src/backend/anthropic_client.rs b/crates/proxy/src/backend/anthropic_client.rs index 906e559..cb6c60e 100644 --- a/crates/proxy/src/backend/anthropic_client.rs +++ b/crates/proxy/src/backend/anthropic_client.rs @@ -144,22 +144,3 @@ impl AnthropicClient { unreachable!("loop runs MAX_RETRIES+1 times and always returns") } } - -impl RateLimitHeaders { - /// Extract rate limit headers from an Anthropic response. - /// Anthropic uses `anthropic-ratelimit-*` headers natively. - pub fn from_anthropic_headers(headers: &reqwest::header::HeaderMap) -> Self { - Self { - requests_limit: super::header_str(headers, "anthropic-ratelimit-requests-limit"), - requests_remaining: super::header_str( - headers, - "anthropic-ratelimit-requests-remaining", - ), - requests_reset: super::header_str(headers, "anthropic-ratelimit-requests-reset"), - tokens_limit: super::header_str(headers, "anthropic-ratelimit-tokens-limit"), - tokens_remaining: super::header_str(headers, "anthropic-ratelimit-tokens-remaining"), - tokens_reset: super::header_str(headers, "anthropic-ratelimit-tokens-reset"), - retry_after: super::header_str(headers, "retry-after"), - } - } -} diff --git a/crates/proxy/src/backend/mod.rs b/crates/proxy/src/backend/mod.rs index d6ee673..4e8e711 100644 --- a/crates/proxy/src/backend/mod.rs +++ b/crates/proxy/src/backend/mod.rs @@ -1,178 +1,49 @@ +/// Passthrough client forwarding Anthropic requests as-is to upstream Anthropic API. pub mod anthropic_client; +/// reqwest client for OpenAI-compatible Chat Completions and Responses APIs with retry/backoff. pub mod openai_client; use crate::config::{BackendAuth, BackendConfig, BackendKind, Config, OpenAIApiFormat, TlsConfig}; use anthropic_client::{AnthropicClient, AnthropicClientError}; -use axum::http::{HeaderMap, HeaderName, HeaderValue}; use openai_client::{OpenAIClient, OpenAIClientError}; -use reqwest::Client; -use serde::Serialize; -use std::time::Duration; -use tokio::time::sleep; -/// DNS resolver that rejects private/loopback IPs at connection time, -/// preventing DNS rebinding attacks where a domain resolves to a public IP -/// at startup validation but later resolves to a private/metadata IP. -struct SsrfSafeDnsResolver; +// Re-export from the client crate so existing code paths (streaming, routes, etc.) keep working. +pub use anyllm_client::rate_limit::RateLimitHeaders; +pub use anyllm_client::retry::{ + backoff_delay, is_retryable, parse_retry_after, RetryableError, MAX_RETRIES, +}; +pub use anyllm_client::sse::{find_double_newline, MAX_SSE_BUFFER_SIZE}; -impl reqwest::dns::Resolve for SsrfSafeDnsResolver { - fn resolve(&self, name: reqwest::dns::Name) -> reqwest::dns::Resolving { - Box::pin(async move { - let name_str = name.as_str().to_string(); - // DNS resolution (ToSocketAddrs) blocks the calling thread. - // Must run on the blocking threadpool to avoid stalling the - // async runtime and all other in-flight requests. - let addrs: Vec = - tokio::task::spawn_blocking(move || -> Result, _> { - use std::net::ToSocketAddrs; - // Port 0 is a placeholder; reqwest replaces it with the actual port. - let lookup = format!("{name_str}:0"); - Ok(lookup.to_socket_addrs()?.collect()) - }) - .await - .map_err(|e| -> Box { Box::new(e) })? - .map_err( - |e: std::io::Error| -> Box { Box::new(e) }, - )?; +use anyllm_client::http::HttpClientConfig; - // Filter out private/loopback IPs to prevent SSRF attacks where - // an attacker-controlled DNS record resolves to internal endpoints - // (e.g., cloud metadata at 169.254.169.254). - let safe: Vec = addrs - .into_iter() - .filter(|addr| !crate::config::is_private_ip(addr.ip())) - .collect(); - - if safe.is_empty() { - return Err(Box::new(std::io::Error::new( - std::io::ErrorKind::PermissionDenied, - "DNS resolved only to private/loopback IPs (SSRF blocked)".to_string(), - )) - as Box); - } - - Ok(Box::new(safe.into_iter()) as Box + Send>) - }) - } -} - -pub(crate) const MAX_RETRIES: u32 = 3; -pub(crate) const BASE_DELAY_MS: u64 = 500; - -/// Backend error types implement this to enable the generic `send_with_retry`. -pub(crate) trait RetryableError: Sized { - fn from_request(e: reqwest::Error) -> Self; - fn from_api_response(status: u16, body: &str) -> Self; -} - -fn apply_auth(rb: reqwest::RequestBuilder, auth: &BackendAuth) -> reqwest::RequestBuilder { - match auth { - BackendAuth::BearerToken(token) => rb.bearer_auth(token), - BackendAuth::GoogleApiKey(key) => rb.header("x-goog-api-key", key), - } +/// Build a reqwest HTTP client from proxy TlsConfig (adapter to client crate). +pub(crate) fn build_http_client(tls: &TlsConfig) -> reqwest::Client { + let config = HttpClientConfig { + p12_identity: tls.p12_identity.clone(), + ca_cert_pem: tls.ca_cert_pem.clone(), + ssrf_protection: true, + ..Default::default() + }; + anyllm_client::build_http_client(&config) } /// Send a POST request with retry on 429/5xx. Returns the raw successful response. +/// Adapter that maps BackendAuth to the client crate's RequestAuth. pub(crate) async fn send_with_retry( - client: &Client, + client: &reqwest::Client, url: &str, auth: &BackendAuth, - body: &impl Serialize, + body: &impl serde::Serialize, label: &str, ) -> Result { - for attempt in 0..=MAX_RETRIES { - let rb = apply_auth(client.post(url).json(body), auth); - let response = rb.send().await.map_err(E::from_request)?; - let status = response.status().as_u16(); - - if (200..300).contains(&status) { - return Ok(response); - } - - if attempt < MAX_RETRIES && is_retryable(status) { - let retry_after = parse_retry_after(response.headers()); - let delay = backoff_delay(attempt, retry_after); - tracing::warn!( - status, - attempt = attempt + 1, - max_retries = MAX_RETRIES, - delay_ms = delay.as_millis() as u64, - "retryable error from {label}, backing off" - ); - // Drain the response body before retrying so the HTTP connection - // returns to the pool. Leaving it unread causes connection leaks. - drop(response.bytes().await); - sleep(delay).await; - continue; - } - - let text = response.text().await.unwrap_or_else(|e| { - tracing::warn!("failed to read error response body: {e}"); - String::new() - }); - return Err(E::from_api_response(status, &text)); - } - - unreachable!("loop runs MAX_RETRIES+1 times and always returns") -} - -/// Build a reqwest HTTP client with optional mTLS identity and custom CA cert. -pub(crate) fn build_http_client(tls: &TlsConfig) -> Client { - let mut builder = Client::builder(); - - if let Some((ref p12_bytes, ref password)) = tls.p12_identity { - let identity = reqwest::Identity::from_pkcs12_der(p12_bytes, password) - .expect("P12 identity was validated at startup"); - builder = builder.identity(identity); - } - - if let Some(ref ca_pem) = tls.ca_cert_pem { - let cert = - reqwest::Certificate::from_pem(ca_pem).expect("CA cert was validated at startup"); - builder = builder.add_root_certificate(cert); - } - - builder - .connect_timeout(Duration::from_secs(10)) - // 15 min read timeout: generous for slow-starting reasoning models - // (o1/o3 can think >5 min before the first chunk) while still - // bounding hung connections that would otherwise pin resources. - .read_timeout(Duration::from_secs(900)) - // Detect dead TCP connections (peer crash, network drop). - .tcp_keepalive(Duration::from_secs(60)) - // Validate resolved IPs at connection time to prevent DNS rebinding SSRF. - .dns_resolver(std::sync::Arc::new(SsrfSafeDnsResolver)) - .build() - .expect("failed to build HTTP client") -} - -/// Check if a status code is retryable (408, 429, or 5xx). -pub(crate) fn is_retryable(status: u16) -> bool { - status == 408 || status == 429 || (500..=599).contains(&status) -} - -/// Parse retry-after header as integer seconds or HTTP date (RFC 7231). -pub(crate) fn parse_retry_after(headers: &reqwest::header::HeaderMap) -> Option { - let value = header_str(headers, "retry-after")?; - // Fast path: integer seconds - if let Ok(secs) = value.parse::() { - return Some(Duration::from_secs(secs)); - } - // HTTP date (RFC 7231). Past dates return None (no wait needed). - let date = httpdate::parse_http_date(&value).ok()?; - date.duration_since(std::time::SystemTime::now()).ok() -} - -/// Compute backoff delay with jitter. -pub(crate) fn backoff_delay(attempt: u32, retry_after: Option) -> Duration { - if let Some(ra) = retry_after { - return ra; - } - let base = Duration::from_millis(BASE_DELAY_MS * 2u64.pow(attempt)); - // Deterministic 25% jitter (upper bound, not random) to keep tests - // predictable while still spreading retry storms across backends. - let jitter_ms = (base.as_millis() as u64) / 4; - base + Duration::from_millis(jitter_ms) + let request_auth = match auth { + BackendAuth::BearerToken(token) => anyllm_client::retry::RequestAuth::Bearer(token), + BackendAuth::GoogleApiKey(key) => anyllm_client::retry::RequestAuth::Header { + name: "x-goog-api-key", + value: key, + }, + }; + anyllm_client::retry::send_with_retry(client, url, &request_auth, body, label).await } /// Backend-agnostic client for dispatching requests to OpenAI, Vertex, Gemini, or Anthropic. @@ -254,6 +125,10 @@ impl From for BackendError { } impl BackendClient { + /// Create a backend client from a single-backend [`Config`]. + /// + /// Dispatches on [`Config::backend`] and [`Config::openai_api_format`] to construct + /// the appropriate variant (OpenAI, OpenAIResponses, Vertex, GeminiOpenAI, or Anthropic). pub fn new(config: &Config) -> Self { match config.backend { BackendKind::OpenAI => match config.openai_api_format { @@ -297,335 +172,3 @@ impl BackendClient { } } } - -/// Parse OpenAI's duration format (e.g., "6ms", "1s", "1m30s", "2m") into a -/// [`Duration`]. Returns `None` for unrecognized formats. -fn parse_openai_duration(s: &str) -> Option { - let s = s.trim(); - if s.is_empty() { - return None; - } - let mut total_ms: u64 = 0; - let mut num_start: Option = None; - let bytes = s.as_bytes(); - let mut i = 0; - while i < bytes.len() { - let c = bytes[i]; - if c.is_ascii_digit() || c == b'.' { - if num_start.is_none() { - num_start = Some(i); - } - i += 1; - } else if c.is_ascii_alphabetic() { - let start = num_start?; - let num_str = &s[start..i]; - let unit_start = i; - while i < bytes.len() && bytes[i].is_ascii_alphabetic() { - i += 1; - } - let unit = &s[unit_start..i]; - let value: f64 = num_str.parse().ok()?; - let ms = match unit { - "ms" => value, - "s" => value * 1_000.0, - "m" => value * 60_000.0, - "h" => value * 3_600_000.0, - _ => return None, - }; - total_ms += ms.round() as u64; - num_start = None; - } else { - return None; - } - } - // Trailing number with no unit is invalid - if num_start.is_some() { - return None; - } - Some(Duration::from_millis(total_ms)) -} - -/// Convert an OpenAI relative duration string to an ISO 8601 UTC timestamp -/// by adding it to the current time. Returns `None` if parsing fails. -/// -/// Accepts an `anchor` time so callers can pin the base for testability. -fn openai_duration_to_iso8601_at(s: &str, anchor: std::time::SystemTime) -> Option { - let dur = parse_openai_duration(s)?; - let reset_time = anchor + dur; - let secs = reset_time - .duration_since(std::time::UNIX_EPOCH) - .ok()? - .as_secs(); - Some(crate::admin::db::epoch_to_iso8601(secs)) -} - -/// Convenience wrapper using the current time as anchor. -fn openai_duration_to_iso8601(s: &str) -> Option { - openai_duration_to_iso8601_at(s, std::time::SystemTime::now()) -} - -/// Convert an OpenAI relative duration to ISO 8601, falling back to the raw -/// value with a warning if parsing fails. -fn convert_reset_duration(raw: &Option, field: &str) -> Option { - raw.as_deref().map(|v| { - openai_duration_to_iso8601(v).unwrap_or_else(|| { - tracing::warn!( - value = v, - field, - "failed to parse reset duration, forwarding raw" - ); - v.to_string() - }) - }) -} - -/// Rate limit headers extracted from backend responses. -/// Forwarded to clients as Anthropic-style `anthropic-ratelimit-*` headers. -/// See: -#[derive(Debug, Default, Clone)] -pub struct RateLimitHeaders { - /// Maximum requests allowed in the current window. - pub requests_limit: Option, - /// Requests remaining before rate limiting kicks in. - pub requests_remaining: Option, - /// ISO 8601 timestamp when the request limit resets. - pub requests_reset: Option, - /// Maximum tokens allowed in the current window. - pub tokens_limit: Option, - /// Tokens remaining before rate limiting kicks in. - pub tokens_remaining: Option, - /// ISO 8601 timestamp when the token limit resets. - pub tokens_reset: Option, - /// Seconds to wait before retrying (from `retry-after` header on 429s). - pub retry_after: Option, -} - -/// Extract a header value as a trimmed string. -fn header_str(headers: &reqwest::header::HeaderMap, name: &str) -> Option { - headers - .get(name) - .and_then(|v| v.to_str().ok()) - .map(|s| s.trim().to_string()) -} - -/// Set a header on an axum HeaderMap if the value is Some. -fn set_if_some(map: &mut HeaderMap, name: &str, value: &Option) { - if let Some(v) = value { - if let (Ok(header_name), Ok(header_value)) = ( - HeaderName::from_bytes(name.as_bytes()), - axum::http::HeaderValue::from_str(v), - ) { - map.insert(header_name, header_value); - } - } -} - -impl RateLimitHeaders { - /// Extract rate limit headers from an OpenAI (or Vertex) response. - pub fn from_openai_headers(headers: &reqwest::header::HeaderMap) -> Self { - Self { - requests_limit: header_str(headers, "x-ratelimit-limit-requests"), - requests_remaining: header_str(headers, "x-ratelimit-remaining-requests"), - requests_reset: header_str(headers, "x-ratelimit-reset-requests"), - tokens_limit: header_str(headers, "x-ratelimit-limit-tokens"), - tokens_remaining: header_str(headers, "x-ratelimit-remaining-tokens"), - tokens_reset: header_str(headers, "x-ratelimit-reset-tokens"), - retry_after: header_str(headers, "retry-after"), - } - } - - /// Inject Anthropic-format response headers (rate limits + version) into a HeaderMap. - /// - /// The `*_reset` fields are converted from OpenAI's relative duration - /// format (e.g., "1s") to Anthropic's ISO 8601 UTC timestamp format. - /// Falls back to the raw value with a warning if parsing fails. - pub fn inject_anthropic_response_headers(&self, map: &mut HeaderMap) { - set_if_some( - map, - "anthropic-ratelimit-requests-limit", - &self.requests_limit, - ); - set_if_some( - map, - "anthropic-ratelimit-requests-remaining", - &self.requests_remaining, - ); - let req_reset = convert_reset_duration(&self.requests_reset, "requests_reset"); - set_if_some(map, "anthropic-ratelimit-requests-reset", &req_reset); - set_if_some(map, "anthropic-ratelimit-tokens-limit", &self.tokens_limit); - set_if_some( - map, - "anthropic-ratelimit-tokens-remaining", - &self.tokens_remaining, - ); - let tok_reset = convert_reset_duration(&self.tokens_reset, "tokens_reset"); - set_if_some(map, "anthropic-ratelimit-tokens-reset", &tok_reset); - set_if_some(map, "retry-after", &self.retry_after); - map.insert( - HeaderName::from_static("anthropic-version"), - HeaderValue::from_static("2023-06-01"), - ); - } -} - -#[cfg(test)] -mod rate_limit_tests { - use super::*; - - #[test] - fn from_openai_headers_extracts_all() { - let mut headers = reqwest::header::HeaderMap::new(); - headers.insert("x-ratelimit-limit-requests", "100".parse().unwrap()); - headers.insert("x-ratelimit-remaining-requests", "99".parse().unwrap()); - headers.insert("x-ratelimit-reset-requests", "1s".parse().unwrap()); - headers.insert("x-ratelimit-limit-tokens", "40000".parse().unwrap()); - headers.insert("x-ratelimit-remaining-tokens", "39500".parse().unwrap()); - headers.insert("x-ratelimit-reset-tokens", "500ms".parse().unwrap()); - headers.insert("retry-after", "2".parse().unwrap()); - - let rl = RateLimitHeaders::from_openai_headers(&headers); - assert_eq!(rl.requests_limit.as_deref(), Some("100")); - assert_eq!(rl.requests_remaining.as_deref(), Some("99")); - assert_eq!(rl.requests_reset.as_deref(), Some("1s")); - assert_eq!(rl.tokens_limit.as_deref(), Some("40000")); - assert_eq!(rl.tokens_remaining.as_deref(), Some("39500")); - assert_eq!(rl.tokens_reset.as_deref(), Some("500ms")); - assert_eq!(rl.retry_after.as_deref(), Some("2")); - } - - #[test] - fn from_openai_headers_missing_are_none() { - let headers = reqwest::header::HeaderMap::new(); - let rl = RateLimitHeaders::from_openai_headers(&headers); - assert!(rl.requests_limit.is_none()); - assert!(rl.requests_remaining.is_none()); - assert!(rl.requests_reset.is_none()); - assert!(rl.tokens_limit.is_none()); - assert!(rl.tokens_remaining.is_none()); - assert!(rl.tokens_reset.is_none()); - assert!(rl.retry_after.is_none()); - } - - #[test] - fn inject_anthropic_response_headers_sets_values() { - let rl = RateLimitHeaders { - requests_limit: Some("100".into()), - tokens_remaining: Some("39500".into()), - retry_after: Some("3".into()), - ..Default::default() - }; - let mut map = HeaderMap::new(); - rl.inject_anthropic_response_headers(&mut map); - - assert_eq!( - map.get("anthropic-ratelimit-requests-limit").unwrap(), - "100" - ); - assert_eq!( - map.get("anthropic-ratelimit-tokens-remaining").unwrap(), - "39500" - ); - assert_eq!(map.get("retry-after").unwrap(), "3"); - assert_eq!(map.get("anthropic-version").unwrap(), "2023-06-01"); - // Fields that were None should not be present - assert!(map.get("anthropic-ratelimit-requests-remaining").is_none()); - assert!(map.get("anthropic-ratelimit-tokens-limit").is_none()); - } - - #[test] - fn inject_anthropic_response_headers_default_sets_version_only() { - let rl = RateLimitHeaders::default(); - let mut map = HeaderMap::new(); - rl.inject_anthropic_response_headers(&mut map); - assert_eq!(map.len(), 1); - assert_eq!(map.get("anthropic-version").unwrap(), "2023-06-01"); - } - - #[test] - fn inject_anthropic_response_headers_converts_reset_to_iso8601() { - let rl = RateLimitHeaders { - requests_reset: Some("1s".into()), - tokens_reset: Some("500ms".into()), - ..Default::default() - }; - let mut map = HeaderMap::new(); - rl.inject_anthropic_response_headers(&mut map); - - let req_reset = map - .get("anthropic-ratelimit-requests-reset") - .unwrap() - .to_str() - .unwrap(); - let tok_reset = map - .get("anthropic-ratelimit-tokens-reset") - .unwrap() - .to_str() - .unwrap(); - // Should be ISO 8601 format, not the raw OpenAI duration. - assert!( - req_reset.contains('T') && req_reset.ends_with('Z'), - "expected ISO 8601 timestamp, got: {req_reset}" - ); - assert!( - tok_reset.contains('T') && tok_reset.ends_with('Z'), - "expected ISO 8601 timestamp, got: {tok_reset}" - ); - } - - #[test] - fn parse_openai_duration_various_formats() { - assert_eq!(parse_openai_duration("6ms"), Some(Duration::from_millis(6))); - assert_eq!( - parse_openai_duration("1s"), - Some(Duration::from_millis(1000)) - ); - assert_eq!( - parse_openai_duration("2m"), - Some(Duration::from_millis(120_000)) - ); - assert_eq!( - parse_openai_duration("1m30s"), - Some(Duration::from_millis(90_000)) - ); - assert_eq!( - parse_openai_duration("1h"), - Some(Duration::from_millis(3_600_000)) - ); - assert_eq!( - parse_openai_duration("1h30m"), - Some(Duration::from_millis(5_400_000)) - ); - } - - #[test] - fn parse_openai_duration_invalid() { - assert_eq!(parse_openai_duration(""), None); - assert_eq!(parse_openai_duration("abc"), None); - assert_eq!(parse_openai_duration("123"), None); // no unit - assert_eq!(parse_openai_duration("1x"), None); // unknown unit - } - - #[test] - fn openai_duration_to_iso8601_at_pinned_time() { - // 2025-06-16T12:00:00Z = 1750075200 epoch seconds - let anchor = std::time::UNIX_EPOCH + Duration::from_secs(1_750_075_200); - assert_eq!( - openai_duration_to_iso8601_at("1s", anchor).unwrap(), - "2025-06-16T12:00:01Z" - ); - assert_eq!( - openai_duration_to_iso8601_at("1m30s", anchor).unwrap(), - "2025-06-16T12:01:30Z" - ); - assert_eq!( - openai_duration_to_iso8601_at("500ms", anchor).unwrap(), - "2025-06-16T12:00:00Z" // 500ms rounds down to same second - ); - } - - #[test] - fn openai_duration_to_iso8601_invalid_returns_none() { - assert!(openai_duration_to_iso8601("garbage").is_none()); - assert!(openai_duration_to_iso8601("").is_none()); - } -} diff --git a/crates/proxy/src/config/url_validation.rs b/crates/proxy/src/config/url_validation.rs index 5c9104a..ec490ed 100644 --- a/crates/proxy/src/config/url_validation.rs +++ b/crates/proxy/src/config/url_validation.rs @@ -4,6 +4,9 @@ use std::net::IpAddr; use url::Url; +// Re-export is_private_ip from the client crate (canonical location). +pub use anyllm_client::http::is_private_ip; + /// Validate that a base URL is safe to use as an upstream target. /// Rejects non-http(s) schemes, private/loopback IPs, and link-local addresses. /// For domain names, also resolves DNS and validates all resolved IPs to prevent @@ -77,29 +80,6 @@ pub fn validate_base_url(raw: &str) -> Result<(), String> { Ok(()) } -/// Returns true for loopback, private (RFC 1918), link-local, and -/// cloud metadata IPs (169.254.169.254). -pub fn is_private_ip(ip: IpAddr) -> bool { - match ip { - IpAddr::V4(v4) => { - v4.is_loopback() - || v4.is_private() - || v4.is_link_local() - || v4.is_broadcast() - || v4.is_unspecified() - // AWS/GCP/Azure metadata endpoint. SSRF to this IP lets - // attackers exfiltrate instance credentials. - || v4 == std::net::Ipv4Addr::new(169, 254, 169, 254) - } - IpAddr::V6(v6) => { - v6.is_loopback() || v6.is_unspecified() - // Check IPv4-mapped IPv6 addresses (::ffff:192.168.x.x) recursively; - // attackers can bypass IPv4 checks using the mapped representation. - || matches!(v6.to_ipv4_mapped(), Some(v4) if is_private_ip(IpAddr::V4(v4))) - } - } -} - #[cfg(test)] mod tests { use super::*; diff --git a/crates/proxy/src/lib.rs b/crates/proxy/src/lib.rs index eade47c..a98de8f 100644 --- a/crates/proxy/src/lib.rs +++ b/crates/proxy/src/lib.rs @@ -1,5 +1,10 @@ +/// Admin server: localhost-only config management, request logging, WebSocket live updates. pub mod admin; +/// Backend HTTP clients for OpenAI, Vertex, Gemini, and Anthropic passthrough. pub mod backend; +/// Environment-based configuration, TLS client cert setup, URL validation. pub mod config; +/// Request count, success/error tracking, exposed via GET /metrics. pub mod metrics; +/// Axum HTTP server: routes, middleware (auth, request ID, size/concurrency limits), SSE streaming. pub mod server; diff --git a/crates/proxy/src/main.rs b/crates/proxy/src/main.rs index 5fea814..7533185 100644 --- a/crates/proxy/src/main.rs +++ b/crates/proxy/src/main.rs @@ -1,4 +1,4 @@ -use anthropic_openai_proxy::{admin, config, server::routes}; +use anyllm_proxy::{admin, config, server::routes}; use std::sync::Arc; use tracing_subscriber::prelude::*; @@ -116,10 +116,8 @@ async fn main() { let (events_tx, _) = tokio::sync::broadcast::channel(1024); let log_tx = admin::db::spawn_write_buffer(db.clone()); - let backend_metrics: std::collections::HashMap< - String, - anthropic_openai_proxy::metrics::Metrics, - > = std::collections::HashMap::new(); + let backend_metrics: std::collections::HashMap = + std::collections::HashMap::new(); let shared = admin::state::SharedState { db: db.clone(), @@ -188,7 +186,7 @@ async fn main() { continue; } let mut backends = std::collections::HashMap::new(); - let mut aggregate = anthropic_openai_proxy::metrics::MetricsSnapshot::default(); + let mut aggregate = anyllm_proxy::metrics::MetricsSnapshot::default(); for (name, m) in snapshot_shared.backend_metrics.iter() { let snap = m.snapshot(); aggregate.requests_total += snap.requests_total; diff --git a/crates/proxy/src/server/mod.rs b/crates/proxy/src/server/mod.rs index 00ed5d4..c431279 100644 --- a/crates/proxy/src/server/mod.rs +++ b/crates/proxy/src/server/mod.rs @@ -1,6 +1,12 @@ +/// Auth validation, request ID injection, size limits, concurrency limits, header logging. pub mod middleware; +/// Anthropic passthrough handler (no translation, forwards as-is). mod passthrough; +/// Axum router setup and request handlers for all API endpoints. pub mod routes; +/// SSE response helpers for Anthropic-format streaming. pub mod sse; +/// SSE streaming handler with pre-stream error propagation and backpressure. mod streaming; +/// Approximate token counting via tiktoken. mod token_counting; diff --git a/crates/proxy/src/server/passthrough.rs b/crates/proxy/src/server/passthrough.rs index 95cc200..4d6c2bb 100644 --- a/crates/proxy/src/server/passthrough.rs +++ b/crates/proxy/src/server/passthrough.rs @@ -2,7 +2,7 @@ // No translation: the proxy receives Anthropic format and returns Anthropic format. use crate::backend::BackendClient; -use anthropic_openai_translate::{anthropic, mapping}; +use anyllm_translate::{anthropic, mapping}; use axum::{ body::Bytes, extract::State, diff --git a/crates/proxy/src/server/routes.rs b/crates/proxy/src/server/routes.rs index 9f53084..71ecbb5 100644 --- a/crates/proxy/src/server/routes.rs +++ b/crates/proxy/src/server/routes.rs @@ -2,7 +2,7 @@ use crate::admin::state::{AdminEvent, RequestLogEntry, RuntimeConfig, SharedStat use crate::backend::{BackendClient, BackendError}; use crate::config::{BackendKind, Config, MultiConfig}; use crate::metrics::Metrics; -use anthropic_openai_translate::{anthropic, mapping, openai}; +use anyllm_translate::{anthropic, mapping, openai}; use axum::{ extract::{rejection::JsonRejection, DefaultBodyLimit, FromRequest, State}, http::StatusCode, @@ -45,7 +45,10 @@ where } } -/// Per-backend state shared across request handlers for one backend. +/// Per-backend state shared across request handlers. +/// +/// In single-backend mode, one `AppState` serves all routes. In multi-backend mode, +/// each backend gets its own `AppState` mounted under a prefix path (e.g., `/openai/v1/messages`). #[derive(Clone)] pub struct AppState { pub backend: BackendClient, @@ -259,7 +262,9 @@ async fn enforce_concurrency( ); return (StatusCode::TOO_MANY_REQUESTS, Json(err)).into_response(); }; - request.extensions_mut().insert(ConcurrencyPermit(Arc::new(permit))); + request + .extensions_mut() + .insert(ConcurrencyPermit(Arc::new(permit))); next.run(request).await } @@ -267,7 +272,9 @@ async fn enforce_concurrency( /// The field is never read directly; it exists as an RAII guard to hold /// the permit until the struct is dropped. #[derive(Clone)] -pub(crate) struct ConcurrencyPermit(#[allow(dead_code)] pub(crate) Arc); +pub(crate) struct ConcurrencyPermit( + #[allow(dead_code)] pub(crate) Arc, +); static MODELS_RESPONSE: std::sync::LazyLock = std::sync::LazyLock::new(|| { serde_json::json!({ diff --git a/crates/proxy/src/server/streaming.rs b/crates/proxy/src/server/streaming.rs index 3489ef7..80eaac1 100644 --- a/crates/proxy/src/server/streaming.rs +++ b/crates/proxy/src/server/streaming.rs @@ -1,8 +1,8 @@ // SSE streaming infrastructure and the messages_stream handler. -use crate::backend::{BackendClient, RateLimitHeaders}; +use crate::backend::{BackendClient, RateLimitHeaders, find_double_newline, MAX_SSE_BUFFER_SIZE}; use crate::metrics::Metrics; -use anthropic_openai_translate::{anthropic, mapping, openai}; +use anyllm_translate::{anthropic, mapping, openai}; use axum::response::sse::{Event, KeepAlive, Sse}; use bytes::BytesMut; use futures::stream::Stream; @@ -31,33 +31,6 @@ async fn send_events( true } -/// Maximum SSE buffer size (10 MB). Protects against unbounded memory growth -/// if the backend sends data without frame delimiters. -const MAX_SSE_BUFFER_SIZE: usize = 10 * 1024 * 1024; - -/// Find the first SSE frame boundary (`\n\n` or `\r\n\r\n`) in a byte slice, -/// starting the search at `start`. Returns `(position, delimiter_length)` so -/// the caller can skip the full delimiter. -fn find_double_newline(buf: &[u8], start: usize) -> Option<(usize, usize)> { - let len = buf.len(); - let mut i = start; - while i < len.saturating_sub(1) { - if buf[i] == b'\n' && buf[i + 1] == b'\n' { - return Some((i, 2)); - } - if buf[i] == b'\r' - && i + 3 < len - && buf[i + 1] == b'\n' - && buf[i + 2] == b'\r' - && buf[i + 3] == b'\n' - { - return Some((i, 4)); - } - i += 1; - } - None -} - /// Why the SSE stream ended. enum StreamOutcome { /// Backend stream completed normally. diff --git a/crates/proxy/src/server/token_counting.rs b/crates/proxy/src/server/token_counting.rs index c23a654..2d0db28 100644 --- a/crates/proxy/src/server/token_counting.rs +++ b/crates/proxy/src/server/token_counting.rs @@ -1,6 +1,6 @@ // Token counting endpoint and helpers. -use anthropic_openai_translate::anthropic; +use anyllm_translate::anthropic; use axum::{http::StatusCode, response::IntoResponse, Json}; use std::sync::LazyLock; use tiktoken_rs::CoreBPE; diff --git a/crates/proxy/tests/body_logging.rs b/crates/proxy/tests/body_logging.rs index eab990f..c81649e 100644 --- a/crates/proxy/tests/body_logging.rs +++ b/crates/proxy/tests/body_logging.rs @@ -1,5 +1,5 @@ -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; fn test_config_with_logging() -> Config { Config { diff --git a/crates/proxy/tests/compatibility.rs b/crates/proxy/tests/compatibility.rs index 692fbc4..b8d3058 100644 --- a/crates/proxy/tests/compatibility.rs +++ b/crates/proxy/tests/compatibility.rs @@ -1,8 +1,8 @@ // Phase 9-10: compatibility endpoint and hardening integration tests // Phase 19: token counting integration tests -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; use reqwest::Client; fn test_config() -> Config { diff --git a/crates/proxy/tests/error_fixtures.rs b/crates/proxy/tests/error_fixtures.rs index e87ce2f..f44e291 100644 --- a/crates/proxy/tests/error_fixtures.rs +++ b/crates/proxy/tests/error_fixtures.rs @@ -1,8 +1,7 @@ #[test] fn malformed_openai_response_fails_deserialization() { let json = include_str!("../../../fixtures/openai/chat_completion_malformed.json"); - let result = - serde_json::from_str::(json); + let result = serde_json::from_str::(json); assert!( result.is_err(), "malformed response should fail deserialization" diff --git a/crates/proxy/tests/health.rs b/crates/proxy/tests/health.rs index 4a0458d..b3f7acf 100644 --- a/crates/proxy/tests/health.rs +++ b/crates/proxy/tests/health.rs @@ -1,4 +1,4 @@ -use anthropic_openai_proxy::{config::Config, server::routes}; +use anyllm_proxy::{config::Config, server::routes}; use tokio::net::TcpListener; #[tokio::test] diff --git a/crates/proxy/tests/live_api.rs b/crates/proxy/tests/live_api.rs index 0e3aa0c..b0d3504 100644 --- a/crates/proxy/tests/live_api.rs +++ b/crates/proxy/tests/live_api.rs @@ -7,8 +7,8 @@ //! OPENAI_API_KEY=sk-... cargo test --test live_api -- --ignored --test-threads=1 //! ``` -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; use serde_json::{json, Value}; use tokio::net::TcpListener; diff --git a/crates/proxy/tests/multi_backend.rs b/crates/proxy/tests/multi_backend.rs index e44fe5f..c4031e4 100644 --- a/crates/proxy/tests/multi_backend.rs +++ b/crates/proxy/tests/multi_backend.rs @@ -1,7 +1,7 @@ // Integration tests for multi-backend path-prefix routing. -use anthropic_openai_proxy::config::MultiConfig; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::MultiConfig; +use anyllm_proxy::server::routes; use reqwest::Client; fn test_multi_config() -> MultiConfig { diff --git a/crates/proxy/tests/shutdown.rs b/crates/proxy/tests/shutdown.rs index 474a58b..bc80327 100644 --- a/crates/proxy/tests/shutdown.rs +++ b/crates/proxy/tests/shutdown.rs @@ -1,7 +1,7 @@ // Test: server shuts down cleanly on signal, in-flight requests complete. -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; use std::time::Duration; fn test_config() -> Config { diff --git a/crates/translator/Cargo.toml b/crates/translator/Cargo.toml index 04c17f7..db110aa 100644 --- a/crates/translator/Cargo.toml +++ b/crates/translator/Cargo.toml @@ -1,5 +1,5 @@ [package] -name = "anthropic_openai_translate" +name = "anyllm_translate" description = "Pure translation layer between Anthropic Messages API and OpenAI Chat Completions" version.workspace = true edition.workspace = true diff --git a/crates/translator/README.md b/crates/translator/README.md index 362dde4..14ab16e 100644 --- a/crates/translator/README.md +++ b/crates/translator/README.md @@ -1,4 +1,4 @@ -# anthropic_openai_translate +# anyllm_translate Pure, IO-free translation between Anthropic Messages API and OpenAI Chat Completions / Responses API formats. Also supports Google Gemini native API translation. @@ -7,8 +7,8 @@ No HTTP clients, no async runtime, no network calls. Just `fn(A) -> B` transform ## Quick Start ```rust -use anthropic_openai_translate::{TranslationConfig, translate_request, translate_response}; -use anthropic_openai_translate::anthropic::MessageCreateRequest; +use anyllm_translate::{TranslationConfig, translate_request, translate_response}; +use anyllm_translate::anthropic::MessageCreateRequest; let config = TranslationConfig::builder() .model_map("haiku", "gpt-4o-mini") @@ -43,10 +43,10 @@ assert_eq!(openai_req.model, "gpt-4o"); ```toml [dependencies] -anthropic_openai_translate = "0.1" +anyllm_translate = "0.1" # With middleware support: -anthropic_openai_translate = { version = "0.1", features = ["middleware"] } +anyllm_translate = { version = "0.1", features = ["middleware"] } ``` ## Modules @@ -74,4 +74,4 @@ anthropic_openai_translate = { version = "0.1", features = ["middleware"] } ## Related -This crate is part of [llm-translate-api](https://github.com/whit3rabbit/llm-translate-api), which also includes a standalone HTTP proxy server. +This crate is part of [anyllm-proxy](https://github.com/whit3rabbit/anyllm-proxy), which also includes a standalone HTTP proxy server. diff --git a/crates/translator/src/anthropic/mod.rs b/crates/translator/src/anthropic/mod.rs index b9e768f..23ae22a 100644 --- a/crates/translator/src/anthropic/mod.rs +++ b/crates/translator/src/anthropic/mod.rs @@ -1,5 +1,8 @@ +/// Anthropic error response types (`ErrorResponse`, `ErrorType`, `ErrorDetail`). pub mod errors; +/// Anthropic Messages API request and response types. pub mod messages; +/// Anthropic SSE streaming event types (`StreamEvent`, `Delta`). pub mod streaming; // Re-export primary types diff --git a/crates/translator/src/lib.rs b/crates/translator/src/lib.rs index d12c708..f62212a 100644 --- a/crates/translator/src/lib.rs +++ b/crates/translator/src/lib.rs @@ -1,12 +1,12 @@ -//! # anthropic_openai_translate +//! # anyllm_translate //! //! Pure, IO-free translation between Anthropic Messages API and OpenAI Chat Completions API. //! //! # Quick start //! //! ```rust -//! use anthropic_openai_translate::{TranslationConfig, translate_request, translate_response}; -//! use anthropic_openai_translate::anthropic::MessageCreateRequest; +//! use anyllm_translate::{TranslationConfig, translate_request, translate_response}; +//! use anyllm_translate::anthropic::MessageCreateRequest; //! //! let config = TranslationConfig::builder() //! .model_map("haiku", "gpt-4o-mini") @@ -35,14 +35,22 @@ //! - [`config`] -- Translation configuration (model mapping, lossy behavior) //! - [`translate`] -- Convenience wrappers combining config with mapping functions +/// Anthropic Messages API types (request, response, streaming events, errors). pub mod anthropic; +/// Translation configuration: model mapping and lossy-translation behavior. pub mod config; +/// Error types for translation failures. pub mod error; +/// Stateless conversion functions between Anthropic and OpenAI API formats. pub mod mapping; +/// HTTP middleware for request/response translation (requires `middleware` feature). #[cfg(feature = "middleware")] pub mod middleware; +/// OpenAI Chat Completions and Responses API types. pub mod openai; +/// Convenience wrappers combining config with mapping functions. pub mod translate; +/// Shared utilities: ID generation, JSON helpers, secret redaction. pub mod util; // Convenience re-exports diff --git a/crates/translator/src/mapping/mod.rs b/crates/translator/src/mapping/mod.rs index fd04a05..3567513 100644 --- a/crates/translator/src/mapping/mod.rs +++ b/crates/translator/src/mapping/mod.rs @@ -1,9 +1,16 @@ +/// HTTP status and error shape translation between APIs. pub mod errors_map; +/// Message and content block translation (system prompt, text, images, documents). pub mod message_map; +/// Anthropic to/from OpenAI Responses API request and response mapping. pub mod responses_message_map; +/// Responses API SSE event stream translation state machine. pub mod responses_streaming_map; +/// Chat Completions SSE event stream translation state machine. pub mod streaming_map; +/// Tool definitions and tool_use/tool_call translation. pub mod tools_map; +/// Token usage field mapping between Anthropic and OpenAI formats. pub mod usage_map; /// Format an OpenAI refusal string as Anthropic text content. diff --git a/crates/translator/src/mapping/responses_streaming_map.rs b/crates/translator/src/mapping/responses_streaming_map.rs index b17faeb..b287680 100644 --- a/crates/translator/src/mapping/responses_streaming_map.rs +++ b/crates/translator/src/mapping/responses_streaming_map.rs @@ -41,6 +41,8 @@ pub struct ResponsesStreamingTranslator { } impl ResponsesStreamingTranslator { + /// Create a new translator for the given model name. + /// Generates a fresh Anthropic message ID for the translated stream. pub fn new(model: String) -> Self { Self { model, diff --git a/crates/translator/src/mapping/usage_map.rs b/crates/translator/src/mapping/usage_map.rs index f38b82f..14731bb 100644 --- a/crates/translator/src/mapping/usage_map.rs +++ b/crates/translator/src/mapping/usage_map.rs @@ -1,13 +1,10 @@ -// Usage field mapping between Anthropic and OpenAI +//! Token usage field mapping between Anthropic and OpenAI APIs. use crate::anthropic; use crate::openai; -/// Convert OpenAI usage to Anthropic usage. -/// -/// OpenAI: -/// Anthropic: /// Extract `cached_tokens` from an OpenAI token details JSON object. +/// /// Used by both Chat Completions (`prompt_tokens_details`) and Responses API /// (`input_token_details`) paths to map to Anthropic's `cache_read_input_tokens`. pub(crate) fn extract_cached_tokens(details: Option<&serde_json::Value>) -> Option { @@ -17,6 +14,13 @@ pub(crate) fn extract_cached_tokens(details: Option<&serde_json::Value>) -> Opti .map(|n| n as u32) } +/// Convert OpenAI token usage to Anthropic usage format. +/// +/// Maps `prompt_tokens` to `input_tokens`, `completion_tokens` to `output_tokens`, +/// and extracts `cached_tokens` from `prompt_tokens_details` into `cache_read_input_tokens`. +/// +/// OpenAI usage: +/// Anthropic usage: pub fn openai_to_anthropic_usage(usage: &openai::ChatUsage) -> anthropic::Usage { // OpenAI reports cached tokens in prompt_tokens_details.cached_tokens; // Anthropic calls the same concept cache_read_input_tokens. diff --git a/crates/translator/src/openai/mod.rs b/crates/translator/src/openai/mod.rs index a8ffd9a..35d7b95 100644 --- a/crates/translator/src/openai/mod.rs +++ b/crates/translator/src/openai/mod.rs @@ -1,6 +1,10 @@ +/// OpenAI Chat Completions API request and response types. pub mod chat_completions; +/// OpenAI error response types. pub mod errors; +/// OpenAI Responses API types. pub mod responses; +/// OpenAI Chat Completions SSE streaming chunk types. pub mod streaming; pub use chat_completions::{ diff --git a/crates/translator/src/util/mod.rs b/crates/translator/src/util/mod.rs index b7e873a..e2fc475 100644 --- a/crates/translator/src/util/mod.rs +++ b/crates/translator/src/util/mod.rs @@ -1,3 +1,6 @@ +/// UUID-based ID generation for Anthropic message and content block IDs. pub mod ids; +/// JSON serialization helpers (pretty-print, merge, normalize). pub mod json; +/// Secret redaction for logging (API keys, tokens). pub mod redact; diff --git a/crates/translator/tests/golden_fixtures.rs b/crates/translator/tests/golden_fixtures.rs index 9de14b1..638fc56 100644 --- a/crates/translator/tests/golden_fixtures.rs +++ b/crates/translator/tests/golden_fixtures.rs @@ -1,7 +1,7 @@ // Golden-file tests: validate that fixture JSON files can be deserialized // and that translation between formats produces the expected shapes. -use anthropic_openai_translate::{anthropic, mapping, openai}; +use anyllm_translate::{anthropic, mapping, openai}; fn fixtures_dir() -> std::path::PathBuf { std::path::Path::new(env!("CARGO_MANIFEST_DIR")) diff --git a/crates/translator/tests/library_usage.rs b/crates/translator/tests/library_usage.rs index 71ea3d3..a0caefe 100644 --- a/crates/translator/tests/library_usage.rs +++ b/crates/translator/tests/library_usage.rs @@ -1,10 +1,8 @@ //! Integration test: verify the crate works as a standalone library without the proxy. -use anthropic_openai_translate::anthropic::{MessageCreateRequest, MessageResponse, Usage}; -use anthropic_openai_translate::openai::ChatCompletionResponse; -use anthropic_openai_translate::{ - translate_request, translate_response, TranslateError, TranslationConfig, -}; +use anyllm_translate::anthropic::{MessageCreateRequest, MessageResponse, Usage}; +use anyllm_translate::openai::ChatCompletionResponse; +use anyllm_translate::{translate_request, translate_response, TranslateError, TranslationConfig}; #[test] fn standalone_translate_request() { diff --git a/crates/translator/tests/middleware_integration.rs b/crates/translator/tests/middleware_integration.rs index 2c1f16c..368059c 100644 --- a/crates/translator/tests/middleware_integration.rs +++ b/crates/translator/tests/middleware_integration.rs @@ -11,10 +11,10 @@ use axum::routing::post; use axum::Router; use tokio::net::TcpListener; -use anthropic_openai_translate::middleware::{ +use anyllm_translate::middleware::{ anthropic_compat_router, AnthropicCompatConfig, AnthropicTranslationLayer, }; -use anthropic_openai_translate::TranslationConfig; +use anyllm_translate::TranslationConfig; // --- Mock OpenAI backend --- diff --git a/docs/ENV.md b/docs/ENV.md index f311b73..f9703fa 100644 --- a/docs/ENV.md +++ b/docs/ENV.md @@ -11,7 +11,7 @@ These are the variables most users need. | `LISTEN_PORT` | `3000` | Port the proxy listens on. | | `BIG_MODEL` | `gpt-4o` | OpenAI model used when the Anthropic request specifies a sonnet or opus model. | | `SMALL_MODEL` | `gpt-4o-mini` | OpenAI model used when the Anthropic request specifies a haiku model. | -| `RUST_LOG` | `info` | Tracing filter. Examples: `debug`, `anthropic_openai_proxy=trace`. | +| `RUST_LOG` | `info` | Tracing filter. Examples: `debug`, `anyllm_proxy=trace`. | ## mTLS Client Certificates @@ -44,5 +44,5 @@ OPENAI_BASE_URL=https://internal-llm.corp.example.com \ TLS_CLIENT_CERT_P12=/etc/proxy/client.p12 \ TLS_CLIENT_CERT_PASSWORD=changeit \ TLS_CA_CERT=/etc/proxy/corp-ca.pem \ -cargo run -p anthropic_openai_proxy +cargo run -p anyllm_proxy ``` diff --git a/docs/dependencies.md b/docs/dependencies.md index 3845c4a..10ffdc2 100644 --- a/docs/dependencies.md +++ b/docs/dependencies.md @@ -1,6 +1,6 @@ # Dependency Versions -## Translator Crate (`anthropic_openai_translate`) +## Translator Crate (`anyllm_translate`) | Dependency | Version | Features | Purpose | |---|---|---|---| @@ -14,11 +14,11 @@ |---|---|---| | pretty_assertions | 1.x | Readable test diffs | -## Proxy Crate (`anthropic_openai_proxy`) +## Proxy Crate (`anyllm_proxy`) | Dependency | Version | Features | Purpose | |---|---|---|---| -| anthropic_openai_translate | path | - | Translation logic | +| anyllm_translate | path | - | Translation logic | | axum | 0.8 | - | HTTP server framework | | tokio | 1.x | full | Async runtime | | reqwest | 0.12 | json, stream | HTTP client | diff --git a/docs/library-integration.md b/docs/library-integration.md index 2bcd8a7..c44fe42 100644 --- a/docs/library-integration.md +++ b/docs/library-integration.md @@ -1,6 +1,6 @@ # Library Integration Guide -Research deliverable for Phase 21b. Evaluates how non-Rust consumers can use the `anthropic_openai_translate` crate's translation logic without running the proxy as a separate process. +Research deliverable for Phase 21b. Evaluates how non-Rust consumers can use the `anyllm_translate` crate's translation logic without running the proxy as a separate process. ## Overview @@ -147,7 +147,7 @@ autogen_warning = "/* Warning: this file is auto-generated by cbindgen. */" exclude = [] # only extern "C" functions are exported ``` -Build: `cbindgen --config cbindgen.toml --crate anthropic_openai_translate --output anthropic_openai_translate.h` +Build: `cbindgen --config cbindgen.toml --crate anyllm_translate --output anyllm_translate.h` ### Language Binding Examples @@ -157,7 +157,7 @@ Build: `cbindgen --config cbindgen.toml --crate anthropic_openai_translate --out import ctypes import json -lib = ctypes.CDLL("./target/release/libanthropic_openai_translate.so") +lib = ctypes.CDLL("./target/release/libanyllm_translate.so") lib.translate_request_ffi.restype = ctypes.c_char_p lib.translate_request_ffi.argtypes = [ctypes.c_char_p, ctypes.c_char_p] lib.translate_free_string.argtypes = [ctypes.c_char_p] @@ -175,7 +175,7 @@ if result_ptr: ```javascript const koffi = require('koffi'); -const lib = koffi.load('./target/release/libanthropic_openai_translate.so'); +const lib = koffi.load('./target/release/libanyllm_translate.so'); const translate_request = lib.func('const char* translate_request_ffi(const char*, const char*)'); const free_string = lib.func('void translate_free_string(char*)'); @@ -188,8 +188,8 @@ const parsed = JSON.parse(result); **Go (cgo):** ```go -// #cgo LDFLAGS: -L./target/release -lanthropic_openai_translate -// #include "anthropic_openai_translate.h" +// #cgo LDFLAGS: -L./target/release -lanyllm_translate +// #include "anyllm_translate.h" import "C" import "unsafe" @@ -414,7 +414,7 @@ pyo3 = { version = "0.22", features = ["extension-module"], optional = true } use pyo3::prelude::*; use pyo3::exceptions::PyValueError; -pyo3::create_exception!(anthropic_openai_translate, TranslateError, pyo3::exceptions::PyException); +pyo3::create_exception!(anyllm_translate, TranslateError, pyo3::exceptions::PyException); #[pyfunction] fn translate_request(config_json: &str, request_json: &str) -> PyResult { @@ -466,7 +466,7 @@ impl StreamingTranslator { } #[pymodule] -fn anthropic_openai_translate(m: &Bound<'_, PyModule>) -> PyResult<()> { +fn anyllm_translate(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_function(wrap_pyfunction!(translate_request, m)?)?; m.add_function(wrap_pyfunction!(translate_response, m)?)?; m.add_class::()?; @@ -478,7 +478,7 @@ fn anthropic_openai_translate(m: &Bound<'_, PyModule>) -> PyResult<()> { ### Type Stubs ```python -# anthropic_openai_translate.pyi +# anyllm_translate.pyi def translate_request(config_json: str, request_json: str) -> str: ... def translate_response(response_json: str, original_model: str) -> str: ... @@ -510,7 +510,7 @@ maturin publish --features python ```python import json -import anthropic_openai_translate as translator +import anyllm_translate as translator config = json.dumps({ "model_map": [["haiku", "gpt-4o-mini"], ["sonnet", "gpt-4o"]], @@ -545,9 +545,9 @@ final_events = json.loads(stream.finish()) A thin Python wrapper can accept/return dicts instead of JSON strings, hiding the serialization: ```python -# anthropic_openai_translate/convenience.py (pure Python, wraps native module) +# anyllm_translate/convenience.py (pure Python, wraps native module) import json -import anthropic_openai_translate._native as _native +import anyllm_translate._native as _native def translate_request(config: dict, request: dict) -> dict: return json.loads(_native.translate_request(json.dumps(config), json.dumps(request))) diff --git a/docs/superpowers/plans/2026-03-22-phase22-graceful-shutdown-ratelimit-fixtures-logging.md b/docs/superpowers/plans/2026-03-22-phase22-graceful-shutdown-ratelimit-fixtures-logging.md index 350bc51..cb5e985 100644 --- a/docs/superpowers/plans/2026-03-22-phase22-graceful-shutdown-ratelimit-fixtures-logging.md +++ b/docs/superpowers/plans/2026-03-22-phase22-graceful-shutdown-ratelimit-fixtures-logging.md @@ -25,8 +25,8 @@ Create `crates/proxy/tests/shutdown.rs`: ```rust // Test: server shuts down cleanly on signal, in-flight requests complete. -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; use std::time::Duration; fn test_config() -> Config { @@ -141,7 +141,7 @@ async fn new_connections_refused_after_shutdown() { - [ ] **Step 2: Run tests to verify they pass (validates the axum primitive)** -Run: `cargo test -p anthropic_openai_proxy --test shutdown -- --nocapture 2>&1` +Run: `cargo test -p anyllm_proxy --test shutdown -- --nocapture 2>&1` Expected: All 3 tests pass. These test the `axum::serve(...).with_graceful_shutdown()` API directly, not our `main.rs`. This validates the mechanism before we wire it into production code. The `main.rs` change (Step 3) applies the same pattern to the real server. - [ ] **Step 3: Update main.rs to use graceful shutdown** @@ -149,7 +149,7 @@ Expected: All 3 tests pass. These test the `axum::serve(...).with_graceful_shutd Replace `crates/proxy/src/main.rs`: ```rust -use anthropic_openai_proxy::{config, server::routes}; +use anyllm_proxy::{config, server::routes}; #[tokio::main] async fn main() { @@ -198,7 +198,7 @@ async fn shutdown_signal() { - [ ] **Step 4: Run all tests to verify nothing broke** -Run: `cargo test -p anthropic_openai_proxy 2>&1` +Run: `cargo test -p anyllm_proxy 2>&1` Expected: All tests pass (existing + new shutdown tests). - [ ] **Step 5: Run clippy** @@ -355,7 +355,7 @@ mod rate_limit_tests { - [ ] **Step 3: Run tests to verify they fail (struct doesn't exist yet)** -Run: `cargo test -p anthropic_openai_proxy rate_limit 2>&1` +Run: `cargo test -p anyllm_proxy rate_limit 2>&1` Expected: Compilation error. - [ ] **Step 4: Implement `RateLimitHeaders` in mod.rs** @@ -364,7 +364,7 @@ Add the struct, `from_openai_headers`, `inject_anthropic_headers`, and helper fu - [ ] **Step 5: Run unit tests** -Run: `cargo test -p anthropic_openai_proxy rate_limit 2>&1` +Run: `cargo test -p anyllm_proxy rate_limit 2>&1` Expected: All 4 rate_limit tests pass. - [ ] **Step 6: Modify OpenAI client to return rate limit headers** @@ -552,7 +552,7 @@ Race condition analysis: `rl_rx.await` blocks until the spawned task sends rate - [ ] **Step 9: Run all tests** -Run: `cargo test -p anthropic_openai_proxy 2>&1` +Run: `cargo test -p anyllm_proxy 2>&1` Expected: All pass. The integration tests don't hit a real backend, so rate limit headers will be absent (default). The unit tests verify the mapping logic. - [ ] **Step 10: Commit** @@ -802,7 +802,7 @@ Create `crates/proxy/tests/error_fixtures.rs`: #[test] fn malformed_openai_response_fails_deserialization() { let json = include_str!("../../../fixtures/openai/chat_completion_malformed.json"); - let result = serde_json::from_str::(json); + let result = serde_json::from_str::(json); assert!(result.is_err(), "malformed response should fail deserialization"); } ``` @@ -915,8 +915,8 @@ Add response logging at each success path in both non-streaming branches. For st Create `crates/proxy/tests/body_logging.rs`: ```rust -use anthropic_openai_proxy::config::{self, Config}; -use anthropic_openai_proxy::server::routes; +use anyllm_proxy::config::{self, Config}; +use anyllm_proxy::server::routes; fn test_config_with_logging() -> Config { Config { @@ -994,7 +994,7 @@ Update TASKS.md Phase 22 section: - [x] Request/response logging toggle (opt-in body logging for debugging, redacted by default) - [ ] OpenAI Responses API backend: wire up `ResponsesRequest`/`ResponsesResponse` types with runtime backend selection - [ ] Live API integration tests (requires OPENAI_API_KEY, currently golden fixtures only) -- [ ] Publish `anthropic_openai_translate` crate to crates.io +- [ ] Publish `anyllm_translate` crate to crates.io ``` - [ ] **Step 2: Run full test suite** diff --git a/tasks/prd-anthropic-domain-types.md b/tasks/prd-anthropic-domain-types.md index 5fea5d8..ab00e60 100644 --- a/tasks/prd-anthropic-domain-types.md +++ b/tasks/prd-anthropic-domain-types.md @@ -70,7 +70,7 @@ Define the Rust types that model the Anthropic Messages API surface: request, re - [ ] Golden fixture files in `fixtures/anthropic/`: `messages_basic.json`, `messages_tool_use.json`, `messages_image.json`, `messages_document.json` - [ ] Tests that deserialize each fixture into typed structs and re-serialize, comparing output - [ ] Test that malformed requests (missing `max_tokens`, invalid `role`) are rejected by serde -- [ ] `cargo test -p anthropic_openai_translate` passes +- [ ] `cargo test -p anyllm_translate` passes ## Functional Requirements @@ -98,4 +98,4 @@ Define the Rust types that model the Anthropic Messages API surface: request, re - All fixture files deserialize without error - Round-trip (deserialize then serialize) produces semantically identical JSON -- `cargo test -p anthropic_openai_translate` passes with zero failures +- `cargo test -p anyllm_translate` passes with zero failures diff --git a/tasks/prd-non-streaming-translation.md b/tasks/prd-non-streaming-translation.md index e1fdb41..462094e 100644 --- a/tasks/prd-non-streaming-translation.md +++ b/tasks/prd-non-streaming-translation.md @@ -115,4 +115,4 @@ Implement the core translation logic that converts Anthropic Messages API reques - All mapping functions have corresponding unit tests - Golden fixture tests: load Anthropic fixture, translate to OpenAI, compare against OpenAI fixture -- `cargo test -p anthropic_openai_translate` passes with all new tests green +- `cargo test -p anyllm_translate` passes with all new tests green diff --git a/tasks/prd-openai-domain-types.md b/tasks/prd-openai-domain-types.md index 0418c6e..ded2537 100644 --- a/tasks/prd-openai-domain-types.md +++ b/tasks/prd-openai-domain-types.md @@ -71,7 +71,7 @@ Define the Rust types that model the OpenAI Chat Completions and Responses APIs: - [ ] Fixture files in `fixtures/openai/`: `chat_completion_basic.json`, `chat_completion_tool_call.json`, `responses_basic.json` - [ ] Tests that deserialize each fixture into typed structs without error - [ ] Round-trip serialization produces semantically identical JSON -- [ ] `cargo test -p anthropic_openai_translate` passes +- [ ] `cargo test -p anyllm_translate` passes ## Functional Requirements @@ -99,4 +99,4 @@ Define the Rust types that model the OpenAI Chat Completions and Responses APIs: - All fixture files deserialize without error - Round-trip serde produces equivalent JSON -- `cargo test -p anthropic_openai_translate` passes +- `cargo test -p anyllm_translate` passes diff --git a/tasks/prd-project-scaffolding.md b/tasks/prd-project-scaffolding.md index 42eb153..58540a8 100644 --- a/tasks/prd-project-scaffolding.md +++ b/tasks/prd-project-scaffolding.md @@ -22,16 +22,16 @@ Set up the Cargo workspace, dependencies, and directory structure for the Anthro - [ ] `cargo test` runs (even if no tests yet) ### US-002: Scaffold translator crate -**Description:** As a developer, I need the `anthropic_openai_translate` library crate with the module structure defined in PLAN.md so subsequent phases have a place to add types and mapping logic. +**Description:** As a developer, I need the `anyllm_translate` library crate with the module structure defined in PLAN.md so subsequent phases have a place to add types and mapping logic. **Acceptance Criteria:** - [ ] `crates/translator/Cargo.toml` with `serde`, `serde_json`, `uuid`, `thiserror` dependencies - [ ] `src/lib.rs` with module declarations for `anthropic`, `openai`, `mapping`, `util` - [ ] Subdirectories: `anthropic/`, `openai/`, `mapping/`, `util/` with `mod.rs` stubs -- [ ] `cargo build -p anthropic_openai_translate` succeeds +- [ ] `cargo build -p anyllm_translate` succeeds ### US-003: Scaffold proxy crate -**Description:** As a developer, I need the `anthropic_openai_proxy` binary crate with axum server skeleton and a health endpoint so I can verify the server starts. +**Description:** As a developer, I need the `anyllm_proxy` binary crate with axum server skeleton and a health endpoint so I can verify the server starts. **Acceptance Criteria:** - [ ] `crates/proxy/Cargo.toml` with `tokio`, `axum`, `reqwest`, `tracing`, `tracing-subscriber` dependencies @@ -59,7 +59,7 @@ Set up the Cargo workspace, dependencies, and directory structure for the Anthro - FR-1: Workspace builds with `cargo build` producing no errors or warnings - FR-2: `cargo test` runs and passes (even with zero test assertions initially) -- FR-3: `cargo run -p anthropic_openai_proxy` starts a server on the configured port +- FR-3: `cargo run -p anyllm_proxy` starts a server on the configured port - FR-4: `GET /health` returns HTTP 200 with JSON body `{"status":"ok"}` - FR-5: Environment variables `OPENAI_API_KEY`, `OPENAI_BASE_URL`, `LISTEN_PORT` are read with sensible defaults diff --git a/tasks/prd-tool-calling-translation.md b/tasks/prd-tool-calling-translation.md index 5424b4f..e0ccadc 100644 --- a/tasks/prd-tool-calling-translation.md +++ b/tasks/prd-tool-calling-translation.md @@ -89,4 +89,4 @@ Implement the conversation history translation for tool calling: converting Anth - Golden fixture test: multi-turn tool conversation from PLAN.md lines 273-386 translates correctly in both directions - Round-trip: Anthropic tool conversation -> OpenAI -> back to Anthropic produces equivalent structure -- `cargo test -p anthropic_openai_translate` passes with all new tests +- `cargo test -p anyllm_translate` passes with all new tests