mirror of
https://github.com/whit3rabbit/anyllm-proxy.git
synced 2026-09-22 00:00:50 +00:00
whit3rabbitandClaude Sonnet 4.6
545bdbbbd3
Adds bedrock_native.rs (Converse/InvokeModel with SigV4) and generic_passthrough.rs catch-all for Translate mode. Adds comprehensive provider reference docs (docs/providers/, docs/ENDPOINTS.md). Fixes managed backend admin UI (BackendForm, ManagedBackendsSection) and admin route/model handler issues. Adds automated model pricing update workflow (scripts/update_pricing.py, .github/workflows/update-pricing.yml). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Provider Documentation
anyllm-proxy supports 74 providers. All OpenAI-compatible providers route through a single HTTP client — adding a new provider is metadata-only (no new HTTP code).
Usage Patterns
Single-backend — set BACKEND=<id> and the provider's API key env var:
BACKEND=groq GROQ_API_KEY=your-key cargo run -p anyllm_proxy
Multi-backend (LiteLLM YAML) — set PROXY_CONFIG=config.yaml:
model_list:
- model_name: fast
litellm_params:
model: groq/llama-3.3-70b-versatile
api_key: "env:GROQ_API_KEY"
- model_name: smart
litellm_params:
model: anthropic/claude-3-5-sonnet-20241022
api_key: "env:ANTHROPIC_API_KEY"
Provider Index
Implemented (fully live-tested)
| Provider | ID | Docs |
|---|---|---|
| OpenAI | openai |
openai.md |
| Anthropic | anthropic |
anthropic.md |
| Google AI Studio | gemini |
gemini.md |
Wired (HTTP client built, not live-tested)
| Provider | ID | Docs |
|---|---|---|
| Google Vertex AI | vertex_ai |
vertex_ai.md |
| Azure OpenAI | azure |
azure.md |
| AWS Bedrock | bedrock |
bedrock.md |
Stub — Cloud (OpenAI-compatible)
| Provider | ID | Docs |
|---|---|---|
| xAI | xai |
xai.md |
| Groq | groq |
groq.md |
| Together AI | together_ai |
together_ai.md |
| OpenRouter | openrouter |
openrouter.md |
| Fireworks AI | fireworks_ai |
fireworks_ai.md |
| Mistral AI | mistral |
mistral.md |
| Codestral | codestral |
codestral.md |
| Perplexity AI | perplexity |
perplexity.md |
| DeepSeek | deepseek |
deepseek.md |
| Cohere | cohere_chat |
cohere_chat.md |
| Cerebras | cerebras |
cerebras.md |
| SambaNova | sambanova |
sambanova.md |
| Nebius AI Studio | nebius |
nebius.md |
| DeepInfra | deepinfra |
deepinfra.md |
| Novita AI | novita |
novita.md |
| Databricks | databricks |
databricks.md |
| Anyscale | anyscale |
anyscale.md |
| HuggingFace | huggingface |
huggingface.md |
| AI21 Labs | ai21 |
ai21.md |
| NVIDIA NIM | nvidia_nim |
nvidia_nim.md |
| Moonshot AI | moonshot |
moonshot.md |
| Volcano Engine | volcengine |
volcengine.md |
| MiniMax | minimax |
minimax.md |
| Zhipu AI | zhipuai |
zhipuai.md |
| Featherless AI | featherless_ai |
featherless_ai.md |
| FriendliAI | friendliai |
friendliai.md |
| Lambda AI | lambda_ai |
lambda_ai.md |
| Hyperbolic | hyperbolic |
hyperbolic.md |
| Nscale | nscale |
nscale.md |
| GitHub Models | github |
github.md |
| Aleph Alpha | aleph_alpha |
aleph_alpha.md |
| NLP Cloud | nlp_cloud |
nlp_cloud.md |
| Clarifai | clarifai |
clarifai.md |
| Predibase | predibase |
predibase.md |
| Replicate | replicate |
replicate.md |
| Chutes AI | chutes |
chutes.md |
| GMI Cloud | gmi_cloud |
gmi_cloud.md |
| Meta Llama API | meta_llama |
meta_llama.md |
| AI/ML API | ai_ml_api |
ai_ml_api.md |
| Voyage AI | voyage |
voyage.md |
| Scaleway | scaleway |
scaleway.md |
| Baseten | baseten |
baseten.md |
| Dashscope (Qwen) | dashscope |
dashscope.md |
| Jina AI | jina |
jina.md |
| OVHCloud | ovhcloud |
ovhcloud.md |
| Gradient AI | gradient_ai |
gradient_ai.md |
| Galadriel | galadriel |
galadriel.md |
| Morph | morph |
morph.md |
| Xiaomi MiMo | xiaomi_mimo |
xiaomi_mimo.md |
| PublicAI | public_ai |
public_ai.md |
| NanoGPT | nanogpt |
nanogpt.md |
| W&B Inference | wandb |
wandb.md |
| Bytez | bytez |
bytez.md |
Stub — Per-Instance URL (requires api_base or OPENAI_BASE_URL)
| Provider | ID | Docs |
|---|---|---|
| Azure AI Foundry | azure_ai |
azure_ai.md |
| IBM WatsonX | watsonx |
watsonx.md |
| Cloudflare Workers AI | cloudflare |
cloudflare.md |
| Snowflake Cortex | snowflake |
snowflake.md |
Stub — Not Yet Routable
| Provider | ID | Docs | Reason |
|---|---|---|---|
| AWS SageMaker | sagemaker |
sagemaker.md | Custom SigV4 protocol, no HTTP client |
Stub — Local / Self-Hosted
| Provider | ID | Docs |
|---|---|---|
| Ollama | ollama |
ollama.md |
| vLLM | hosted_vllm |
hosted_vllm.md |
| LM Studio | lm_studio |
lm_studio.md |
| llamafile | llamafile |
llamafile.md |
| Xinference | xinference |
xinference.md |
| Petals | petals |
petals.md |
| NVIDIA Triton | triton |
triton.md |
| Infinity | infinity |
infinity.md |
| Lemonade | lemonade |
lemonade.md |
| Docker Model Runner | docker_model_runner |
docker_model_runner.md |