mirror of
https://github.com/whit3rabbit/anyllm-proxy.git
synced 2026-09-22 16:00:51 +00:00
whit3rabbitandClaude Sonnet 4.6
816b64ae6a
New providers: assemblyai, baidu, blackboxai, brave, cartesia, deepgram, elevenlabs, exa, iflytek, lmsys, playht, pollinations, serper, siliconflow, stability, tavily. Registered in mod.rs and registry.rs. Adds a Providers tab to the admin UI with a ProviderIcon component and catalog route. Updates backend, streaming, config, and test infrastructure to align with the expanded provider set. Bumps all crate versions to 0.9.0. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
753 B
753 B
LMSYS (FastChat)
Self-hosted FastChat server for open models (Vicuna, etc.). Point api_base at your FastChat instance.
LiteLLM prefix: (not in LiteLLM)
Status: Stub — routes through OpenAI-compatible client
Default API base: http://localhost:8000/v1 (override via managed backend api_base)
Docs: https://github.com/lm-sys/FastChat
Authentication
No API key required by default. FastChat servers can optionally require a key.
Quick Start
# Start FastChat first, then:
BACKEND=lmsys PROXY_OPEN_RELAY=true cargo run -p anyllm_proxy
Capabilities
| Feature | Supported |
|---|---|
| Chat Completions | ✓ |
| Streaming | ✓ |
| Tool Use | — |
| Embeddings | — |
| Vision | — |
| Batch | — |