mlx-v is a Rust candle-backed vision-language inference toolkit whose
`vlm serve` exposes an OpenAI-compatible endpoint, defaulting to
localhost:8080. Registering it means the proxy can front a local VLM the same
way it fronts LM Studio or Ollama, which in practice means Anthropic
/v1/messages against a local candle model.
Capabilities differ from the neighbouring local backends in two ways worth
naming: embeddings is false (mlx-v has no /v1/embeddings route, unlike
lm_studio and ollama which advertise one), and tool_choice is false (mlx-v
honors "auto" and "none" but rejects "required" and named functions, having
no constrained decoding).
No litellm_snapshot change: mlx-v is not in LiteLLM, which is what the
LEGACY_ONLY_* arrays are for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
anyllm-proxy supports 74 providers. All OpenAI-compatible providers route through a single HTTP client — adding a new provider is metadata-only (no new HTTP code).
Usage Patterns
Single-backend — set BACKEND=<id> and the provider's API key env var:
BACKEND=groq GROQ_API_KEY=your-key cargo run -p anyllm_proxy
Multi-backend (LiteLLM YAML) — set PROXY_CONFIG=config.yaml:
whit3rabbitandClaude Opus 5
8f778eb1d0