mirror of
https://github.com/whit3rabbit/anyllm-proxy.git
synced 2026-09-22 00:00:50 +00:00
whit3rabbitandClaude Opus 5
8f778eb1d0
mlx-v is a Rust candle-backed vision-language inference toolkit whose `vlm serve` exposes an OpenAI-compatible endpoint, defaulting to localhost:8080. Registering it means the proxy can front a local VLM the same way it fronts LM Studio or Ollama, which in practice means Anthropic /v1/messages against a local candle model. Capabilities differ from the neighbouring local backends in two ways worth naming: embeddings is false (mlx-v has no /v1/embeddings route, unlike lm_studio and ollama which advertise one), and tool_choice is false (mlx-v honors "auto" and "none" but rejects "required" and named functions, having no constrained decoding). No litellm_snapshot change: mlx-v is not in LiteLLM, which is what the LEGACY_ONLY_* arrays are for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>