whit3rabbitandClaude Opus 4.6 d6b9d61071 feat: add caching, batch API, cost tracking, budget/RBAC, audio/image passthrough ...

LiteLLM parity features:
- Response caching (in-memory moka, optional Redis tier) with per-request TTL
- Batch API (file upload, job creation/listing via OpenAI delegation)
- Per-key budget enforcement (daily/monthly/lifetime) with lazy period reset
- RBAC (admin/developer key roles, developer keys blocked from /admin/)
- Audio transcription/speech and image generation passthrough
- Fallback chain config (YAML-based, 5xx/429 failover)
- Cost tracking foundation (model pricing DB, per-key spend queries)

Code quality cleanup (simplify pass):
- Extract try_cache_response helper (was copy-pasted 4x)
- Extract common_routes for batch/models (was duplicated across 3 HandlerMode arms)
- Deduplicate embeddings_passthrough to delegate to raw_passthrough
- Remove dead code: inject_cost_header, BudgetDuration::from_str_lossy, duplicate CacheConfig
- Collapse epoch_to_ymd wrapper into pub(crate) days_to_ymd

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-26 20:23:38 -05:00

..

model_pricing.json

feat: add caching, batch API, cost tracking, budget/RBAC, audio/image passthrough

2026-03-26 20:23:38 -05:00