Commit Graph
14 Commits
Author SHA1 Message Date
whit3rabbitandClaude Sonnet 4.6 8ac9ef2677 Merge feat/batch-engine-phase1 into main
Resolves conflicts: take HEAD (security audit) for mcp.rs imports,
register_server_blocking error handling, and maybe_execute_tools loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 16:00:37 -05:00
whit3rabbitandClaude Sonnet 4.6 bdf73c9057 test(admin): update virtual_keys tests for one-time CSRF tokens
CSRF tokens are now one-time use. Tests updated to:
- Pre-insert TEST_CSRF_TOKEN in test_admin_router() for unit tests
- Call reinsert_csrf() before each additional mutation in multi-step tests
- Add fetch_csrf() helper for real-server tests
- update create_key_via_admin() to fetch a fresh token per call

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:48:44 -05:00
whit3rabbit 9803668b7d feat(proxy): wire batch_engine into proxy crate
- Replace batch/mod.rs types with re-exports from anyllm_batch_engine
- Strip batch/db.rs to Anthropic->OpenAI ID mapping only (batch_file/batch_job owned by engine)
- Rewrite batch/routes.rs to use BatchEngine for upload, create, get, list, cancel
- Remove batch_file/batch_job table creation from admin/db.rs init_db
- Add batch_engine parameter to app_multi_with_shared (5th arg, Option<Arc<BatchEngine>>)
- Initialize BatchEngine in main.rs with its own SQLite connection (admin-enabled path)
- Update batch_api.rs tests to use make_test_batch_engine() helper
- Fix anthropic_batch.rs to call init_anthropic_batch_map_table instead of removed init_batch_tables
- Add cancel_queued_batch integration test
2026-03-31 06:48:16 -05:00
whit3rabbitandClaude Sonnet 4.6 3a62796a75 feat(tools): wire ToolEngineState into AppState with config-driven setup
Add ToolEngineState struct (registry, policy, loop_config, mcp_manager)
and tool_engine field to AppState. Update app_multi_with_shared signature
to accept the new parameter; all callers pass None until config-driven
wiring is implemented in a future task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:07:23 -05:00
whit3rabbitandClaude Opus 4.6 b6277a8b86 feat: operator observability dashboard with error classification
- Admin UI: operator view with request volume, token usage, latency,
  cost charts, failure table, and request timeline
- Backend: add error_kind() method and infer_error_kind() for stable
  error classification (rate_limit, timeout, backend_error, client_error)
- Admin DB: error_kind column in request_log, observability aggregate
  queries (bucketed timeseries, failure breakdown, timeline)
- Gemini: improved streaming translation, thinking block support,
  grounding metadata passthrough
- Request timeout: configurable REQUEST_TIMEOUT_SECS with streaming
  watchdog
- Model pricing: MODEL_PRICING_FILE for external pricing overrides
- Degradation header: ANYLLM_DEGRADATION_WARNINGS env var control

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:12:03 -05:00
whit3rabbitandClaude Sonnet 4.6 f15af5969b fix(config): add expose_degradation_warnings to all Config struct literals in tests
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 17:06:35 -05:00
whit3rabbitandClaude Sonnet 4.6 4fbaf43011 fix(security): populate source_ip in all admin audit log entries
Add ConnectInfo<SocketAddr> extractor to all seven admin mutation
handlers (put_config, delete_config_override, create_key, update_key,
revoke_key, add_model, remove_model). Pass addr.ip().to_string() as
source_ip in each emit_audit call.

Update integration tests to use into_make_service_with_connect_info
and MockConnectInfo so the ConnectInfo extractor is satisfied in test
servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:05 -05:00
whit3rabbitandClaude Opus 4.6 5c73adfaea feat(admin): team-focused UI overhaul with key editing and audit log
Backend:
- PUT /admin/api/keys/{id}: update virtual key fields (role immutable),
  refreshes DashMap, emits key_updated audit entry
- update_virtual_key(): resets spend period when budget_duration changes
- query_audit_log(): action, target_type, since, until filter params
- query_request_log(): add until filter param; wire through RequestsQuery
- GET /admin/api/metrics: include streaming counters (started/completed/failed/disconnected)
- GET /admin/api/env: include RATE_LIMIT_FAIL_POLICY

UI:
- Dashboard: streaming metrics stat row
- Request Log: key filter dropdown, since/until date pickers, Key column
- Access Control tab: allowed_models tag input, edit modal, budget progress
  bars, key prefix link navigates to filtered request log
- Settings: override badges from overridden_keys, Security section
  (IP allowlist, rate limit fail policy)
- Audit tab: action/target/date filters, paginated table

Tests:
- 7 unit tests for update_virtual_key and query_audit_log filters
- 4 integration tests for PUT /admin/api/keys/{id} lifecycle

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 18:08:34 -05:00
whit3rabbit 6cd37068e3 feat(admin): add CSRF token middleware and /admin/csrf-token endpoint
- validate_csrf middleware: rejects POST/PUT/DELETE without matching X-CSRF-Token header + csrf_token cookie
- GET /admin/csrf-token: public endpoint returning JSON + Set-Cookie (not httpOnly, SameSite=Strict)
- CSRF middleware layered inside validate_admin_token (auth checked first)
- 7 new unit tests in routes.rs for CSRF behavior
- Updated virtual_keys.rs integration tests to include CSRF headers on all POST/DELETE admin calls
2026-03-27 23:45:03 -05:00
whit3rabbitandClaude Sonnet 4.6 f3f5179a64 security: add rate limiting to admin API endpoints (10 RPM per IP)
Defense-in-depth against token brute-force on the admin API. Uses a
DashMap-based sliding window (60s) per client IP, applied as the
outermost middleware layer on protected admin routes. Admin server now
uses into_make_service_with_connect_info to expose client IP. Limit
is configurable at runtime via set_admin_rpm for test flexibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 15:45:37 -05:00
whit3rabbitandClaude Sonnet 4.6 915a703a5a security: HMAC-SHA256 keyed hashing for virtual keys with dual-mode migration
New virtual keys are hashed with HMAC-SHA256 using a per-installation
secret (auto-generated and stored in SQLite settings table). Auth
middleware tries HMAC hash first, falls back to legacy SHA-256 for
pre-existing keys. This binds key hashes to the installation, so a
stolen database cannot be used to brute-force keys elsewhere.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 15:14:34 -05:00
whit3rabbitandClaude Sonnet 4.6 25279835a7 feat: add LiteLLM config.yaml compatibility and model-level routing
Accept LiteLLM config.yaml directly via PROXY_CONFIG=config.yaml. Parses
model_list with provider/model format, supports multiple deployments per
model name with round-robin + RPM-aware load balancing, cross-backend
dispatch, and os.environ/VAR env var syntax. Adds env var aliases for
LITELLM_MASTER_KEY, AZURE_API_KEY, AZURE_API_BASE, LITELLM_CONFIG.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 05:39:51 -05:00
whit3rabbitandClaude Opus 4.6 d6b9d61071 feat: add caching, batch API, cost tracking, budget/RBAC, audio/image passthrough
LiteLLM parity features:
- Response caching (in-memory moka, optional Redis tier) with per-request TTL
- Batch API (file upload, job creation/listing via OpenAI delegation)
- Per-key budget enforcement (daily/monthly/lifetime) with lazy period reset
- RBAC (admin/developer key roles, developer keys blocked from /admin/)
- Audio transcription/speech and image generation passthrough
- Fallback chain config (YAML-based, 5xx/429 failover)
- Cost tracking foundation (model pricing DB, per-key spend queries)

Code quality cleanup (simplify pass):
- Extract try_cache_response helper (was copy-pasted 4x)
- Extract common_routes for batch/models (was duplicated across 3 HandlerMode arms)
- Deduplicate embeddings_passthrough to delegate to raw_passthrough
- Remove dead code: inject_cost_header, BudgetDuration::from_str_lossy, duplicate CacheConfig
- Collapse epoch_to_ymd wrapper into pub(crate) days_to_ymd

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 20:23:38 -05:00
whit3rabbitandClaude Sonnet 4.6 0942072b29 chore: docs, CI, and polish for litellm gap fill
Update CLAUDE.md and README.md to document new backends (Azure, Bedrock),
virtual key management, rate limiting, OTEL, and client v0.2.0. Add CI
workflow for the new integration test suite. Final task-list completions
and virtual key integration tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 06:36:09 -05:00