starred/anyllm-proxy
1
0
Fork 0
mirror of https://github.com/whit3rabbit/anyllm-proxy.git synced 2026-09-22 08:00:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
194 Commits 14 Branches 21 Tags
8ac9ef2677c14fc024494d846894de3c086510a5
Commit Graph
12 Commits
Author SHA1 Message Date
whit3rabbitandClaude Sonnet 4.6 8ac9ef2677 Merge feat/batch-engine-phase1 into main 
Resolves conflicts: take HEAD (security audit) for mcp.rs imports,
register_server_blocking error handling, and maybe_execute_tools loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-31 16:00:37 -05:00
whit3rabbitandClaude Opus 4.6 66864d8198 docs: Phase 1 implementation plan for batch orchestration engine 
12 tasks covering: crate scaffold, core types, error types, JSONL
validation migration, DB schema + file storage, JobQueue trait +
SqliteQueue, webhook queue + dispatcher, BatchEngine facade, build
verification, proxy wiring, integration tests, and final verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 22:02:09 -05:00
whit3rabbitandClaude Opus 4.6 2d04a2e95f docs: apply review feedback to batch orchestration spec 
- dispatch() is now async (webhook enqueue is async)
- All event emission routed through NotificationManager, not direct
  event_bus.emit() calls
- Native batch poller emits Started/Progress events during polling
- SSE handler sends snapshot event on connect for late subscribers
- WebhookQueue trait gets reclaim_expired_leases() for stuck deliveries
- WebhookDispatcher runs lease reclaim loop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 21:52:47 -05:00
whit3rabbitandClaude Opus 4.6 8378065ca0 docs: batch orchestration engine design spec 
Unified batch orchestration layer with hybrid execution (provider-native
delegation + proxy-native item processing), SQLite-backed job queue,
in-process workers with graceful shutdown, and event-driven notifications
(webhooks + SSE). New batch_engine crate with enforced boundaries for
future worker extraction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 21:50:08 -05:00
whit3rabbitandClaude Opus 4.6 6ceee1edd0 docs: add tool execution engine implementation plan 
14 tasks covering: policy types, trace types, execution engine core,
config parsing, MCP server manager, MCP admin endpoints, handler
integration (non-streaming + streaming), and integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 21:42:39 -05:00
whit3rabbitandClaude Opus 4.6 35178f5a58 docs: add tool execution engine design spec 
Covers bounded execution loop, tool execution policy, MCP integration
(SSE transport), failure modeling, loop trace observability, and config.
Phase C (single-round, max_iterations=1) with clean upgrade path to
Phase B (multi-round agentic loop).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 19:33:03 -05:00
whit3rabbitandClaude Opus 4.6 b6277a8b86 feat: operator observability dashboard with error classification 
- Admin UI: operator view with request volume, token usage, latency,
  cost charts, failure table, and request timeline
- Backend: add error_kind() method and infer_error_kind() for stable
  error classification (rate_limit, timeout, backend_error, client_error)
- Admin DB: error_kind column in request_log, observability aggregate
  queries (bucketed timeseries, failure breakdown, timeline)
- Gemini: improved streaming translation, thinking block support,
  grounding metadata passthrough
- Request timeout: configurable REQUEST_TIMEOUT_SECS with streaming
  watchdog
- Model pricing: MODEL_PRICING_FILE for external pricing overrides
- Degradation header: ANYLLM_DEGRADATION_WARNINGS env var control

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-30 19:12:03 -05:00
whit3rabbit 373d55d36e security: warn at startup when serving API keys over plaintext HTTP 
Add warning logs when the proxy listener is bound to a non-loopback
address and either PROXY_API_KEYS is configured or virtual keys are
loaded from the database. Warns operators to place a TLS-terminating
reverse proxy in front of the service to protect credentials.

Does not block startup; purely informational.
 2026-03-27 15:52:40 -05:00
whit3rabbitandClaude Opus 4.6 c82d3529e2 feat: audit log, model allowlist policy, and Phase 2 plan 
- Audit log table with event recording for admin mutations
- Per-key model allowlist policy (exact match + prefix wildcard)
- Phase 2 implementation plan document
- Formatting fixes from parallel agent work

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 07:37:45 -05:00
whit3rabbitandClaude Opus 4.6 727083b2ad docs: add Phase 1 implementation plan and AGENTS.md 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 06:53:24 -05:00
whit3rabbitandClaude Sonnet 4.6 f1df50ff37 refactor: extract shared client crate and rename to anyllm_* 
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.

Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.

proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.

streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.

Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-25 06:11:01 -05:00
whit3rabbitandClaude Opus 4.6 0a9ca1ae31 fix: simplify retry loop, sanitize request IDs, and apply rustfmt (Phase 22) 
Collapse retry + final-attempt into single inclusive loop, drain response
body before retry to return connections to pool, and replace panic on
invalid x-request-id with UUID fallback. Update CLAUDE.md/TASKS.md for
Phase 22 status. Apply cargo fmt across touched files.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-22 13:41:06 -05:00