Commit Graph
194 Commits
Author SHA1 Message Date
whit3rabbitandClaude Sonnet 4.6 8ac9ef2677 Merge feat/batch-engine-phase1 into main
Resolves conflicts: take HEAD (security audit) for mcp.rs imports,
register_server_blocking error handling, and maybe_execute_tools loop.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 16:00:37 -05:00
whit3rabbitandClaude Sonnet 4.6 e6dd396b05 fix(security): apply 2026-03-31 security audit hardening
- Use getrandom for CSPRNG HMAC secret generation
- Case-insensitive admin path check prevents bypass via /Admin/, /ADMIN/
- Add TRUSTED_PROXY_DEPTH for multi-hop proxy X-Forwarded-For extraction
- Add dangerous-builtin-tools feature flag with security warning
- Remove repomix-output.xml, add .syntext to .gitignore

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 15:56:30 -05:00
whit3rabbitandClaude Sonnet 4.6 ef5f0f596c refactor(proxy): wire batch handlers through BatchEngine crate
Proxy batch handlers now use BatchEngine for job lifecycle, file storage,
and webhook delivery instead of direct SQLite calls. Old batch/db.rs
stripped to Anthropic-specific mapping only. Cancel endpoint at
POST /v1/batches/{id}/cancel. BatchEngine initialized in main.rs startup
with second SQLite connection. Cancel integration test added.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:49:40 -05:00
whit3rabbitandClaude Sonnet 4.6 bdf73c9057 test(admin): update virtual_keys tests for one-time CSRF tokens
CSRF tokens are now one-time use. Tests updated to:
- Pre-insert TEST_CSRF_TOKEN in test_admin_router() for unit tests
- Call reinsert_csrf() before each additional mutation in multi-step tests
- Add fetch_csrf() helper for real-server tests
- update create_key_via_admin() to fetch a fresh token per call

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:48:44 -05:00
whit3rabbit 9803668b7d feat(proxy): wire batch_engine into proxy crate
- Replace batch/mod.rs types with re-exports from anyllm_batch_engine
- Strip batch/db.rs to Anthropic->OpenAI ID mapping only (batch_file/batch_job owned by engine)
- Rewrite batch/routes.rs to use BatchEngine for upload, create, get, list, cancel
- Remove batch_file/batch_job table creation from admin/db.rs init_db
- Add batch_engine parameter to app_multi_with_shared (5th arg, Option<Arc<BatchEngine>>)
- Initialize BatchEngine in main.rs with its own SQLite connection (admin-enabled path)
- Update batch_api.rs tests to use make_test_batch_engine() helper
- Fix anthropic_batch.rs to call init_anthropic_batch_map_table instead of removed init_batch_tables
- Add cancel_queued_batch integration test
2026-03-31 06:48:16 -05:00
whit3rabbitandClaude Sonnet 4.6 8bfc04e583 fix(tools): warn at startup when execute_bash policy is Allow
Emits tracing::warn! when execute_bash is configured with policy: allow
so operators see an explicit reminder that this permits arbitrary OS
command execution. The dangerous-builtin-tools compile-time feature flag
remains the primary gate; this is an additional runtime visibility measure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:43:16 -05:00
whit3rabbitandClaude Sonnet 4.6 2019f92824 fix(admin): enforce one-time CSRF tokens tracked server-side
GET /admin/csrf-token now stores the generated token in
SharedState::issued_csrf_tokens (DashMap). validate_csrf middleware
verifies the X-CSRF-Token header was server-issued and removes it on
first use, preventing replay of previously issued tokens across multiple
mutating requests. validate_csrf switched to from_fn_with_state to
receive SharedState. Adds test: post_with_unissued_csrf_returns_403.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:42:41 -05:00
whit3rabbitandClaude Sonnet 4.6 0efe5ae671 fix(mcp): use SSRF-safe HTTP client for MCP tool calls and discovery
Replaces reqwest::Client::new() in McpServerManager::new() and
discover_tools() with build_http_client(ssrf_protection: true), which
attaches SsrfSafeDnsResolver. This prevents DNS rebinding: a domain that
passes the registration-time check but later resolves to a private/metadata
IP (e.g. 169.254.169.254) is blocked at connection time by the resolver.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:38:25 -05:00
whit3rabbitandClaude Sonnet 4.6 52173ea8eb fix(admin): validate model_name/backend_name/actual_model in add_model
Applies is_safe_model_name to all three AddModelRequest fields before use,
preventing log injection via newlines or control characters in audit log
detail entries. Consistent with existing validation in put_config.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:37:08 -05:00
whit3rabbitandClaude Sonnet 4.6 060c0e418f fix(mcp): validate server names to prevent tool routing ambiguity
MCP tool names use mcp_{server}_{tool}; underscores in server names make
parse_mcp_tool_name ambiguous. is_valid_mcp_server_name rejects names
containing underscores (allows alphanumerics + hyphens only).
register_server_blocking now returns Result<(), String> so callers handle
invalid names explicitly. Callers in main.rs and admin routes updated.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:36:41 -05:00
whit3rabbitandClaude Sonnet 4.6 babd0e6ff0 fix(tools): enforce allowed_dirs allowlist in ReadFileTool
Adds allowed_dirs config field to BuiltinToolConfig. ReadFileTool now
rejects reads outside the configured base directories after canonicalize(),
blocking both path traversal and symlink attacks. Logs a warning when
allowed_dirs is empty. Threads config through register_all so constructors
receive per-tool settings.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 06:33:47 -05:00
whit3rabbit d2a56c5ad2 feat(proxy): wire batch_engine into proxy crate
- Add anyllm_batch_engine dependency to proxy Cargo.toml
- Add batch_engine field to AppState (Option<Arc<BatchEngine<...>>>)
- Add POST /v1/batches/{batch_id}/cancel route and handler
- cancel_batch handler uses BatchEngine when available, falls back to not_implemented
2026-03-31 06:30:25 -05:00
whit3rabbit 2b885a0938 feat(batch_engine): scaffold crate with core types, queue, file store, webhook, and engine facade
- BatchId, ItemId, BatchJob, BatchItem, RequestCounts, BatchStatus, ExecutionMode
- EngineError, QueueError types
- JSONL validation migrated from proxy
- SQLite schema init with migration from old tables
- FileStore for batch file storage
- JobQueue trait + SqliteQueue with full job lifecycle
- WebhookQueue trait + SqliteWebhookQueue with durable delivery
- WebhookDispatcher background loop with HMAC signing
- BatchEngine facade: submit, get, list, cancel, get_items
- 32 tests passing, clippy clean
2026-03-31 06:24:49 -05:00
whit3rabbit 3c1571473b chore: scaffold batch_engine crate directory structure 2026-03-31 06:13:35 -05:00
whit3rabbitandClaude Opus 4.6 cf417c9cfb fix(tools): share reqwest client and add SSRF validation for MCP URLs
McpServerManager now holds a shared reqwest::Client (built once in new()).
call_tool uses self.client instead of creating a new client per call.
discover_tools_impl extracted as a free fn; both the instance method
(discover_tools_with_client) and the static fallback delegate to it.

SSRF protection added at both registration points:
- admin add_mcp_server endpoint: validate_base_url() before calling discover
- main.rs startup: skip and log any MCP server URL that fails SSRF check

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:47:56 -05:00
whit3rabbitandClaude Opus 4.6 a5b095f988 fix(tools): handle Deny policy as error ToolResult instead of silent PassThrough
partition_tool_calls now returns three buckets (auto_execute, pass_through,
denied). Denied tools generate ToolResult entries with is_error:true and a
"denied by policy" message, which are fed back to the LLM in a follow-up
turn so it can see the rejection. Updated all callers (maybe_execute_tools,
streaming handler in chat_completions.rs) and added unit + integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:47:46 -05:00
whit3rabbitandClaude Sonnet 4.6 a88a48b50e fix: wire tool engine and MCP manager into main.rs from config
ToolEngineState and McpServerManager were hardcoded to None. Now:
- SimpleParsed carries a ToolStartupConfig with the three tool sections
- LoadResult exposes that config to main.rs
- main.rs constructs ToolEngineState (registry, policy, loop config) when
  any tool section is present; MCP servers are discovered async at startup
  with a warning on failure (no panic)
- tool_engine and mcp_manager on SharedState are populated from the same
  Arc so both proxy handlers and admin API share the same instance

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:35:44 -05:00
whit3rabbit d9c02c5204 docs: add tool execution engine and MCP integration to CLAUDE.md 2026-03-30 22:25:15 -05:00
whit3rabbitandClaude Sonnet 4.6 d7082b8812 test: add integration tests for tool execution engine
Covers partition_tool_calls, execute_tool_calls (success + ordering),
tool_results_to_user_message (success + error is_error flag),
is_duplicate, extract_tool_calls, and passthrough-policy behavior.
10 tests, all green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:24:14 -05:00
whit3rabbitandClaude Opus 4.6 a56bc95b5f feat(tools): add collect-then-execute streaming tool execution
Accumulates tool call fragments during the chat_completions streaming
loop, then after the initial stream completes, checks for auto-executable
tools via the ToolEngine. If found, executes tools in parallel, builds a
follow-up request with tool results, and streams the follow-up response
through the same SSE channel. Defers [DONE] until after any tool
execution follow-up completes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:21:37 -05:00
whit3rabbitandClaude Opus 4.6 4a6604f640 feat(tools): integrate non-streaming tool execution into messages handler
When tool_engine is configured, the non-streaming /v1/messages handler
now checks for tool_use blocks in the response, executes registered
tools, and makes a follow-up backend call with the results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:14:24 -05:00
whit3rabbitandClaude Opus 4.6 8675dec585 feat(tools): integrate non-streaming tool execution into chat completions handler
When tool_engine is configured, the non-streaming /v1/chat/completions
handler now checks for tool_use blocks in the response, executes registered
tools, and makes a follow-up backend call with the results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:14:20 -05:00
whit3rabbitandClaude Opus 4.6 a4119dd75c feat(tools): add helper functions for tool call extraction and result building
Adds extract_tool_calls, tool_results_to_user_message, and
response_to_assistant_message to execution.rs for use by non-streaming
handlers. Includes two tests for extract_tool_calls.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:14:16 -05:00
whit3rabbitandClaude Sonnet 4.6 3a62796a75 feat(tools): wire ToolEngineState into AppState with config-driven setup
Add ToolEngineState struct (registry, policy, loop_config, mcp_manager)
and tool_engine field to AppState. Update app_multi_with_shared signature
to accept the new parameter; all callers pass None until config-driven
wiring is implemented in a future task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:07:23 -05:00
whit3rabbitandClaude Sonnet 4.6 0253b47819 feat(admin): add MCP server management endpoints (list, add, remove)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 22:03:55 -05:00
whit3rabbitandClaude Opus 4.6 66864d8198 docs: Phase 1 implementation plan for batch orchestration engine
12 tasks covering: crate scaffold, core types, error types, JSONL
validation migration, DB schema + file storage, JobQueue trait +
SqliteQueue, webhook queue + dispatcher, BatchEngine facade, build
verification, proxy wiring, integration tests, and final verification.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 22:02:09 -05:00
whit3rabbitandClaude Sonnet 4.6 254ed8d9b6 feat(tools): add McpToolAdapter bridging MCP tools to Tool trait
Implements the Tool trait on McpToolAdapter, delegating execute() to
McpServerManager::call_tool(). Adds register_mcp_tools() to bulk-register
all discovered MCP tools into a ToolRegistry.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:59:47 -05:00
whit3rabbitandClaude Sonnet 4.6 3e7fe0633e feat(tools): add McpServerManager with tool discovery and JSON-RPC execution
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:58:23 -05:00
whit3rabbitandClaude Sonnet 4.6 412501e66f feat(config): add build_tool_config() to convert YAML to policy + loop config
Adds SimpleConfig::build_tool_config() that converts builtin_tools and
mcp_servers entries into a ToolExecutionPolicy (with exact-match and
glob rules) and converts tool_execution into a LoopConfig with sensible
defaults. Two tests cover both paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:55:37 -05:00
whit3rabbitandClaude Opus 4.6 2d04a2e95f docs: apply review feedback to batch orchestration spec
- dispatch() is now async (webhook enqueue is async)
- All event emission routed through NotificationManager, not direct
  event_bus.emit() calls
- Native batch poller emits Started/Progress events during polling
- SSE handler sends snapshot event on connect for late subscribers
- WebhookQueue trait gets reclaim_expired_leases() for stuck deliveries
- WebhookDispatcher runs lease reclaim loop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 21:52:47 -05:00
whit3rabbitandClaude Sonnet 4.6 c505012de0 feat(config): add tool_execution, builtin_tools, and mcp_servers config sections
Adds three new optional top-level fields to SimpleConfig for upcoming
tool execution support: ToolExecutionConfig (loop limits/timeouts),
BuiltinToolConfig (per-tool enable/policy/timeout), and McpServerConfig
(name/url/policy). Two unit tests verify both present and absent cases.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:52:10 -05:00
whit3rabbitandClaude Opus 4.6 8378065ca0 docs: batch orchestration engine design spec
Unified batch orchestration layer with hybrid execution (provider-native
delegation + proxy-native item processing), SQLite-backed job queue,
in-process workers with graceful shutdown, and event-driven notifications
(webhooks + SSE). New batch_engine crate with enforced boundaries for
future worker extraction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 21:50:08 -05:00
whit3rabbitandClaude Sonnet 4.6 8ab14cc2fa feat(tools): add ToolExecutionEngine core with partition and parallel execution
Implements ToolCall/ToolResult types, LoopConfig, partition_tool_calls(),
execute_tool_calls() (parallel via JoinSet with per-tool timeout), and
is_duplicate(). Adds list_names() to ToolRegistry. All 6 unit tests pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:49:41 -05:00
whit3rabbitandClaude Sonnet 4.6 e647b5349c feat(tools): add LoopTrace, ToolOutcome, and TerminationReason types
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:47:28 -05:00
whit3rabbitandClaude Sonnet 4.6 792570c18a feat(tools): add ToolExecutionPolicy with Allow/Deny/PassThrough actions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 21:45:39 -05:00
whit3rabbitandClaude Opus 4.6 6ceee1edd0 docs: add tool execution engine implementation plan
14 tasks covering: policy types, trace types, execution engine core,
config parsing, MCP server manager, MCP admin endpoints, handler
integration (non-streaming + streaming), and integration tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 21:42:39 -05:00
whit3rabbitandClaude Opus 4.6 9e1e769c4b fix: harden builtin tools and remove redundant error classification
BashTool: add 30s subprocess timeout and 256KB output truncation.
ReadFileTool: require absolute paths, canonicalize to block traversal,
enforce 1MB size limit. ToolRegistry::get returns &dyn Tool instead of
&Box<dyn Tool>. Remove wasted infer_error_kind call from log_entry()
since every error path overwrites via set_backend_error_kind(). Cancel
pending observability refresh timer in loadDashboard to prevent
duplicate concurrent fetches.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 21:38:54 -05:00
whit3rabbitandClaude Opus 4.6 35178f5a58 docs: add tool execution engine design spec
Covers bounded execution loop, tool execution policy, MCP integration
(SSE transport), failure modeling, loop trace observability, and config.
Phase C (single-round, max_iterations=1) with clean upgrade path to
Phase B (multi-round agentic loop).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:33:03 -05:00
whit3rabbitandClaude Opus 4.6 b6277a8b86 feat: operator observability dashboard with error classification
- Admin UI: operator view with request volume, token usage, latency,
  cost charts, failure table, and request timeline
- Backend: add error_kind() method and infer_error_kind() for stable
  error classification (rate_limit, timeout, backend_error, client_error)
- Admin DB: error_kind column in request_log, observability aggregate
  queries (bucketed timeseries, failure breakdown, timeline)
- Gemini: improved streaming translation, thinking block support,
  grounding metadata passthrough
- Request timeout: configurable REQUEST_TIMEOUT_SECS with streaming
  watchdog
- Model pricing: MODEL_PRICING_FILE for external pricing overrides
- Degradation header: ANYLLM_DEGRADATION_WARNINGS env var control

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-30 19:12:03 -05:00
whit3rabbit e053ac8bbc Merge branch 'feat/model-routing-ux' 2026-03-30 18:24:21 -05:00
whit3rabbitandClaude Sonnet 4.6 1ea64a9973 merge: feat/simple-advanced-modes into main; resolve routes.rs conflict
Merge adds expose_degradation_warnings alongside stream_timeout_secs in
AppState. Also fixes pre-existing borrow error in admin/routes.rs (until
moved into closure then used in json response).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 18:12:36 -05:00
whit3rabbit cca8f2740a ci: add nightly workflow for live OpenAI and Bedrock integration tests 2026-03-30 17:23:14 -05:00
whit3rabbit 85d5364810 docs: create compatibility matrix 2026-03-30 17:22:32 -05:00
whit3rabbitandClaude Sonnet 4.6 d510a2c09f test: add boundary tests for MAX_TOOL_CALL_INDEX cap in streaming_map
Verifies that idx == MAX_TOOL_CALL_INDEX (128) is accepted (> not >=)
and idx == MAX_TOOL_CALL_INDEX + 1 (129) is silently dropped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 17:21:05 -05:00
whit3rabbit 35d6489913 fix(config): clearer error for invalid YAML format; document Bedrock region convention 2026-03-30 17:20:18 -05:00
whit3rabbit 810a436b96 feat: add MODEL_PRICING_FILE env var to override embedded model pricing at startup 2026-03-30 17:18:47 -05:00
whit3rabbitandClaude Sonnet 4.6 bccddd9a7e fix: apply stream_timeout_secs to chat_completions streaming path
Captures stream_timeout_secs from AppState before the tokio::spawn and
wraps the byte-stream loop in tokio::time::timeout, mirroring the same
guard already present in streaming.rs. A stalled backend on
/v1/chat/completions no longer holds the connection indefinitely.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 17:17:19 -05:00
whit3rabbit ae10a851e6 docs: document simple YAML config format in CLAUDE.md 2026-03-30 17:17:14 -05:00
whit3rabbit 8375efd272 chore(config/simple): verify default_base_url uses owned Strings (no Box::leak) 2026-03-30 17:16:17 -05:00
whit3rabbit 81fc2dc2aa docs(README): add simple/advanced mode split; gate advanced features under heading 2026-03-30 17:15:56 -05:00