Commit Graph
31 Commits
Author SHA1 Message Date
whit3rabbit ebf5455b10 docs: simplify README, add docs/CLI.md, center ASCII art and add WebUI screenshots, and add copy-command to Auto Router 2026-07-16 20:41:24 -05:00
whit3rabbitandClaude 75a5a3a230 chore: release v0.16.0
Bump workspace + all inter-crate version refs from 0.15.1 to 0.16.0
(workspace Cargo.toml, anyllm_client pinned version, batch_engine,
proxy's 8 internal deps, optimizer core/passes/scorer + cli + benches).
Refresh Cargo.lock. Move CHANGELOG [Unreleased] -> [0.16.0] - 2026-07-16
and add a fresh empty [Unreleased]; bump the README deb filename to
anyllm-proxy_0.16.0-1_amd64.deb and add the v0.16.0 compare link.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-16 20:12:53 -05:00
whit3rabbitandClaude Opus 4.8 063af0d423 fix: admin UI provider model cache wipe, chip removal, stale-backend edit (v0.15.1)
- discover/refresh no longer wipe provider_models_cache on an empty result
  (DELETE-then-INSERT upsert skipped when zero ids), preserving autorouter
  model suggestions.
- Provider editor: removed model chips stick; persisted names seeded once
  instead of re-unioned on every cache refetch.
- Managed-backend edit falls back to SQLite on an in-memory miss, so a row
  skipped at startup no longer 404s on every save (heals value-fixable
  configs; unknown provider_id still 400s).
- Add get_managed_backend DB helper + round-trip test.
- Bump workspace to 0.15.1; CHANGELOG + README deb filename.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 21:50:42 -05:00
whit3rabbitandClaude Opus 4.8 3dc4f8d491 chore: release v0.15.0
Bump workspace + all crate versions 0.14.1 -> 0.15.0, move CHANGELOG
[Unreleased] into [0.15.0], backfill compare links, update README deb URL.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:59:27 -05:00
whit3rabbitandClaude Opus 4.8 e28a28f099 docs: changelog + README for router, dashboard sparklines, settings help
- CHANGELOG: router, dashboard sparklines/trend deltas, settings help
  grouping, module split, Makefile targets.
- README: dashboard sparkline note + new Routing tab description.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:51:15 -05:00
whit3rabbitandClaude Opus 4.8 e96ec18b24 feat: admin UI on by bare launch + tier-router review fixes
Admin/browser:
- Enable admin UI on bare (zero-arg) launch; auto-open default browser
  (main_helpers::bootstrap::admin_enabled / is_default_launch, browser.rs).
- Guard Docker (docker-entrypoint.sh) and systemd (packaging/anyllm-proxy.service)
  so headless server installs keep admin opt-in (DISABLE_ADMIN default off).

Claude Code tier router fixes (from code review):
- put.rs: validate only *enabled* tiers, and accept statically-configured
  (all_backends) targets via new SharedState.static_backends, not just managed.
- openai_signals: drop historical reasoning_content check so a plain follow-up
  in a reasoning conversation isn't misrouted to the Think tier.
- resolve_router_tier: warn! on fail-open when an active tier's backend is
  unknown instead of silently bypassing the router.

Tests: static-config backend acceptance; existing router coverage still green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 20:38:25 -05:00
whit3rabbitandClaude Opus 4.8 d12309d701 chore: release 0.14.1
Re-cut of the 0.14.0 release. The v0.14.0 tag was pushed before the CI
publish job knew about the new leaf crates (pxpipe, rtk, optimize_*), so
its pipeline would have failed to publish anyllm_proxy to crates.io. The
publish-order fix landed on main after the tag, and a tag can't be moved
under auto mode, so the release is re-cut as 0.14.1 with the fix in place.

Bump 0.14.0 -> 0.14.1 (workspace, inter-crate deps, client pinned version,
README deb filename) and rename the CHANGELOG section (0.14.0 was never
released).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 21:36:24 -05:00
whit3rabbitandClaude Opus 4.8 ff03a7ebd7 chore: release 0.14.0
Move Unreleased changelog entries (RTK compression, opt-in prompt
optimizer, release binary archives, Bedrock allowlist fix, model
discovery fix) into the 0.14.0 section. Bump workspace + inter-crate
versions 0.13.0 -> 0.14.0 and the README deb filename.

Fix clippy missing_const_for_thread_local in optimize-core alloc_budget
test (const-init the thread-locals) so cargo clippy -D warnings is clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 21:21:14 -05:00
whit3rabbitandClaude Opus 4.8 c2e77c59c7 chore: release 0.13.0
Bump workspace + inter-crate versions 0.12.0 -> 0.13.0, README deb filename,
and move CHANGELOG [Unreleased] into [0.13.0] - 2026-07-12.

Also bump crossbeam-epoch 0.9.18 -> 0.9.20 (Cargo.lock) to clear
RUSTSEC-2026-0204 (invalid pointer dereference), which was failing the
CI Security audit step.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 08:00:28 -05:00
whit3rabbitandClaude Opus 4.8 b89e0bd359 feat: hyphenated binary name, TTY-aware logs, one-click admin login
- Emit binary as `anyllm-proxy` (hyphen) via explicit [[bin]]; package/crate
  name stays `anyllm_proxy`. Threads one hyphenated name through deb/brew/
  release archives and CI; drops the rename-in-archive shim.
- Terminal logs default to human-readable on a TTY, JSON when piped
  (Docker/systemd). Override with LOG_FORMAT=json|text.
- On loopback admin binds, startup banner prints a tokenized `?token=` admin
  URL plus a bare Token line; SPA reads `?token=` to auto-login then strips it
  from the address bar. Omitted on non-loopback binds to avoid token leak.
- `--webui`/`--admin` with no backend prints a one-line hint instead of the
  full backend cheat-sheet (UI already covers it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 15:35:06 -05:00
whit3rabbitandClaude Opus 4.8 88e9ccad4b docs: bump README deb version to 0.12.0, add README step to release process
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 14:05:49 -05:00
whit3rabbitandClaude Sonnet 5 a87cae6ca4 feat: ANTHROPIC_FORWARD_CLIENT_AUTH with live admin-UI toggle
Adds opt-in client-credential forwarding for Anthropic passthrough
(single-key/BYOK deployments), plus fixes found in review:

- Startup safeguard now shares one check (server/middleware/auth.rs)
  with the live admin PUT /admin/api/config path, closing a bypass
  where PROXY_OPEN_RELAY=true alongside 2+ PROXY_API_KEYS entries
  slipped past the old startup-only check.
- x-goog-api-key is now recognized as a forwardable credential
  (renamed to x-api-key upstream, since Anthropic doesn't understand
  that header name), matching validate_auth's precedence.
- Managed (admin-API) backends no longer carry a dead
  forward_client_auth field that could never take effect.
- ClientAuthPath forwarding decisions are now double-checked against
  live VirtualKeyContext/JwtClaims presence, not just the enum, to
  fail closed if the two ever desync.
- Moved from a per-backend BackendConfig field to a global
  RuntimeConfig field (like anthropic_thinking_repair), making it
  live-toggleable from the admin UI with no restart, and uniform
  across every Anthropic-kind backend in a multi-backend deployment.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-04 15:40:53 -05:00
whit3rabbitandClaude Opus 4.8 ee6a39d5d4 fix: streaming/cost/cache correctness, litellm compat, provider docs
- client/proxy SSE + streaming loop cleanups (sse.rs, streaming.rs,
  runtime/stream.rs, chat_completions stream/anthropic/generic, gemini_input)
- cost + cache module fixes with added tests
- provider catalog/registry adjustments
- litellm YAML config support + tests; single-config tweaks
- docs: scope Vertex auth to VERTEX_API_KEY/GOOGLE_ACCESS_TOKEN,
  Bedrock LiteLLM YAML limits, Docker Hub image/tag docs in README
- CI: docker workflow multi-arch tag handling

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 12:24:07 -05:00
whit3rabbitandClaude Sonnet 4.6 22e4ee73aa chore: bump workspace version to 0.9.0 to match release tag
Aligns crate versions with the GitHub release tag so the deb package
filename, crates.io versions, and release tag all agree.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:05:20 -05:00
whit3rabbitandClaude Sonnet 4.6 98fdefe4c3 docs: fix crates.io badge (use anyllm_translate) and deb URL version
- anyllm_proxy not yet on crates.io; badge now points to anyllm_translate
- deb package version is 0.2.0-1 (Cargo workspace), not 0.9.0-1 (release tag)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:01:24 -05:00
whit3rabbitandClaude Sonnet 4.6 c2efd77c28 docs: add CI/crates badges, improve install section, update CLAUDE.md CI gotchas
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:47:58 -05:00
whit3rabbitandClaude Sonnet 4.6 422175b1df docs: add macOS Homebrew and Linux deb install instructions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:40:27 -05:00
whit3rabbitandClaude Sonnet 4.6 5d9ce74316 chore: fmt and doc cleanup for v0.9.0
- cargo fmt formatting fixes across admin and config modules
- README: add OpenRouter dedicated provider key example, clarify allowed_models wildcard syntax

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:29:29 -05:00
whit3rabbitandClaude Sonnet 4.6 dc301c489e docs: restructure README for user-first experience
Move install methods, admin env vars, virtual key operations, Docker
smoke tests, LiteLLM aliases, and library usage into collapsible
<details> blocks. Consolidate backend examples into one section.
Remove numbered sections and redundant "What/Why/How" explainer.
Add run subcommand and OMIT_STREAM_OPTIONS tips.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:52:47 -05:00
whit3rabbitandClaude Sonnet 4.6 dde1f902b6 feat: model persistence, model discovery, config directory, and admin UI updates
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 21:24:44 -05:00
whit3rabbitandClaude Sonnet 4.6 0fa7495574 feat: docker smoke tests, admin UI polish, and misc fixes
- ci: replace inline smoke test with docker-compose.test.yml + scripts/docker-smoke-test.sh (9 checks)
- add .env.example, .env.example.test, .anyllm.env.example, docker-compose.test.yml
- gitignore: add .anyllm.env*, *.db-shm, *.db-wal patterns
- admin UI: fix auth store, vite config, App.tsx updates; rebuild dist
- admin routes: keys, traffic, mod, db, health_check updates
- server: gemini_input, mod, routes adjustments
- translator: lib.rs, gemini_message_map, tools_map fixes
- batch: anthropic_batch cleanup
- docs: add React admin UI plan

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 18:58:38 -05:00
whit3rabbitandClaude Sonnet 4.6 148ced3324 docs: add Claude Code env vars for proxy mode
Document ANTHROPIC_AUTH_TOKEN, ANTHROPIC_API_KEY="", and
CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1 alongside ANTHROPIC_BASE_URL
so users know the full set of vars needed to run Claude Code against the proxy.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 13:39:16 -05:00
whit3rabbit 81fc2dc2aa docs(README): add simple/advanced mode split; gate advanced features under heading 2026-03-30 17:15:56 -05:00
whit3rabbitandClaude Sonnet 4.6 4fbaf43011 fix(security): populate source_ip in all admin audit log entries
Add ConnectInfo<SocketAddr> extractor to all seven admin mutation
handlers (put_config, delete_config_override, create_key, update_key,
revoke_key, add_model, remove_model). Pass addr.ip().to_string() as
source_ip in each emit_audit call.

Update integration tests to use into_make_service_with_connect_info
and MockConnectInfo so the ConnectInfo extractor is satisfied in test
servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 06:38:05 -05:00
whit3rabbitandClaude Sonnet 4.6 3588f52ccb docs: expand library usage examples; borrow header slices in passthrough
Rewrote the "Using as a Library" README section: added cargo dependency
snippets, reorganized HTTP client/pure translation/reverse translation
examples, and added error handling + tool calling code blocks.

Changed AnthropicClient::forward/forward_stream to accept
`&[(&str, &str)]` instead of `&[(String, String)]` so the passthrough
handler can pass string literals directly without allocating.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 14:48:19 -05:00
whit3rabbitandClaude Opus 4.6 79d4768125 docs: add LiteLLM migration guide, env var alias table, and README section
Add "Coming from LiteLLM?" section to README with config.yaml example and
env var compatibility table. Add migration guide to COMPARISON_LITELLM.md
covering config file, env var aliases, secret syntax, and unsupported
features. Add AZURE_API_VERSION and AWS_REGION_NAME aliases. Document
PROXY_CONFIG and aliases in CLAUDE.md.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 05:44:41 -05:00
whit3rabbitandClaude Sonnet 4.6 0942072b29 chore: docs, CI, and polish for litellm gap fill
Update CLAUDE.md and README.md to document new backends (Azure, Bedrock),
virtual key management, rate limiting, OTEL, and client v0.2.0. Add CI
workflow for the new integration test suite. Final task-list completions
and virtual key integration tests.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 06:36:09 -05:00
whit3rabbitandClaude Sonnet 4.6 f1df50ff37 refactor: extract shared client crate and rename to anyllm_*
Introduce `anyllm_client` crate containing HTTP client construction,
SSRF-safe DNS resolver, retry/backoff logic, rate limit header parsing,
and SSE frame parsing. These were previously inlined in the proxy crate.

Rename crates from `anthropic_openai_proxy`/`anthropic_openai_translate`
to `anyllm_proxy`/`anyllm_translate` throughout.

proxy/backend: now re-exports retry, rate limit, and SSE symbols from
the client crate; `send_with_retry` and `build_http_client` are thin
adapters bridging BackendAuth/TlsConfig to the client crate's types.

streaming: remove duplicate `find_double_newline` and
`MAX_SSE_BUFFER_SIZE` definitions; import from `crate::backend` instead.

Fix missing `pub mod` declarations in translator and proxy that were
accidentally replaced by doc comments (streaming, usage_map, server,
redact).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-25 06:11:01 -05:00
whit3rabbitandClaude Opus 4.6 c607620c19 fix: local LLM compat, protocol headers, and doc cleanup
- Strip markdown code fences from tool call arguments (DeepSeek/Qwen)
- Add OMIT_STREAM_OPTIONS for backends that reject unknown fields
- Strip n/temperature/top_p for o-series models
- Add anthropic-version header to all responses
- Add x-token-count-warning header to count_tokens endpoint
- Rewrite README for clarity; remove stale PLAN.md, TASKS.md, CHANGELOG.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 21:24:57 -05:00
whit3rabbitandClaude Opus 4.6 a4e2458ecb docs: update README and CLAUDE.md for multi-backend and admin features
README: rewrite with use cases, Claude Code / local LLM / OpenRouter
examples, multi-backend TOML config, admin dashboard documentation,
and updated endpoint table.

CLAUDE.md: update test counts, add Anthropic backend, PROXY_API_KEYS,
admin module, and correct architecture references.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 19:34:56 -05:00
whit3rabbitandClaude Opus 4.6 d37d1e25f1 Add phases 12-20: release infra, transparent proxy, model mapping, mTLS, extended thinking, Gemini research
Phases 12-18: release infrastructure (LICENSE, README, Dockerfile, CI,
CHANGELOG), transparent proxy with anthropic-version/anthropic-beta header
passthrough and lossy translation warnings, BIG_MODEL/SMALL_MODEL env-based
model mapping, mTLS client cert support (P12/PEM), max_completion_tokens
and reasoning_effort passthrough via serde flatten, extended thinking type
support (thinking blocks stripped in translation), top_k typed field.

Phase 20: Gemini backend research with docs/gemini-api-diffs.md covering
native API format, tool calling, streaming, auth, schema restrictions,
and Vertex AI OpenAI-compatible endpoint. Task roadmap through Phase 22.

Test count: 169 -> expanded with new fixture and unit tests for all phases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-21 20:06:50 -05:00