Commit Graph
325 Commits
Author SHA1 Message Date
whit3rabbit bb2d0e6bb3 fix(proxy): enforce virtual-key model allowlist for batch create (#16) 2026-05-23 15:21:09 -05:00
whit3rabbit 85477cee8b fix: enforce virtual-key model allowlist on model-selecting passthrough routes (#15)
* fix(proxy): enforce virtual key model allowlist on passthrough endpoints

* fix: enforce virtual key policy on model passthroughs
2026-05-23 15:20:58 -05:00
whit3rabbit 7161f2b28b fix(proxy): enforce virtual-key TPM when Redis rate limiter is enabled (#14)
* fix(proxy): record virtual-key TPM usage in redis limiter

* fix: keep redis TPM key only in redis builds
2026-05-23 15:20:44 -05:00
whit3rabbit c29b523554 fix: reject unknown models when model router is active (#13)
* fix: reject unknown models when model router is active

* fix: use existing invalid request error for model router
2026-05-23 15:20:29 -05:00
whit3rabbit 98bc1c833a fix: include pricing asset in proxy crate v0.9.4 2026-05-23 15:03:09 -05:00
whit3rabbit 4cdfef82e3 chore: prepare 0.9.3 release v0.9.3 2026-05-23 12:28:08 -05:00
whit3rabbit 1f300d8566 Fix virtual-key accounting across generation routes 2026-05-23 12:22:39 -05:00
whit3rabbit 158815acbd Block virtual-key passthrough bypass 2026-05-23 12:22:39 -05:00
whit3rabbit 025d992ca3 Add in-process chat completion runtime 2026-05-23 12:22:39 -05:00
github-actions[bot] fea7a8e9cd bot: update model pricing from LiteLLM [skip ci] 2026-05-18 07:11:57 +00:00
github-actions[bot] 5618e975e9 bot: update model pricing from LiteLLM [skip ci] 2026-05-11 07:00:50 +00:00
github-actions[bot] 856159684c bot: update model pricing from LiteLLM [skip ci] 2026-04-27 06:45:30 +00:00
github-actions[bot] 69929e494b bot: update model pricing from LiteLLM [skip ci] 2026-04-20 06:37:28 +00:00
whit3rabbitandClaude Opus 4.7 3d62da3a68 chore: fix Rust 1.95 clippy lints
CI runs stable (now 1.95) which surfaces two new lints my 1.94 local
check missed:

- collapsible_match in reverse_message_map.rs: collapse the empty
  guard branch into a match arm with a guard.
- manual_checked_ops in admin/db.rs: replace the explicit zero check
  with checked_div(...).unwrap_or(0).
- collapsible_match in admin/ws.rs: clippy's suggested guard form
  fails to compile because Bytes can't be moved in a pattern guard.
  Apply a targeted #[allow] with a comment explaining why.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
v0.9.2
2026-04-18 19:41:46 -05:00
whit3rabbitandClaude Opus 4.7 4cc854f6ef ci: clean up publish job for tag-triggered crates.io release
Drop job-level continue-on-error: true (was masking real failures since
each step already handles exit 101 inline). Remove the duplicate
anyllm_batch_engine step that lacked the exit-101 swallow. Canonical
publish order is now: translate -> providers -> client -> batch_engine
-> proxy with 30s sleeps for index propagation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 19:26:36 -05:00
whit3rabbitandClaude Opus 4.7 6de126893f chore: fix all clippy lints across workspace
- tools_map: use std::slice::from_ref over &[clone()] in 4 tests
- gemini_streaming_map: rewrite match as matches!
- backend/mod.rs: move impl BackendClient before #[cfg(test)] mod tests
- middleware: replace .filter().last() with .rfind() (xff parsing)
- middleware: drop dead `|| true` in is_ip_allowed smoke test
- sse: drop blank line between doc comment and assert_sse_ok
- streaming example: collapse nested if-let into outer match arms
- tool_execution tests: array literal over vec! for one-off slices
- live_bedrock: use is_some_and instead of map_or(false, _)
- live_api / live_responses: contains() over iter().any() on &[&str]

All test-only / example changes; no production behavior change.
1130 tests pass, fmt clean, clippy -D warnings clean.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 19:11:35 -05:00
whit3rabbitandClaude Opus 4.7 25f37dd9a4 docs(crates): add READMEs for client, batch_engine, proxy; bump 0.9.2
Each new README places the crate inside the anyllm-proxy workspace and
includes verified library-use examples for downstream consumers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 19:04:38 -05:00
whit3rabbitandClaude Opus 4.7 857f6e81c5 docs(providers): add README covering purpose, library use, and catalog
Documents the anyllm_providers crate as a pure compile-time metadata
catalog within the anyllm-proxy workspace, with a TUI integration walk-
through defaulting to Ollama and LM Studio and a categorized list of
all 90 registered providers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-18 18:49:08 -05:00
whit3rabbitandClaude Opus 4.6 58bd323242 fix(deps): bump rustls-webpki 0.103.10 -> 0.103.12, version 0.9.1
Fixes RUSTSEC-2026-0098 (URI name constraint bypass) and
RUSTSEC-2026-0099 (wildcard name constraint bypass).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v0.9.1
2026-04-15 20:36:31 -05:00
whit3rabbitandClaude Opus 4.6 d28226a6b2 feat(admin-ui): redesign providers tab with tiered tile grid and detail panel
Replace modal-based add-provider flow with a tile grid grouped by popularity
tier (top, popular, notable, more). Clicking a tile opens a detail panel
overlay showing capabilities, models, configured backends, and inline add-key
form. Extracted tier grouping logic into providerTiers.ts utility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 20:33:14 -05:00
whit3rabbitandClaude Sonnet 4.6 f583a1bc19 feat(providers): expand provider catalog with models and capabilities
Populate model definitions and refine capability flags across provider
stubs, adding context windows, output limits, and per-model capabilities
for chat, tool use, vision, and streaming where applicable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 20:05:20 -05:00
whit3rabbitandClaude Sonnet 4.6 de8c534107 fix: allow dead_code on allowed_routes pending enforcement wiring
Field is populated from the DB but scoping enforcement is not yet wired,
so clippy flags it. Keep the field to avoid reshuffling VirtualKeyContext
when enforcement lands.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 18:58:19 -05:00
whit3rabbitandClaude Sonnet 4.6 2ff669a10b style: apply cargo fmt
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 18:56:48 -05:00
whit3rabbitandClaude Sonnet 4.6 9ab1166e81 feat: admin UI redesign with sidebar nav, routes tab, and split rate limiters
- Replace top nav with sidebar layout; add shared Modal, Toast, ConfirmDialog, AsyncBoundary components
- Add Routes tab with full CRUD UI backed by new routes_api admin endpoints
- Split admin rate limiter into separate read/write buckets with method-aware middleware
- Add allowed_routes scoping to virtual keys

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 18:15:06 -05:00
whit3rabbit 2706a9d63f Merge pull request #4 from whit3rabbit/feat/add-providers-catalog
feat: add 16 new provider stubs and providers catalog UI tab
2026-04-13 17:43:12 -05:00
whit3rabbitandClaude Sonnet 4.6 816b64ae6a feat: add 16 new provider stubs and providers catalog UI tab
New providers: assemblyai, baidu, blackboxai, brave, cartesia, deepgram,
elevenlabs, exa, iflytek, lmsys, playht, pollinations, serper, siliconflow,
stability, tavily. Registered in mod.rs and registry.rs.

Adds a Providers tab to the admin UI with a ProviderIcon component and
catalog route. Updates backend, streaming, config, and test infrastructure
to align with the expanded provider set. Bumps all crate versions to 0.9.0.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 17:42:46 -05:00
whit3rabbit 96dc118bba docs: add version bumping gotchas to CLAUDE.md 2026-04-09 15:22:14 -05:00
whit3rabbitandClaude Sonnet 4.6 22e4ee73aa chore: bump workspace version to 0.9.0 to match release tag
Aligns crate versions with the GitHub release tag so the deb package
filename, crates.io versions, and release tag all agree.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v0.9.0
2026-04-09 15:05:20 -05:00
whit3rabbitandClaude Sonnet 4.6 98fdefe4c3 docs: fix crates.io badge (use anyllm_translate) and deb URL version
- anyllm_proxy not yet on crates.io; badge now points to anyllm_translate
- deb package version is 0.2.0-1 (Cargo workspace), not 0.9.0-1 (release tag)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 15:01:24 -05:00
whit3rabbitandClaude Sonnet 4.6 c2efd77c28 docs: add CI/crates badges, improve install section, update CLAUDE.md CI gotchas
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:47:58 -05:00
whit3rabbitandClaude Sonnet 4.6 422175b1df docs: add macOS Homebrew and Linux deb install instructions
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:40:27 -05:00
whit3rabbitandClaude Sonnet 4.6 02a9bb91d8 fix: handle crates.io version-already-exists (exit 101) in publish job
- Add continue-on-error: true to publish job (non-blocking for release)
- Add exit code 101 handling per step: already-published = success
- Add missing anyllm_batch_engine publish step

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:38:37 -05:00
whit3rabbitandClaude Sonnet 4.6 32272f2420 fix: add create-release job and complete crates.io publish sequence
- Add create-release job before upload jobs (gh release upload requires
  the release to exist first)
- Add anyllm_providers and anyllm_batch_engine to publish sequence;
  proxy depends on both, missing them caused publish to fail
- release-assets and brew-release now depend on create-release

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:29:00 -05:00
whit3rabbitandClaude Sonnet 4.6 98ff681ae5 fix: add --legacy-peer-deps to Docker npm ci for vite 8 compat
@vitejs/plugin-react@4.7.0 peer deps don't declare vite 8 support yet.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 14:01:53 -05:00
whit3rabbitandClaude Sonnet 4.6 ccaa4fd32b fix: correct GitHub Actions secrets context usage in if conditions
- secrets context is not allowed in job/step-level if conditions;
  move checks into shell (exit 0 if secret unset)
- Fixes YAML/semantic error causing 0s workflow file issue failures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:53:34 -05:00
whit3rabbitandClaude Sonnet 4.6 d3474792c8 fix: resolve CI YAML parse error and vite 8 peer dep conflict
- Replace heredoc in brew-release job with printf statements to fix YAML
  parse error (heredoc content at col 0 broke YAML literal block scalar)
- Add --legacy-peer-deps to npm ci: @vitejs/plugin-react@4.7.0 peer deps
  only declare vite ^4-7 but is runtime-compatible with vite 8

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:35:55 -05:00
whit3rabbitandClaude Sonnet 4.6 5d9ce74316 chore: fmt and doc cleanup for v0.9.0
- cargo fmt formatting fixes across admin and config modules
- README: add OpenRouter dedicated provider key example, clarify allowed_models wildcard syntax

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 13:29:29 -05:00
whit3rabbitandClaude Sonnet 4.6 e94b4e2acf feat: add Homebrew tap release job and ignore tool/config dirs
Add brew-release CI job that builds versioned macOS tarballs, uploads
them to the GitHub Release, and updates the homebrew-tap cask formula.
Ignore .mcp.json, .repowise/, and .superpowers/ in .gitignore. Add
Makefile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 03:25:40 -05:00
whit3rabbitandClaude Opus 4.6 af9a87ec6e fix: sanitize API keys and strip /v1 suffix from base URLs
Add sanitize_api_key() to strip curly/smart quotes silently injected by
copy-paste from rich-text sources (Slack, docs). Add strip_v1_suffix()
to prevent doubled /v1/v1 paths when provider URLs already include /v1.
Applied across all config paths (env, simple YAML, LiteLLM YAML, TOML).
Also adds crate structure section to proxy-architecture.md and rebuilds
admin UI dist after vite upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 20:20:32 -05:00
whit3rabbitandClaude Sonnet 4.6 0dd3ef98f6 fix: upgrade vite to 8.0.7 to address CVE path traversal and arbitrary file read
Fixes Dependabot alerts #2 (high) and #3 (medium): vite arbitrary file read
via dev server WebSocket and path traversal in optimized deps .map handling.
Patched version required >= 6.4.2; upgraded to 8.0.7. Build verified.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 16:38:40 -05:00
whit3rabbitandClaude Sonnet 4.6 545bdbbbd3 feat: add bedrock native passthrough, generic passthrough, provider docs, and managed backend fixes
Adds bedrock_native.rs (Converse/InvokeModel with SigV4) and
generic_passthrough.rs catch-all for Translate mode. Adds comprehensive
provider reference docs (docs/providers/, docs/ENDPOINTS.md). Fixes
managed backend admin UI (BackendForm, ManagedBackendsSection) and
admin route/model handler issues. Adds automated model pricing update
workflow (scripts/update_pricing.py, .github/workflows/update-pricing.yml).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 16:36:16 -05:00
whit3rabbitandClaude Sonnet 4.6 be038e4164 feat: add provider catalog, bedrock native passthrough, and generic passthrough
Introduces anyllm_providers crate as metadata-only catalog (ProviderDef,
ModelDef, registry). Wires any provider-id as BACKEND via OpenAIClient.
Adds bedrock_native.rs (Converse/InvokeModel with SigV4) and
generic_passthrough.rs catch-all for Translate mode. Updates config,
backend clients, streaming, token counting, and compatibility tests
to support the expanded provider surface. Updates model pricing and docs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 16:33:59 -05:00
whit3rabbitandClaude Sonnet 4.6 c0a49d2eab fix: stale updated_at in PUT response, add rpm/tpm hint, fix type safety and comments
- managed_backends.rs: stamp updated_row.updated_at after SQLite write succeeds,
  before inserting into in-memory map, so the PUT response reflects a post-update
  timestamp rather than the pre-update value
- providerFields.ts: add hint to rpm/tpm FieldDef entries noting they are stored
  for reference and not enforced on managed backends
- queries.ts: type useCatalogProviders queryFn with explicit generic so the
  server-guaranteed shape { providers: CatalogProvider[] } is reflected at compile
  time; removes the ?? r fallback
- state.rs: correct managed_backends field comment from UUID key to name key

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:59:39 -05:00
whit3rabbit 1cebefa3e2 fix: use axum v0.7 path syntax {id} for catalog route 2026-04-08 13:50:55 -05:00
whit3rabbitandClaude Sonnet 4.6 4ed09158f4 fix: use display_name in provider table, fix stale provider state in BackendForm
Fix 1: ManagedBackendsSection.tsx line 24 now uses p.display_name instead of
p.name, matching what the Rust catalog handler serializes.

Fix 2: BackendForm.tsx now uses useEffect to initialize providerId when providers
load, eliminating the split identity between state ('') and compensation logic
('value={providerId || providers[0]?.id ?? ''}'). The effect sets providerId to
the first provider when the list first populates and no initial provider is set.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:48:59 -05:00
whit3rabbitandClaude Sonnet 4.6 ca7b27cfc6 fix: add catalog routes, complete CatalogProvider type, group providers by status
- Copy catalog.rs from main; add pub mod catalog + GET routes in admin router
- Add price_per_million_for_model to cost module (missing from this branch)
- Expand CatalogProvider with display_name, status, capabilities, model_count
- BackendForm provider dropdown: use display_name, group by status via optgroup

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:45:57 -05:00
whit3rabbitandClaude Sonnet 4.6 7a3587682c feat: add managed backends UI components
Add BackendForm and ManagedBackendsSection for creating/editing managed
backends at runtime. Integrate ManagedBackendsSection as first section in
Settings tab. Extend Models form with optional backend_name datalist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:41:55 -05:00
whit3rabbitandClaude Sonnet 4.6 d8d34538ca fix: add ManagedBackendsResponse type and useCatalogProviders hook
- Add ManagedBackendsResponse interface to types.ts matching the /admin/api/backends/managed endpoint response shape
- Update useManagedBackends return type from inline { backends: ManagedBackend[] } to ManagedBackendsResponse for reusability
- Add useCatalogProviders hook to queries.ts following existing list hook patterns, handles both { providers: [...] } and bare array responses from /admin/api/catalog/providers

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:37:14 -05:00
whit3rabbitandClaude Sonnet 4.6 887661f6dc feat: add managed backend types, queries, and provider field schema
- Add CatalogProvider, ManagedBackend, CreateManagedBackendRequest, UpdateManagedBackendRequest to api/types.ts
- Add useManagedBackends, useCreateManagedBackend, useUpdateManagedBackend, useDeleteManagedBackend hooks to api/queries.ts
- Create utils/providerFields.ts with getProviderFields() mapping protocol+auth to FieldDef[]

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:32:21 -05:00
whit3rabbitandClaude Sonnet 4.6 0ecd5aeab6 fix: remove updated_at divergence in update handler, add audit detail
Fix 1: Remove manual updated_at assignment from in-memory state in update
handler. The db::update_managed_backend function independently generates a
new timestamp, so we avoid divergence between in-memory and SQLite. Add
comment explaining the pre-update state.

Fix 2: Add audit detail for update handler showing provider_id, matching
pattern used in create handler.

Fix 3: Add comment to ManagedBackendPatch struct documenting that Option<T>
fields offer no way to clear/NULL a field, guiding UI behavior.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-08 13:30:26 -05:00